Date of Release: September 1, 2022
Resolved Issues
JIRA ID | Issue | Severity |
|---|---|---|
IPD-18862 | Support required for fixed width source file type. | Highest |
IPD-18878 | Databricks token is visible in clear text in the job_object.json file from Infoworks job logs. | Highest |
IPD-18972 | Unable to connect to SFTP due to limited algorithm. | Highest |
IPD-18986 | Projection support needs to be added for Onboarding tables in Snowflake environment. | Highest |
IPD-19027 | The merge statement execution mode is not configurable between sync and async. | Medium |
IPD-19028 | UPDATE query is not supported for pipelines that are configured in Merge mode. | Medium |
IPD-19044 | Job status is incorrect even if job is completed successfully when Databricks cluster is pinned and unpinned on UI. | High |
IPD-17988 | API endpoint is not updating the Snowflake warehouse assigned to a pipeline. | High |
IPD-17954 | Logging issue while adding tables and crawl metadata API. | High |
IPD-18171 | Data Connection is not getting mapped while importing pipeline configuration for Snowflake target. | High |
IPD-18311 | Error occurred while adding the table due to mandatory | Highest |
IPD-18820 | Ingestion from ADLS Gov cloud is not supported. | Medium |
IPD-18695 | File Preview/Schema Crawl fails for CSV ingestion from S3 Buckets hosted on Gov Cloud. | High |
IPD-18805 | ADLS and WASB Datalakes are not Gov cloud aware. | Highest |
IPD-18804 | The Compute tab of Snowflake environment does not include a Gov Cloud checkbox for Databricks Regions. | Highest |
IPD-18623 | The DISTINCT function in SQL is not getting imported properly in Infoworks Pipelines | Highest |
IPD-18777 | Direct incremental ingestion to existing table on snowflake without any audit column is not allowed. | Highest |
IPD-18835 | The user managed table configuration could not enabled due to "is_table_user_managed" not being present. | Highest |
IPD-18911 | The REST API doesn’t support “HIVE_UDF” source extensions for columns. | Highest |
IPD-18921 | The "in_notin" node under pipelines does not support multiple columns. | Highest |
IPD-18931 | Infoworks metacrawl job for CSV source fetches incorrect number of columns. | Highest |
Internet-free Upgrade
For upgrading from 5.2.0 to 5.2.0.7, execute the following commands:
Step 1: SSH to the Infoworks VM and switch to Infoworks User.
Step 2: Initialize the Variables:
full_version=<Full Version> # Full Version of Target Patch e.g. 5.2.0.7
major_version=<Major Version> # Major Version of Target Patch e.g. 5.2
os_suffix=<OS Suffix> # One of [ ubuntu2004 amazonlinux2 rhel8 ]
Step 3: Download the infoworks package, installer package, and the update script using the following links:
https://iw-saas-setup.s3-us-west-2.amazonaws.com/${major_version}/patch/infoworks-${full_version}-${os_suffix}.tar.gz
https://iw-saas-setup.s3-us-west-2.amazonaws.com/${major_version}/deploy_5.2.0.tar.gz
https://iw-saas-setup.s3.us-west-2.amazonaws.com/${major_version}/update.sh
Step 4: SCP above files to these paths respectively:
Infoworks Package -> ${IW_HOME}/temp/downloads/.
Deploy Package -> ${IW_HOME}/scripts/.
Update Script -> ${IW_HOME}/scripts/.
Step 5: Create downloads directory if needed, and create the skip_binary_cleanup file:
[[ ! -d ${IW_HOME}/temp/downloads ]] && mkdir -p ${IW_HOME}/temp/downloads
touch ${IW_HOME}/temp/downloads/skip_binary_cleanup
Step 6: Initialize Environment Variables
source ${IW_HOME}/bin/env.shexport IW_PLATFORM=saas``
Step 7: Run the update
cd ${IW_HOME}/scripts/
./update.sh -v ${full_version}-${os_suffix}
Rollback
To go back to previous checkpoint version:
Step 1: In a web browser, go to your Infoworks system, scroll-down to the bottom, and click the Infoworks icon.
Step 2: The Infoworks Manifest Information page opens in a new tab. Scroll down and check the Last Checkpoint Version.
Step 3: ssh to Infoworks VM and switch to {{IW_USER}}.
Step 4: Initialize the variables in the bash shell.
full_version= 5.2.0.7
unpatched_version=$(echo $full_version | cut -d "." -f 1-3)
major_version=$(echo $full_version | cut -d "." -f 1-2)
previous_version=<Previous Version> # Last Checkpoint Version from step 1
os_suffix=<OS Suffix> # One of [ ubuntu2004 amazonlinux2 rhel8 ]
Step 5: Download the required deployer for the current applied patch.
https://iw-saas-setup.s3-us-west-2.amazonaws.com/${major_version}/deploy_${unpatched_version}.tar.gz
Step 6: Execute the SCP command for the above mentioned files to the following path.
${IW_HOME}/scripts/.
Step 7: Extract the deployed tar file in case it does not exist.
cd ${IW_HOME}/scripts
[[ -d iw-installer ]] && rm -rf iw-installer
tar xzf deploy_${unpatched_version}.tar.gz
cd iw-installer
Step 8: Initialize the environment variables.
source ${IW_HOME}/bin/env.sh
export IW_PLATFORM=saas
Step 9: Run the Rollback command.
./rollback.sh -v ${previous_version}-${os_suffix}