v5.4.2.4

Date of Release: August 2023

Resolved Issues

NOTE The "*" symbol next to ID refers to the issues that have been resolved in the current release.

JIRA ID

Issue

IPD-23313*

Environment details API throws authentication error for prodops user, however, the same can be viewed from UI by prodops user.

IPD-23451*

Rest API - Bug in 5.4.1 config migration where few keys are in camelCase (should be in snake_case).

IPD-23589*

Issues when selecting all tables in UI for onboarding, metadata crawl.

IPD-23634*

List table APIs bringing duplicates during different offsets parameters in v5.4.2.3.

IPD-23714*

Source connection details API not returning the additional connection parameters in 5.4.x.

IPD-23716*

Add File mapping API failing, causing the mass ingestion script failure.

IPD-23733*

"Make available in Infoworks domains" not coming in source details API.

IPD-23734*

Source Config migration API on changing the table group names in iw_mappings not reflecting after import.

IPD-23783*

Source config migration API doesn't return use_staging_schema_for_infoworks_managed_tables.

IPD-23949*

Source Config Migration API on 5.4.2.4 latest build started to fail.

IPD-23963*

Issues with table-level email notification.

IPD-22972

Azure Key Vault drop down doesn't show context of the secret for non-Admin users.

IPD-23079

Sync to target job for Postgres with authentication issue with special characters is fixed.

IPD-23080

Passwords will no longer be visible in Sync to Target Job logs.

IPD-23128

Unsaved changes popup when updated table configurations via API in v5.4.2.

IPD-23129

For SCD 2 type, few fields are not getting saved in Table Document in v5.4.2.

IPD-23200

GET source configuration call is failing on snowflake meta sync source.

IPD-23217

Unable to unlock locked entities.

IPD-23239

Fixed snowflake meta sync source creation from RestAPI.

IPD-23264

Relax scale and precision check in snowflake onboarding.

IPD-23228

Blob storage changes for supporting 17.2 TPT. set tpt_add_directory_separator_to_object false to for single writer or TPT version lower than 17.20

set tpt_add_directory_separator_to_object to true(default) when using multiple writers.

IPD-22755

The GET and PUT methods now returns JSON table containing the user_managed_target_path key for user-managed tables.

IPD-22760

The SQL Import now correctly executes the GROUP BY ROLLUP queries.

IPD-22857

The Segment Load job with multi-watermark column now completes without any delay.

IPD-21954

While importing the BTEQs with table joins containing the InNotIn condition, the ClassCastException failure - io.infoworks.awb.pipeline.dto.InNotInNode to io.infoworks.awb.pipeline.dto.JoinNode error does not appear anymore.

IPD-21955

While parsing the BTEQs with UPDATE, GROUP BY, and HAVING conditions into the SQL pipelines, the ClassCastException failure - org.apache.calcite.sql.SqlIdentifier to org.apache.calcite.sql.SqlBasicCall error does not appear anymore.

IPD-21956

Each expression value in the Select SQL query is now assigned to a unique derived column.

IPD-22025

If you set the dt_skip_sql_validation configuration and then delete it, the configuration will be set to false (default value).

IPD-22032

The importSql REST API is now working correctly.

IPD-22034

The non-visual Infoworks SQL Pipeline can now execute the SnowSQL query.

IPD-22037

The user-managed table options are now available for streaming sources in the CDW environment.

IPD-22048

During the workflow migration, the dropdown under the Interactive Compute Template Mapping section is now enabled.

IPD-22051

Input ports of the Union node are getting assigned correctly during the pipeline validation.

IPD-22076

The Sync to External Target functionality for the Delimited files is now successfully displaying the data connection options.

IPD-22332

User can now set the active_version_id using REST API.

IPD-22421

The import workflow configuration migration API now includes pipeline version mapping.

Upgrade

Prerequisites

NOTE Before going through the below mentioned prerequisites for this section, ensure that all the Prerequisites for Installing Infoworks on AKS are validated.

  • Ensure the current deployment’s chart is present in /opt/infoworks.

  • Python 3.8 or later version with the pip module is installed in the Bastion VM.

  • Stable internet connectivity on the Bastion VM to download the required python packages from the python repository during installation/upgrades.

  • Validate that the version of the old chart is 5.4.2.3.

cat /opt/infoworks/iw-k8s-installer/infoworks/Chart.yaml | grep appVersion
appVersion: 5.4.2.3
Important

Ensure to take backup of MongoDB Atlas and PostgresDB PaaS. In case you don't take the backup, jobs will fail after the rollback operation. For more information, refer to the MongoDB Backup and PostgresDB PaaS Backup

Upgrade Instructions

To upgrade Infoworks on Kubernetes:

ASSUMPTION

It is assumed that the existing chart is placed in the /opt/infoworks directory and the user has the access permission.

export IW_HOME="/opt/infoworks"

Before selecting the type of upgrade execute the following commands.

Step 1: Create the required directories and change the path to that directory.

mkdir -p $IW_HOME/downloads cd $IW_HOME/downloads

Internet-free Upgrade

NOTE If you are upgrading via Internet-based procedure, skip to the next section.

Step 1: Download the upgrade tar files shared by the Infoworks team to the Bastion (Jump host) VM and place it under $IW_HOME/downloads.


Step 2: To configure Internet-free upgrade, execute the following command:

export INTERNET_FREE=true

Internet-based Upgrade

Step 1: Download the Update script tar file.

wget https://iw-saas-setup.s3-us-west-2.amazonaws.com/5.4/iwx_updater_k8s_5.4.2.4.tar.gz

Common Steps for Both Internet-free and Internet-based

NOTE Once you have selected the type of upgrade, the below mentioned steps are common for both Internet-free and Internet-based.

Step 1: Extract the iwx_updater_k8s_5.4.2.4.tar.gz under $IW_HOME/downloads.

WARNING

Do not extract the tar file to /opt/infoworks/iw-k8s-installer as it would result in loss of data.

tar xzvf iwx_updater_k8s_5.4.2.4.tar.gz

This should create two new files as follows - update-k8s.sh and configure.sh.

Step 2: Run the script.

./update-k8s.sh -v 5.4.2.4

NOTE At the end of the above command's execution, if you want to run helm upgrade manually, then type N and press Enter. There is a 30-second timeout set to abandon the deployment of the upgraded version. If no input is received within the timeout duration, the deployment is triggered.

helm upgrade 531 /opt/infoworks/iw-k8s-installer/infoworks -n aks-upgrade-540 -f /opt/infoworks/iw-k8s-installer/infoworks/values.yaml Enter N to skip running the above command to upgrade the helm deployment. (timeout: 30 seconds): N Upgrade success

Rollback

Prerequisites

  • Before executing the rollback script, ensure that IW_HOME variable is set.

  • Assuming Infoworks home directory is /opt/infoworks, run the below command to set the IW_HOME variable.

  • Validate that the version of the old chart is 5.4.2.4.

export IW_HOME=/opt/infoworks
  • Ensure the current deployment’s chart is present in /opt/infoworks.

  • Execute the below command to check the appVersion.

cat $IW_HOME/iw-k8s-installer/infoworks/Chart.yaml | grep appVersion
appVersion: 5.4.2.4
Important

Ensure to restore MongoDB Atlas and PostgresDB PaaS. In case you don't take the backup, jobs will fail after the restore operation. For more information, refer to the MongoDB Restore and PostgresDB Restore.

Rollback Instructions

NOTE Since IW_HOME has been exported in the Prerequisites section mentioned above, the following steps can be executed from any location for users with read/write access to the aforementioned IW_HOME.

Step 1: Download the rollback script.

wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.4/rollback-k8s.sh

Step 2: Place the Update script in the same directory as that of the existing iw-k8s-installer.

Step 3: Ensure you have permission to the $IW_HOME directory.

Step 4: Give executable permission to the rollback script using the below command.

chmod +x rollback-k8s.sh

Step 5: Run the script.

./rollback-k8s.sh -v 5.4.2.3

Step 6: You will receive the following prompt, "Enter N to skip running the above command to upgrade the helm deployment. (timeout: 30 seconds): ", type Y and press Enter.

NOTE If you want to run helm upgrade manually, then type N and press Enter.

Enter N to skip running the above command to upgrade the helm deployment. (timeout: 30 seconds): Y

NOTE There is a 30-second timeout set to abandon the deployment of the downgraded version. If no input is received within the timeout duration, the deployment is triggered.