Date of Release: August 2024
JIRA ID | Issue |
---|---|
IPD-26469 | Infoworks - Dynamics 365 Sales connector |
JIRA ID | Issue |
---|---|
IPD-26794* | The pipeline version not getting mapped during workflow import. |
IPD-26759 | Unable to migrate workflow from DEV to PROD from UI on 5.5.1.4. |
IPD-26749 | The pipeline build job is failing with Table not found error when the source table of the pipeline belongs to a different domain's pipeline (target table). |
IPD-26746 | Platform CPU Utilization hike to 2500+%. |
IPD-26718 | Spark config spark.yarn.maxAppAttempts doesn't take effect if we set it as an advanced config in infoworks. |
IPD-26547 | Spike in CPU Utilization for Platform Service in Aflac Dev v5.5.1.2. |
IPD-26304 | Ingestion on user-managed tables failing with schema mismatch error post 5.5.1.2 upgrade. |
IPD-25940 | Snowflake pipeline export job on the existing artifacts (created in 5.0) creating the DB/SCHEMA/TABLE names as case sensitive (in lower case). |
IPD-25952 | Status.sh showing connection error warnings/exception stack trace when all services are in stopped state. |
IPD-26017 | Ingestion jobs fail when access control list parameter is configured with 'group'. |
IPD-26110 | The upload schema option is erroring if the user tries to update the column name. |
IPD-26115 | CICD failing for Pipeline group migration. |
IPD-26143 | Issue with executing snowflake stored procedure. |
IPD-26178 | Merge to Snowflake table fails SQL compilation error. |
IPD-25719 | Data validation for failed failure |
IPD-25837 | API to change table group scheduled user is not working |
IPD-25839 | GET table group call gives a refresh token of the scheduled user in the response |
IPD-25868 | Incorrect success response of verify refresh token API |
IPD-25840 | Source_schema_name and Source_table_name are interchanged in the ingestion metrics response in v5.5.0.5 |
IPD-25869 | Add table action of the file mapping doesn't validate the uniqueness of the target DB, schema, and table name combination |
IPD-25952 | Status.sh showing connection error warnings/exception stack trace when all services are in stopped state. |
Assuming IW_HOME variable is set to /opt/infoworks
To support rollback after metadata migration, you need to take backup of metadata. Following are the steps:
Step 1: Install/Download MongoDB tool: mongodump. (if needed).
Step 2: Create a directory to store the database backup dump using the below command.
xxxxxxxxxx
mkdir -p $IW_HOME/mongo_bkp
cd $IW_HOME/mongo_bkp
Step 3: Use the below command to take a dump (backup) of the databases from the mongodb server.
If MongoDB is hosted on Atlas
xxxxxxxxxx
mongodump "mongodb+srv://<username>:
<password>@<mongodb_server_hostname>/<db_name>"
If MongoDB is installed with Infoworks on the same VM
xxxxxxxxxx
mongodump "mongodb://infoworks:IN11**rk@localhost:27017/infoworks-new"
For upgrading from 5.5.1/5.5.1.x to 5.5.1.6, execute the following commands:
Step 1: Use the deployer to upgrade from 5.5.1 to 5.5.1.6.
Step 2: Go to $IW_HOME/scripts
folder of the machine.
Step 3: To ensure that there is no pre-existing update script, execute the following command:
xxxxxxxxxx
[[ -f update_5.5.1.6.sh ]] && rm update_5.5.1.6.sh
Step 4: Download the update_5.5.1.6.sh
wget
https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.5/update_5.5.1.6.sh
Step 5: Give update.sh executable permission
chmod +x update_5.5.1.6.sh
Step 6 (Optional): If the patch requires Mongo Metadata to be migrated, run export METADB_MIGRATION=Y
. This ensures that the metadata will be migrated, else run export METADB_MIGRATION=N
.
Alternatively, you can enter it in the prompt while running the script.
Step 7: Update the package to the hotfix
xxxxxxxxxx
source $IW_HOME/bin/env.sh
./update_5.5.1.6.sh -v 5.5.1.6-amazonlinux2
You will receive a "Please select whether metadb migration needs to be done([Y]/N)" message. If you need to perform metadb migration, enter Y, else, enter N.
Step 8: (Applicable only for EMR Persistent Clusters)
After successfully upgrading the environment, it is required to restart the Job Submission Driver running on the EMR cluster. This step ensures that the jobs utilize the updated JAR files.
To rollback the migrated metadata:
Step 1: Install/Download MongoDB tool: mongorestore. (if needed)
Step 2: Switch to the directory where the backup is saved on the local system.
xxxxxxxxxx
cd ${IW_HOME}/mongo_bkp/dump
Step 3: Use the below command to restore the dump (backup) of the databases to the Mongodb Server.
If MongoDB is hosted on Atlas
xxxxxxxxxx
mongorestore "mongodb+srv://<username>:<password>@<mongodb_server_hostname>/<db_name>” --drop ./<db_name>
If MongoDB is installed with Infoworks on the same VM
xxxxxxxxxx
mongorestore "mongodb://infoworks:IN11**rk@localhost:27017/infoworks-new" --drop ./<db_name>
To go back to previous checkpoint version:
Step 1: In a web browser, go to your Infoworks system, scroll-down to the bottom, and click the Infoworks icon.
Step 2: The Infoworks Manifest Information page opens in a new tab. Scroll down and check the Last Checkpoint Version.
Step 3: ssh
to Infoworks VM and switch to {{IW_USER}}.
Step 4: Initialise the variables in the bash shell.
xxxxxxxxxx
full_version=5.5.1.6
major_version=$(echo $full_version | cut -d "." -f 1-2)
previous_version=<Previous Version> # Last Checkpoint Version from step 1
os_suffix=<OS Suffix> # One of [ ubuntu2004 amazonlinux2 rhel8 ]
Step 5: Download the required deployer for the current applied patch.
https://iw-saas-setup.s3-us-west-2.amazonaws.com/${major_version}/deploy_${full_version}.tar.gz
Step 6: Execute the SCP command for the above mentioned files to the following path.
deploy_${full_version}.tar.gz
file in ${IW_HOME}/scripts/
directory.
${IW_HOME}/scripts/.
Step 7: Extract the deployed tar file in case it does not exist.
xxxxxxxxxx
cd ${IW_HOME}/scripts
[[ -d iw-installer ]] && rm -rf iw-installer
tar xzf deploy_${full_version}.tar.gz
cd iw-installer
Step 8: Initialise the environment variables.
xxxxxxxxxx
source ${IW_HOME}/bin/env.sh
export IW_PLATFORM=saas
Step 9: Run the Rollback command.
xxxxxxxxxx
./rollback.sh -v ${previous_version}-${os_suffix}