Date of Release: December 2023
JIRA ID | Issue |
---|---|
IPD-24939* | Configuration key to use custom port for Teradata TPT job. |
IPD-24821* | Accessible sources and make available in infoworks domains are not in sync. |
IPD-24502 | Support ingestion from Azure Event Hub using OAuth. Limitation: Ingestion job for confluent (while using OAuth) should be submitted to a non interactive cluster. |
JIRA ID | Issue |
---|---|
IPD-24747* | Update pipelines failing with invalid identifier of update column. |
IPD-24785* | Issue in Onboarding tables with same name from different schema. |
IPD-24820* | Scheduled workflows continue to trigger even if we make the source unavailable to the domain. |
IPD-24825* | Pipeline Group Jobs Next URL in API is not working as expected. |
IPD-24826* | Status of the pipeline is showing as Pending even though the pipeline-group job is aborted. |
IPD-24833* | Unable to set schedule job time correctly when minute is chosen as '00'. |
IPD-24850* | REST APIs do not encode # in the URL. |
IPD-24873* | Job Metrics are returned empty for a completed job via API. |
IPD-24754 | SQL node pipeline failing with query validation. |
IPD-24740 | Infoworks Config Migration API fails to update table configurations for tables if any of the tables within the source have already been ingested. |
IPD-24696 | Change in behaviour of upload schema in the 5.5 version. |
IPD-24680 | Ingestion jobs failing with class not found exception after 5.5.0.2 upgrade. |
IPD-24672 | Pipeline Config migration API automatically adds the audit columns to target node when the config json doesn't have any audit columns in target (This is not acceptable for an existing target table). |
IPD-24576 | Missing Teradata Drivers after Upgrade. |
IPD-24602 | Pipeline config import not proceeding further, giving a blank screen. |
IPD-24601 | All the reference tables are not listed in the reference table dropdown of the target configuration. |
IPD-24598 | All the source tables (included in the domain) are not listed in the table mappings dropdown. |
IPD-24201 | Unable to use derived column as watermark column for incremental ingestion. |
IPD-24529 | Subset of users unable to authenticate via SAML after upgrade to 5.5. |
IPD-24537 | Uncheck "enable watermark offset" is not working 5.5. |
IPD-24546 | Ingestion job marked as failed after successfully ingesting data. |
IPD-24555 | Filter sources by associated_domains using Infoworks API in v5.5. |
IPD-24529 | Subset of users unable to authenticate via SAML after upgrade to 5.5. |
IPD-24093 | Config-maps are getting overwritten during upgrade. |
xxxxxxxxxx
cat /opt/infoworks/iw-k8s-installer/infoworks/Chart.yaml | grep appVersion
xxxxxxxxxx
appVersion: 5.5.0.3
Ensure to take backup of MongoDB Atlas and PostgresDB PaaS. In case you don't take the backup, jobs will fail after the rollback operation. For more information, refer to the MongoDB Backup and PostgresDB PaaS Backup |
For the procedure of libraries uninstalling, refer Step 4 below.
To upgrade Infoworks on Kubernetes:
It is assumed that the existing chart is placed in the /opt/infoworks directory and the user has the access permission. |
xxxxxxxxxx
export IW_HOME="/opt/infoworks"
Before selecting the type of upgrade execute the following commands.
Step 1: Create the required directories and change the path to that directory.
xxxxxxxxxx
mkdir -p $IW_HOME/downloads
cd $IW_HOME/downloads
IW_HOSTED_REGISTRY=false
), you should download the Docker image ‘infoworks-mongo-utils_v5.5.0.tar.gz
’ following the instructions that have been shared with you. After downloading, push this image to your configured registry, ensuring that you do not change the image tags.
Step 1: Download the upgrade tar files shared by the Infoworks team to the Bastion (Jump host) VM and place it under $IW_HOME/downloads.
Step 2: To configure Internet-free upgrade, execute the following command:
xxxxxxxxxx
export INTERNET_FREE=true
Step 1: Download the Update script tar file.
xxxxxxxxxx
wget https://iw-saas-setup.s3-us-west-2.amazonaws.com/5.5/iwx_updater_k8s_5.5.0.4.tar.gz
Step 1: Extract the iwx_updater_k8s_5.5.0.4.tar.gz under $IW_HOME/downloads.
Do not extract the tar file to /opt/infoworks/iw-k8s-installer as it would result in loss of data. |
xxxxxxxxxx
tar xzvf iwx_updater_k8s_5.5.0.4.tar.gz
This should create two new files as follows - update-k8s.sh and configure.sh.
Step 2: Run the script.
xxxxxxxxxx
./update-k8s.sh -v 5.5.0.4
helm upgrade 531 /opt/infoworks/iw-k8s-installer/infoworks -n aks-upgrade-540 -f /opt/infoworks/iw-k8s-installer/infoworks/values.yaml
Enter N to skip running the above command to upgrade the helm deployment. (timeout: 30 seconds): N
Upgrade success
Step 3: To modify any of the configurations listed below, follow the steps:
Step 3A: Navigate to the directory IW_HOME/iw-k8s-installer
. And Edit the values.yaml
file
Name | Description | Default Values | |
---|---|---|---|
Email configuration | smtpHost | The SMTP host URL to connect to | smtp.gmail.com |
smtpPort | SMTP port | 587 | |
smtpUsername | The SMTP User to authenticate as | email address | |
smtpPassword | The Password for the SMTP user | Encrypted password | |
sslEnabled | The SSL flag | true | |
DT | timeoutSeconds | Timeout for DT | 7200 |
Ingestion | timeoutSeconds | Timeout for Ingestion | 7200 |
nginx.ingress | kubernetes.io/proxy-body-size | Nginx proxy body size | 10m |
Step 3B: After editing the annotations, the values.yaml file should look as shown below:
Update the values and save the file.
Step 3C: Navigate to the directory IW_HOME/iw-k8s-installer. And Run the iw_deploy script
xxxxxxxxxx
./iw_deploy.sh
Step 3D: Restart all the deployments
xxxxxxxxxx
kubectl rollout restart deployment -n <namespace>
Step 4: (Applicable only for Databricks Persistent Clusters): A change in the Infoworks jar requires libraries being uninstalled and cluster restart. Without this step, there will be stale jars. Perform the following steps:
(i) Go to the Databricks workspace, navigate to the Compute page, and select the cluster that has stale jars.
(ii) In the Libraries tab, select all the Infoworks jars and click Uninstall.
(iii) From Infoworks UI or Databricks dashboard, select Restart Cluster.
xxxxxxxxxx
export IW_HOME=/opt/infoworks
xxxxxxxxxx
cat $IW_HOME/iw-k8s-installer/infoworks/Chart.yaml | grep appVersion
xxxxxxxxxx
appVersion: 5.5.0.4
Ensure to restore MongoDB Atlas and PostgresDB PaaS. In case you don't take the backup, jobs will fail after the restore operation. For more information, refer to the MongoDB Restore and PostgresDB Restore. |
Step 1: Download the rollback script.
xxxxxxxxxx
Step 2: Place the Update script in the same directory as that of the existing iw-k8s-installer.
Step 3: Ensure you have permission to the $IW_HOME directory.
Step 4: Give executable permission to the rollback script using the below command.
xxxxxxxxxx
Step 5: Run the script.
xxxxxxxxxx
Step 6: You will receive the following prompt, "Enter N to skip running the above command to upgrade the helm deployment. (timeout: 30 seconds): ", type Y and press Enter.
xxxxxxxxxx