Infoworks Installation on Azure Kubernetes Service (AKS)

NOTE If you have not installed Azure Kubernetes Service, refer to Installing Azure Kubernetes Service (AKS).

Prerequisites

NOTE

If you are using MAC OS to deploy Infoworks on to cluster, you must install the following package:

• Ensure that AKS Kubernetes cluster is connected to internet.
• Set up AKS Kubernetes cluster. For more information, refer to the Azure Docs.
• Ensure that Kubernetes version should be 1.23.x.
• Infoworks recommends creating the AKS Kubernetes cluster with private access and a VM as a Bastion host with Ubuntu 20.04 OS should be created within the VPC.
• To use an external Azure Container Registry (ACR) for pulling the images for Infoworks setup, ensure that all the required images are pushed to the the specified external registry and it is integrated with AKS. This is applicable if IW_HOSTED_REGISTRY is set to false. To get the required container images, contact your Infoworks representative.
• If INGRESS_CONTROLLER_CLASS is set to nginx, then Infoworks recommends setting up ingress-controller externally with the required configurations. To set up nginx ingress-controller externally, refer to External Setup for Ingress Controller.
• Install Azure CLI, Helm, and Kubectl on the Bastion host VM instance.
• Verify the following prerequisites
  • Run az version to ensure that az is installed.
  • Run helm version to ensure that Helm is installed.
  • Run kubectl version to ensure that Kubectl is installed.
  • Run python3 -V to ensure that python (with venv) is installed.
  • Run sudo apt install python3-venv to install python Virtual Environment package.

xxxxxxxxxx

root@aks-dev-qa-bastion:~$ az version

{

  "azure-cli": "2.0.81",

   "azure-cli-core": "2.0.81",

   "azure-cli-telemetry": "1.0.4",

   "extensions": {

    "azure-devops": "0.17.0"

  }

}

root@aks-dev-qa-bastion:~$ helm version

version.BuildInfo{Version:"v3.9.0", GitCommit:"7ceeda6c585217a19a1131663d8cd1f7d641b2a7", GitTreeState:"clean", GoVersion:"go1.17.5"}

root@aks-dev-qa-bastion:~$ kubectl version

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.6", GitCommit:"f59f5c2fda36e4036b49ec027e556a15456108f0", GitTreeState:"clean", BuildDate:"2022-01-19T17:33:06Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.6", GitCommit:"07959215dd83b4ae6317b33c824f845abd578642", GitTreeState:"clean", BuildDate:"2022-03-30T18:28:25Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"}

root@aks-dev-qa-bastion:~# python3 -V

Python 3.8.10

• Set up Kubernetes Cluster in AKS for connection using az

NOTE The following procedure is a one-time activity for the user.

Step 1: Execute az login.

xxxxxxxxxx

root@aks-dev-qa-bastion:~$ az login

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code RS6X8HQZQ to authenticate.

After successful verification, the following confirmation message appears.

xxxxxxxxxx

[

{

"cloudName": "AzureCloud",

"id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",

"isDefault": true,

"name": "Subscription-Name",

"state": "Enabled",

"tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",

"user": {

"name": "xxxxxxxxx@infoworks.domain",

"type": "user"

}

}

Step 2: Identify the cluster name, zone/region, and project you want to connect to. Run the following command with these details:

xxxxxxxxxx

az aks get-credentials --resource-group <resourceGroupName> --name <AKSClusterName> --subscription <SubscriptionID>

xxxxxxxxxx

Merged "<AKSClusterName>" as current context in /home/infoworks/.kube/config

Persistent Storages

NOTE Azure Kubernetes Service RBAC Cluster Admin is required to run Infoworks installation.

Persistence ensures to persist the data even if a pod restarts or fails due to various reasons. Infoworks needs the following persistent storages to be configured:

• Databases (MongoDB and Postgres) and RabbitMQ
• Infoworks Job Logs and Uploads

Run the following command to fetch the storage classes:

xxxxxxxxxx

kubectl get storageclass --no-headers

NOTE The storageclass should have the reclaim policy Retain.

xxxxxxxxxx

Azurefile               file.csi.azure.com 

azurefile-csi           file.csi.azure.com 

azurefile-csi-premium   file.csi.azure.com 

azurefile-premium       file.csi.azure.com 

default (default)       disk.csi.azure.com 

managed                 disk.csi.azure.com 

managed-csi             disk.csi.azure.com 

managed-csi-premium     disk.csi.azure.com 

managed-premium         disk.csi.azure.com

Installing Infoworks on Kubernetes

NOTE Assuming IW_HOME=/opt/infoworks.

NOTE The following steps must be performed by the user who has completed the prerequisites procedure mentioned earlier.

Step 1: Create Infoworks directory under /opt.

sudo mkdir -p /opt/infoworks

Step 2: Change permissions of /opt/infoworks directory to a desired user.

sudo chown -R <user>:<group/user> /opt/infoworks

Step 3: Change the directory path to /opt/infoworks.

cd /opt/infoworks

Step 4: To download Infoworks Kubernetes template, execute the following command:

xxxxxxxxxx

wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/5.3/iwx_installer_k8s_5.3.1.tar.gz

Step 5: Extract the downloaded file.

xxxxxxxxxx

tar xzf iwx_installer_k8s_5.3.1.tar.gz

Step 6: Navigate to the extracted directory iw-k8s-installer.

Step 7: Open configure.sh file in the directory.

Step 8: Configure the following parameters as described in the table, and then save the file

NOTE Namespace and release names should not contain underscore (_). Release names should not start with numbers.

NOTE If you want to enable SSL before installing Infoworks, refer to Enabling SSL section.

The following configurations are generic configuration for the Infoworks deployment.

External Container Registry Configurations

The following table lists the External Container Registry Configurations for Infoworks Setup. These configurations should be set only if the Container Registry used to pull the images is different from the one hosted by Infoworks.

The following fields are valid if IW_HOSTED_REGISTRY set to false

Service Mesh Configurations for Security

NOTE Infoworks supports linkerd as of now.

NOTE

The following are the External MongoDB Configurations for MongoDB Atlas. These configurations are not required if MongoDB is internal.

The following fields are applicable if EXTERNAL_MONGO is set to true.

External POSTGRESDB Configurations for Postgres

NOTE

The below configurations are not required if PostgresDB is internal.

Below fields are applicable if EXTERNAL_POSTGRESDB is true.

Step 9 (Optional): To run Infoworks jobs on separate workloads, edit values.yaml file under infoworks folder. Specifically, you need to edit jobnodeSelector and jobtolerations fields based on the node pool you created in the Installing Azure Kubernetes Service.

NOTE If you want to run Infoworks services on other node pools, you can edit nodeSelector and tolerations fields.

Step 10 (Optional): To define the PaaS passwords, there are two methods:

First method

The password must be put in pre-existing secrets in the same namespace.

For MongoDB

(i) Set MONGODB_USE_SECRET_PASSWORD=true

(ii) To create the custom secret resource, run the following commands from the iw-k8s-installer directory.

xxxxxxxxxx

$ encrypted_mongo_password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<mongo-password>" | xargs echo -n | base64 -w 0)

$ IW_NAMESPACE=<IW_NAMESPACE> 

$ MONGODB_SECRET_NAME=<MONGODB_SECRET_NAME>

$ kubectl create ns ${IW_NAMESPACE}

$ kubectl apply -f - <<EOF

apiVersion: v1

kind: Secret

metadata:

  name: ${MONGODB_SECRET_NAME}

  namespace: ${IW_NAMESPACE}

data:

  MONGO_PASS: ${encrypted_mongo_password}

type: Opaque

EOF

NOTE Set the MONGODB_SECRET_NAME and IW_NAMESPACE according to the inputs given to the automated script. <mongo-password> is the plaintext password.

For Postgres

(i) Set POSTGRESDB_USE_SECRET_PASSWORD=true

(ii) To create the custom secret resource, run the following commands from the iw-k8s-installer directory.

xxxxxxxxxx

$ encrypted_postgres_password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<postgres-password>" | xargs echo -n | base64 -w 0)

$ IW_NAMESPACE=<IW_NAMESPACE> 

$ POSTGRESDB_SECRET_NAME=<POSTGRESDB_SECRET_NAME>

$ kubectl create ns ${IW_NAMESPACE}

$ kubectl apply -f - <<EOF

apiVersion: v1

kind: Secret

metadata:

  name: ${POSTGRESDB_SECRET_NAME}

  namespace: ${IW_NAMESPACE}

data:

  POSTGRES_PASS: ${encrypted_postgres_password}

type: Opaque

EOF

NOTE Set the POSTGRESDB_SECRET_NAME and IW_NAMESPACE according to the inputs given to the automated script. postgres-password is the plaintext password.

Second Method

You can give the password to the Automated Script, which will encrypt it to store it in the templates.

Step 11 (Optional): Enable NodeSelector/Toleration and Custom annotations etc. by editing values.yaml file manually before deploying Infoworks deployment.

Step 12: To run the script, you must provide execute permission beforehand by running the following command.

xxxxxxxxxx

chmod 755 iw_deploy.sh

Step 13: Run the script.

NOTE If you see this error, "INSTALLATION FAILED: failed post-install: timed out waiting for the condition", you can ignore this as it does not affect Infoworks installation.

xxxxxxxxxx

./iw_deploy.sh

Checking for Pre requisite. 

NOTE: (Optional) Enable NodeSelector/Toleration and  Custom annotations  etc., by editing values.yaml file manually before deploying infoworks app

Checking for basic Pre requisite.

Found HELMv3

Found KUBECTL

Testing Kubernetes basic cluster connection

Validation is done: Kubernetes Cluster is Authorized

Enter kubernetes namespace to deploy Infoworks

v1

Enter release name for infoworks 

v1

Creating v1 namespace on kubernetes cluster 

namespace/v1 created

Input the Kubernetes Cluster Cloud Provider Environment- aws/gcp/azure 

azure

List of available StorageClass in Kubernetes Cluster 

azurefile

azurefile-csi

azurefile-csi-premium

azurefile-premium

default

managed

managed-csi

managed-csi-premium

managed-premium

INFO: NFS and Database (Disk) persistance is recommended and always set to True 

Enter NFS StorageClass: Please select StorageClass from list

azurefile-premium

Enter DATABASE StorageClass: Please select StorageClass from list

managed-premium

ENABLE INGRESS: true or false Default: "true"

true

Select Ingress Controller Class: cloud native "cloud" or external "nginx" Default: "nginx"

nginx

Select Ingress type: internal or external Default: "internal"

internal

​

​

Provisioning Nginx Ingress controller automatically.

NOTE: If the Ingress-Nginx is already provisioned manually skip this by selecting 'N'  

Do you want to continue y/n? Default: "y"

y

Enter DNS Hostname to access Infoworks: for example: iwapp.infoworks.local 

sample.infoworks.technology

ENABLE SSL for the Infoworks Ingress deployment (This enables port and protocol only): true or false Default: "true"

true

ENABLE HA for Infoworks Setup: true or false Default: "true"

true

ENABLE external MongoDB access for Infoworks Setup: true or false Default: "false"

true

ENABLE SRV connection string for MongoDB access for Infoworks Setup: true or false, MongoDB Atlas default is true Default: "true"

true

Input MongoDB DNS connection string for Infoworks Setup: Private link ex - {DB_DEPLOYMENT_NAME}-pl-0.{RANDOM}.mongodb.net 

mongo-pl-0.1234.mongodb.net

Input MongoDB Username for Infoworks Setup. Assuming the user have permissions to create databases if doesn't exist. 

infoworks

MongoDB Encrypted password stored in Secrets of the Infoworks namespace: true or false  Default: "false"

false

Input the MongoDB password for Infoworks database Setup. Infoworks will encrypt the MongoDB password.

MongoDB force drop existing database and restore the initial seed data: true or false  Default: "false"

false

Input the database name of MongoDB for Infoworks Setup. default: infoworks-db 

infoworks-db

Input the scheduler database name of MongoDB for Infoworks Setup. default: quartzio 

quartzio

ENABLE external PostgresDB access for Infoworks Setup: true or false Default: "false"

true

Input Postgres DNS / IP connection string for Infoworks Setup: Private link ex - {DB_DEPLOYMENT_NAME}.postgres.com or IP address 

postgres.postgres.com

Input postgresDB Username for Infoworks Setup. Assuming the user have permissions to create databases if doesn't exist. 

infoworks

Postgres user Encrypted password stored in Secrets of the Infoworks namespace: true or false  Default: "false"

false

Input the Postgres user password for Infoworks database Setup. Infoworks will encrypt the Postgres password. 

Input the database name of Postgres for Infoworks Setup. default: airflow 

airflow

Enable PostgresDB SSL: true or false  Default: "true"

true

ENABLE Service mesh for Infoworks Setup, Only Linkerd supported: true or false Default: "false"

true

Input service mesh name for Infoworks Setup: Defaults to linkerd 

linkerd

Since the above installation was configured for ingress-controller, run the following command to get the domain mapping done.

xxxxxxxxxx

NAME: intrue

LAST DEPLOYED: Fri Jul  2 17:25:20 2021

NAMESPACE: intrue

STATUS: deployed

REVISION: 1

xxxxxxxxxx

kubectl get ingress --namespace sample

xxxxxxxxxx

NAME         CLASS    HOSTS                         ADDRESS          PORTS   AGE

v1-ingress   <none>   sample.infoworks.technology   43.13.121.142   80      3m43s

NOTE Make sure to enable DNS mapping for IP address as per the above sample output.

Get the application URL by running these commands: http://sample.infoworks.technology

Enabling Linkerd Service Mesh for Ingress Controller

To enable the communication between the ingress controller and the infoworks pods via linkerd service mesh, perform the following steps:

There are two options to enable the communication which depends on the method by which ingress controller was set up.

Option 1 - Infoworks deployed the ingress controller.

If INGRESS_AUTO_PROVISIONER is set to true during installation:

Step 1: Retrieve Name of the nginx deployment.

NOTE Set the IW_NAMESPACE according to the inputs given to the automated script.

xxxxxxxxxx

kubectl get deploy -n $IW_NAMESPACE | grep nginx | awk '{print $1}'

Output: This prints the name of the ingress-controller pod.

xxxxxxxxxx

ingress-nginx-controller

Step 2: Edit nginx deployment.

xxxxxxxxxx

kubectl  edit deploy <ingress-deployment-name> -n $IW_NAMESPACE

Step 3: Update .spec.template.metadata.annotations (ensure that the full yaml path is updated, not just the annotations field) and save this file.

xxxxxxxxxx

spec:

  template:

    metadata:

      annotations:

        linkerd.io/inject: enabled

Step 4: Wait for the ingress-controller pods to be updated.

Option 2 - User deployed the Ingress Controller

If INGRESS_AUTO_PROVISIONER is set to false during installation, refer to the official Linkerd documentation to enable service mesh for your corresponding ingress controller.

Enabling SSL

If you set INGRESS_CLASS to nginx, add SSL Termination in the TLS section of values.yaml file either before running the automated script or after the deployment.

Step 1: Log in to Linux machine on the latest Debian-based OS.

Step 2: Ensure libssl-dev package is installed.

Step 3: Provide DNS Name for Infoworks deployment

Generating Self-Signed SSL Certificate:

To generate SSL, run the following commands:

xxxxxxxxxx

mkdir certificates

xxxxxxxxxx

cd certificates

xxxxxxxxxx

openssl genrsa -out ca.key 2048   # Creates a RSA key

NOTE Refer the following commands to replace "Infoworks.domain" and "subdomain.infoworks.domain" keywords with required domain and subdomain name.

xxxxxxxxxx

openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=CA/L=US/O=Infoworks, Inc./CN=Infoworks Root CA" -out ca.crt

xxxxxxxxxx

openssl req -newkey rsa:2048 -nodes -keyout server.key -subj "/C=CN/ST=CA/L=US/O=Infoworks, Inc./CN=*.infoworks.domain" -out server.csr

xxxxxxxxxx

openssl x509 -req -extfile <(printf "subjectAltName=DNS:infoworks.domain,DNS:subdomain.infoworks.domain") -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt

Keep a note of server.crt and server.key files for self-signed certificates for Nginx SSL Termination and provide the valid values for ingress_tls_secret_name and namespace_of_infoworks.

Run the following command to add the tls certificates to the Kubernetes cluster.

xxxxxxxxxx

kubectl create secret tls <ingress_tls_secret_name> --cert=server.crt --key=server.key -n <namespace_of_infoworks>

Edit values.yaml file to look similar to the following sample file.

It is suggested to make changes in the values.yaml file and add the below parameters as annotations in the ingress block.

xxxxxxxxxx

nginx.ingress.kubernetes.io/enable-cors: "true"

nginx.ingress.kubernetes.io/configuration-snippet: |

  proxy_hide_header Access-Control-Allow-Origin;

  add_header "Access-Control-Allow-Origin" "<URL>" always;

  add_header "Access-Control-Allow-Methods" "GET, PUT, POST, OPTIONS" always;

  add_header "Access-Control-Allow-Headers" "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization" always;

  add_header "Access-Control-Expose-Headers" "Content-Length,Content-Range" always;

nginx.ingress.kubernetes.io/cors-allow-credentials: "true"

After adding the annotations, the values.yaml file should look as shown below.

NOTE If you have already performed the deployment and edit the values.yaml file, then run helm upgrade command.

Enabling High-Availability and Scalability

Enabling High-Availability

Infoworks installation enables high-availability configuration while setting up Infoworks in Kubernetes. You can enable high-availability by editing the helm file called values.yaml.

Step 1: To edit values.yaml file, perform the action given in the following snippet.

xxxxxxxxxx

global:

​

  haEnabled: true

Step 2: Run HELM upgrade command.

xxxxxxxxxx

helm upgrade <release_name> infoworks/ --values infoworks/values.yaml -n <namespace>

This enables the high availability for Infoworks.

Limitation For Kubernetes HA setup, Postgres database is not completely HA compliant. In certain conditions, if the Postgres containers crash, it might result in workflow failures.

Enabling Scalability

Infoworks installation supports manual scaling of any pods.

For a scalable solution:

• There must be a minimum of two replicas.
• They can be scaled to any number based on available resources (CPU and memory).
• Infoworks supports scalability of source, pipeline, and workflow jobs out of the box. Ensure that there are available resources in the Kubernetes cluster.

There are two ways to enable scalability:

1. By editing the values.yaml file.

Step 1: Edit the values.yaml file.

xxxxxxxxxx

infoworks(deploymentname):

   replicas: 4

NOTE The “deploymentname” mentioned in the above parenthesis is given just for the ease of understanding. This deployment name can be a platform, UI, etc. with actual name.

For example:

xxxxxxxxxx

InfoworksUI:

    replicas: 4

Step 2: To increase the scalability manually, run HELM upgrade command:

xxxxxxxxxx

helm upgrade <release_name> infoworks/ --values infoworks/values.yaml -n <namespace>

2. Using Kubectl

xxxxxxxxxx

kubectl scale --replicas=3 rs/<deploymentName>

NOTE The “deploymentname” mentioned in the above parenthesis is given just for the ease of understanding. This deployment name can be a “releasename-platform-id” with the actual name.

For example:

xxxxxxxxxx

kubectl scale --replicas=3 rs/releasename-platform-id -n <namespace>

Compute Environment Support

Increasing the Size of PVCs

To scale the size of PVCs attached to the pods:

Step 1: Note the storage class of the PVCs to be scaled.

xxxxxxxxxx

kubectl -n <namespace> get pvc

Step 2: Ensure allowVolumeExpansion is set to true in the storageClass.

xxxxxxxxxx

kubectl edit storageClass <storage-class-of-pvc>

allowVolumeExpansion: true

Step 3: Delete the managing statefulset without deleting the pods.

xxxxxxxxxx

kubectl -n <namespace> get sts

kubectl -n <namespace> delete sts --cascade=orphan <statefulset name>

Step 4: For each PVC, upscale the size (ensure all PVCs attached managed by a single statefulset have the same size. For example, all Postgres managed PVCs must have the same size).

xxxxxxxxxx

kubectl -n <namespace> get pvc

kubectl -n <namespace> edit pvc <pvc-name>

Step 5: Navigate to the helm chart used for Infoworks deployment.

Step 6: Edit the values.yaml file to update the size of the corresponding database to the new value.

Step 7: Run the helm upgrade command.

xxxxxxxxxx

helm upgrade --recreate-pods --reuse-values -f <path-to-your-values.yaml> <your-release-name> <path-to-your-chart> -n <your-namespace>

Warning

Above upgrade command will recreate all pods with the same PVCs.

Updating the MongoDB and PostgresDB Credentials

To update the MongoDB and/or PostgresDB credentials in the Infoworks deployment, follow the below given procedure.

Updating the MongoDB Credentials

Updating Encrypted Passwords Stored in values.yaml

There are two methods to update password:

Method 1

To update MongoDB encrypted passwords that are stored in values.yaml file, with the existing configure.sh file, use the IW_DEPLOY script to populate values.yaml:

Step 1: Download and untar the Infoworks kubernetes template, if not already present, according to the iwx-version in your existing deployment.

xxxxxxxxxx

version="5.3.1"

major_version=$(echo $version | cut -d '.' -f 1,2)

wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/$major_version/iwx_installer_k8s_$version.tar.gz

tar xzf iwx_installer_k8s_$version.tar.gz

Step 2: If a new template was downloaded, replace the iw-k8s-installer/configure.sh as well as iw-k8s-installer/infoworks/values.yaml with the older file.

xxxxxxxxxx

mv /path/to/older/configure.sh iw-k8s-installer/configure.sh

mv /path/to/older/values.yaml iw-k8s-installer/infoworks/values.yaml

Step 3: Change the directory path to iw-k8s-installer.

xxxxxxxxxx

cd iw-k8s-installer

Step 4: Replace the following values with a blank string in the configure.sh file.

xxxxxxxxxx

MONGODB_USERNAME=""

MONGODB_ENCRYPTED_PASSWORD=""

Step 5: Run iw_deploy.sh. Once you receive "Seems like you have already configured Infoworks once. Do you want to override? y/n Default: n", enter “Y”. This will prompt the user to provide input for the values that were blank in the previous step. The script will then replace the infoworks/values.yaml file with the updated values.

xxxxxxxxxx

infoworks@bastion-host:~/iw-k8s-installer$ ./iw_deploy.sh 

NOTE: (Optional) Enable NodeSelector/Toleration and  Custom

annotations  etc., by editing values.yaml file manually before deploying infoworks app 

Seems like you have already configured Infoworks once. Do you want to override? y/n  Default: n

y

Checking for basic Pre requisite.

Found HELMv3

Found KUBECTL

Testing Kubernetes basic cluster connection

Validation is done: Kubernetes Cluster is Authorized 

qa-531 Namespace already exists

​

Input MongoDB Username for Infoworks Setup. Assuming the user have permissions to create databases if doesn't exist. 

updated-mongouser

Input the MongoDB password for Infoworks database Setup. Infoworks will encrypt the MongoDB password. 

Upgrade INFOWORKS ...

 helm upgrade release-name ./infoworks --values ./infoworks/values.yaml -n namespace

Step 6: Run the following command to upgrade by specifying your namespace and helm release name according to the values given in the configure.sh file.

xxxxxxxxxx

helm upgrade $IW_RELEASE_NAME ./infoworks --values ./infoworks/values.yaml -n $IW_NAMESPACE

Method 2

To update MongoDB encrypted passwords, you can directly modify the values.yaml file.

Step 1: Download and untar the Infoworks Kubernetes Template, if not already present, according to the iwx-version in your existing deployment.

xxxxxxxxxx

version="5.3.1"

major_version=$(echo $version | cut -d '.' -f 1,2)

wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/$major_version/iwx_installer_k8s_$version.tar.gz

tar xzf iwx_installer_k8s_$version.tar.gz

Step 2: If a new template was downloaded, replace the iw-k8s-installer/infoworks/values.yaml with the older file.

xxxxxxxxxx

mv /path/to/older/values.yaml iw-k8s-installer/infoworks/values.yaml

Step 3: Change the directory path to iw-k8s-installer directory.

xxxxxxxxxx

cd iw-k8s-installer

Step 4: Generate the encrypted passwords as needed. To generate any encrypted string, execute the following command.

xxxxxxxxxx

encrypted-mongo-password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<password>")

This generates your passwords in a secure encrypted format, which has to be provided in the following steps.

Step 5: Replace the following yaml keys with the new values in the infoworks/values.yaml file, if needed.

xxxxxxxxxx

databases:

metaDB:

  auth:

    username: "mongo-username"

    encryptedMongoPass: "encrypted-mongo-password"

Step 6: Run the following command to upgrade by specifying your namespace and helm release name according to the installed kubernetes deployment specifications.

xxxxxxxxxx

helm upgrade $IW_RELEASE_NAME ./infoworks --values ./infoworks/values.yaml -n $IW_NAMESPACE

Updating Encrypted Passwords Stored as a Separate Secret

To update the MongoDB password:

Step 1: Run the following commands from the iw-k8s-installer directory.

xxxxxxxxxx

$ encrypted_mongo_password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<mongo-password>" | xargs echo -n | base64 -w 0)

$ IW_NAMESPACE=<IW_NAMESPACE> 

$ MONGODB_SECRET_NAME=<MONGODB_SECRET_NAME>

$ kubectl patch secret -n ${IW_NAMESPACE} ${MONGODB_SECRET_NAME} --type='json' -p="[{'op' : 'replace' ,'path' : '/data/MONGO_PASS ,'value' : '${encrypted_mongo_password}'}]"

Step 2: Restart all pods except the databases.

xxxxxxxxxx

kubectl get pods -n ${IW_NAMESPACE} --no-headers=true | awk '!/-rabbitmq-|-postgres/{print $1}' | xargs  kubectl delete -n ${IW_NAMESPACE} pod

Updating the PostgresDB Credentials

Updating Encrypted Passwords Stored in values.yaml

There are two methods to update password:

Method 1

To update PostgresDB passwords that are stored in values.yaml file, with the existing configure.sh file, use the IW_DEPLOY script to populate values.yaml.

Step 1: Download and untar the Infoworks Kubernetes Template, if not already present, according to the iwx-version in your existing deployment.

xxxxxxxxxx

version="5.3.1"

major_version=$(echo $version | cut -d '.' -f 1,2)

wget

https://iw-saas-setup.s3.us-west-2.amazonaws.com/$major_version/iwx_installer_k8s_$version.tar.gz

tar xzf iwx_installer_k8s_$version.tar.gz

Step 2: If a new template was downloaded, replace the iw-k8s-installer/configure.sh as well as iw-k8s-installer/infoworks/values.yaml with the older file.

xxxxxxxxxx

mv /path/to/older/configure.sh iw-k8s-installer/configure.sh

mv /path/to/older/values.yaml iw-k8s-installer/infoworks/values.yaml

Step 3: Change the directory path to iw-k8s-installer.

xxxxxxxxxx

cd iw-k8s-installer

Step 4: Replace the following values with a blank string in the configure.sh file.

xxxxxxxxxx

POSTGRESDB_USERNAME=""

POSTGRESDB_ENCRYPTED_PASSWORD=""

Step 5: Run iw_deploy.sh. Once you receive "Seems like you have already configured Infoworks once. Do you want to override? y/n Default: n", enter “Y”. This will prompt the user to provide input for the values that were blank in the previous step. The script will then replace the infoworks/values.yaml file with the updated values.

xxxxxxxxxx

infoworks@bastion-host:~/iw-k8s-installer$ ./iw_deploy.sh 

NOTE: (Optional) Enable NodeSelector/Toleration and  Custom annotations  etc., by editing values.yaml file manually before deploying infoworks app 

Seems like you have already configured Infoworks once. Do you want to override? y/n  Default: n

y

Checking for basic Pre requisite.

Found HELMv3

Found KUBECTL

Testing Kubernetes basic cluster connection

Validation is done: Kubernetes Cluster is Authorized 

qa-531 Namespace already exists

​

Input postgresDB Username for Infoworks Setup. Assuming the user have permissions to create databases if doesn't exist. 

updated-postgresuser

Input the Postgres user password for Infoworks database Setup. Infoworks will encrypt the Postgres password.

Upgrade INFOWORKS ...

 helm upgrade release-name ./infoworks --values ./infoworks/values.yaml -n namespace

Step 6: Run the following command to upgrade by specifying your namespace and helm release name according to the values given in the configure.sh file.

xxxxxxxxxx

helm upgrade $IW_RELEASE_NAME ./infoworks --values ./infoworks/values.yaml -n $IW_NAMESPACE

Method 2

To update PostgresDB encrypted passwords, you can directly modify the values.yaml file.

Step 1: Download and untar the Infoworks Kubernetes Template, if not already present, according to the iwx-version in your existing deployment.

xxxxxxxxxx

version="5.3.1"

major_version=$(echo $version | cut -d '.' -f 1,2)

wget https://iw-saas-setup.s3.us-west-2.amazonaws.com/$major_version/iwx_installer_k8s_$version.tar.gz

tar xzf iwx_installer_k8s_$version.tar.gz

Step 2: If a new template was downloaded, replace the iw-k8s-installer/infoworks/values.yaml with the older file.

xxxxxxxxxx

mv /path/to/older/values.yaml iw-k8s-installer/infoworks/values.yaml

Step 3: Change the directory path to iw-k8s-installer.

xxxxxxxxxx

cd iw-k8s-installer

Step 4: Generate the encrypted passwords as needed. To generate any encrypted string, execute the following command.

xxxxxxxxxx

encrypted-postgres-password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<password>")

This generates your passwords in a secure encrypted format, which has to be provided in the following steps.

Step 5: Replace the following yaml keys with the new values in the infoworks/values.yaml file, if needed.

xxxxxxxxxx

databases:

postgresDB

  auth:

    username: "postgres-username"

    encryptedPostgresPass: "encrypted-postgres-password"

Step 6: Run the following command to upgrade by specifying your namespace and helm release name according to the installed kubernetes deployment specifications.

xxxxxxxxxx

helm upgrade $IW_RELEASE_NAME ./infoworks --values ./infoworks/values.yaml -n $IW_NAMESPACE

Updating Encrypted Passwords Stored as a Separate Secret

To update the PostgresDB password:

Step 1: Run the following commands from the iw-k8s-installer directory.

xxxxxxxxxx

$ encrypted_postgres_password=$(./infoworks_security/infoworks_security.sh --encrypt -p "<postgres-password>" | xargs echo -n | base64 -w 0)

$ IW_NAMESPACE=<IW_NAMESPACE> 

$ POSTGRESDB_SECRET_NAME=<POSTGRESDB_SECRET_NAME>

$ kubectl patch secret -n ${IW_NAMESPACE} ${POSTGRESDB_SECRET_NAME} --type='json' -p="[{'op' : 'replace' ,'path' : '/data/POSTGRES_PASS' ,'value' : '${encrypted_postgres_password}'}]"

Step 2: Restart the orchestrator and orchestrator-scheduler pods.

xxxxxxxxxx

kubectl get pods -n ${IW_NAMESPACE} --no-headers=true | awk '/-orchestrator-/{print $1}' | xargs  kubectl delete -n ${IW_NAMESPACE} pod

Limitations

MongoDB Limitations

With HA enabled, scaling the pods from higher to lower has the following limitations:

• Pods need to be manually deleted from replication configuration.
• Disabling HA to Non-HA is not supported once HA is enabled.

Database Limitations

Applicable to PostgresDB, MongoDB, and RabbitMQ.

• PVC’s size can’t be decreased.
• Increasing a PVC’s size requires downtime.
• After downscaling pods, the extra PVCs needs to be manually deleted.

PostgresDB Limitations

In the current HA architecture, on Postgres connection disruption, airflow is unable to reconnect via new connection. Furthermore, the current Postgres proxy is too simplistic to handle connection pools. Hence, if a Postgres master goes down, all running workflows will fail.