Migrating Pipeline Configurations

For a successful pipeline configuration migration, the sources and tables in the exported file must exactly match the names in the environment where the pipeline configuration is being imported.

This feature allows you to perform the following:

• map table to any source table or pipeline target.
• map a source table to pipeline target and vice versa.
• add to new columns during configuration Import.
• manually configure critical Target Node properties, like HDFS path, in the imported pipeline.

Downloading Pipeline Configurations

Following are the steps to export a pipeline configuration:

1. Click Domains menu and click the domain where the required pipeline is created.
2. On the Summary page of the domain, click the pipeline for which configuration must be migrated.
3. Click the Settings icon and locate the Download Configuration section as follows:

4. Select a version from the drop-down list. By default, the active version will be selected.

5. EnablePretty print configuration to download the configuration file with JSON syntax.
6. Click Download. The version configuration is downloaded.

Uploading Pipeline Configurations

Following are the steps to import pipeline configuration:

1. Click Domains menu and click the domain where the required pipeline is created.
2. On the Summary page of the domain, click the pipeline for which configuration must be migrated.
3. Click the Settings icon and locate the Upload Configuration section as follows:

4. Click Choose A File and select the required pipeline version configuration file.

5. The Import Configuration for Pipeline page is displayed with the table mapping between the imported file and the tables in the domain.

Perform the following:

1. Select the required table from Table in this Domain to map it with the table in the uploaded configuration.

2. Retain properties such as target table name, target schema name, target hdfs location, analytics model name, analytics model hdfs location in import: Enable or disable this option to retain or skip target configurations respectively during the pipeline configuration import.
3. Run Build Tables Metadata job and copy export configuration after upload.
4. Enable this option to build target metadata and copy export configuration for the pipeline targets.
5. Make newly created version active: Enable this option to set the newly created version as Active.
6. Click Import Configurations. The configurations are imported and a success message is displayed.

NOTE Creating pipeline through uploading pipeline configuration will create a new pipeline version by auto incrementing the existing pipeline version number by one. In order to retain version number from the uploaded configuration file, set retain_version_number to true in admin configurations.