Exporting Data to Custom Target

Prerequisites

Perform the following:

• Navigate to the IW_HOME/opt/infoworks/conf/conf.properties file.
• Set the extension path in the user_extensions_base_path configuration. For example, user_extensions_base_path=/opt/infoworks/extensions/.
• Ensure that the Infoworks user has write access to the folder.
• Restart the UI and Data Transformation services.

Registering Custom Target

Registering Custom Target is same as registering Custom Transformation except that you must select the Extension Type as Custom Target. For details see the Registering Custom Transformations and Performing Data Transformation sections.

Adding Custom Target to Domain

Adding Custom Target to domain is same as adding Custom Transformation to domain. For details see the Adding Custom Transformation to Domain section.

Infoworks already provides snowflake as a custom target. For details, see Using Pipeline Targets.

Using Custom Target

Following are the steps to apply Custom Target node in pipeline:

1. Double-click the Custom Target node. The properties page is displayed.
2. Click Edit Properties, select the Extension Name as snowflake.
3. Enter a Key and Value which will be added to the custom target node and will be available via UserProperties class from the public void initialiseContext(SparkSession sparkSession, UserProperties userProperties, ProcessingContext processingContext) API

Note : Map of key vault credentials which have been added in the custom target configuration can be accessed via processingContext.getAdditionalConf() in the custom target interface.

4. Set the following keys:

• buildMode: overwrite or merge
• insert-overwrite: Overwrites all the rows based on combination of natural keys (OR condition) and inserts new rows.
• tableName: table name of the snowflake target
• naturalKeyColumns: comma-separated natural keys
• sfConfFile: the configuration property file in the edge node which includes the connection details of snowflake.

4. Click Save.