Editing Version Parameters
Data Transformation supports parameters in expressions within a pipeline. Navigate to Version Actions > Edit Version Parameters in the pipeline editor. The Pipeline Version Parameters page allows you to add Key and Value for parameters. These parameters can be used in any expression within a pipeline. The value of a parameter can be a constant, an expression, or a reference to another parameter.
The value can be of the following datatypes:
- Numeric: Enter the value. For example, 10.
- Character: Enter the value within single quotes. For example, ‘US’.
- String: Enter the value within double quotes. For example, “US”.
- Variable: Enter the value prefixed with $, followed by the key which has already been defined as a pipeline parameter.
The parameters must be referenced in the following format: $<param_name>. For example, $group_id.
Sample Parameter
Pipeline parameters are useful when a constant is used in multiple expressions. For example, replace null or empty values of all string columns with NULL.
In the above example, null_value_replacement is defined as a pipeline parameter. This provides the flexibility to change the value of the parameter, null_value_replacement, without changing the expressions used in the pipeline.
Pipeline parameters can also be used to override the source table watermark values. Following pipeline parameters can be used to override the values of low and high watermarks:
- Low watermark:
<source_name>.lowWatermark=<low_watermark_value> - High watermark:
<source_name>.highWatermark=<high_watermark_value>
The usage of column names as pipeline parameter values is not supported.