Troubleshooting
Interactive Service Timeout/Error in Dataproc Environment
If the interactive service times out or an error occurs on the Dataproc environment, perform the following:
- Log in to the admin console and check if the interactive server is running.
- If the interactive server is not running, start the interactive server.
- If the interactive server is running, log in to the edge node machine using SSH and stop the DT service using the following command:
/opt/infoworks/bin/stop.sh dt - Start the Data Transformation service using the following command:
/opt/infoworks/bin/start.sh dt - The interactive mode in the Infoworks edge node must work now.
Target Overwrite Failure for Delta Storage Format
If data overwrite fails for the delta storage format, set the following advanced configuration to overwrite the schema for the delta storage format:
Key: dt_spark_df_writer_additional_options
Value: overwriteSchema=true
Pipeline Build Failure for Delta Format on Persistent Cluster
Sometimes, when delta format is used for target node, the pipeline build fails on persistent clusters for Azure Databricks environment. You will receive the following error message "org.apache.spark.sql.AnalysisException: Table or view not found". If you encounter this issue, try restarting the persistent cluster, if it still does not work, build the pipeline again with ephemeral cluster.
Pipeline Target Metadata and Build Failure for Existing Pipeline Target Path
If Target Metadata build or Pipeline Build job fails for an existing Pipeline Target Relative path with the error : Error: Path Exist : [object Object], set the following advanced configuration to skip the HDFS Path validation for the required pipeline(s):
Key: PIPELINE_BUILD_HDFS_PATH_SKIP_VALIDATION
Value: single or list of pipeline Ids separated by comma
Pipeline Build Failure for Oracle Sink Target
If Pipeline Build job fails for an oracle sink target with the error : ORA-01882: timezone region not found, set the following advanced configuration to the pipeline:
Key: dt_target_rowcount_enabled
Value: false
and add the following additional params in oracle target data connection :
oracle.jdbc.timezoneAsRegion=false