Crawl metadata failing with Response Row size exceeds 64K bytes and is incompatible with the Client software error in a

Crawl metadata failing with "Response Row size exceeds 64K bytes and is incompatible with the Client software" error in a Teradata source.

Problem Description:

Metacrawl of a Teradata source failing with below error if the row size exceeds 64K bytes,

[ERROR] 2020-07-28 10:40:00,512 [main] io.infoworks.saas.ingestion.core.metacrawl.sources.RDBMSMetaCrawler:396 :: Error while fetching metadata for demo_cl

io.infoworks.saas.ingestion.commons.exception.IWException: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 9990] [SQLState HY000] Response Row size exceeds 64K bytes and is incompatible with the Client software.

Root cause:

Error 9990 points that response row size exceeds 64K bytes limit. This issue will occur if the Teradata drivers are in version 15.10.

Solution/Workaround for Infoworks Datafoundry v3.x

Upgrading the Teradata drives to version 16.10 will resolve this issue.

Steps:

• Login to the Infoworks edge node as Infoworks user and navigate to $IW_HOME/lib/ingestion/jdbc.
• Move terajdbc4.jar and tdgssconfig.jar jars out of $IW_HOME/lib/ingestion/jdbc.
• Add below attached jars to $IW_HOME/lib/ingestion/jdbc directory.
• Rerun the metacrawl.

Applicable Infoworks DataFoundry on Databricks versions:

Solution/Workaround for Infoworks Datafoundry v5.x

• Login to the Infoworks edge node as Infoworks user and navigate to/opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/
• Move terajdbc4.jar and tdgssconfig.jar jars out of /opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/
• Add below attached jars to/opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/ directory.
• Rerun the metacrawl.