Ingestion

Teradata

SQL Server

Oracle

Netezza

Vertica

PostgreSQL

MySQL

DB2 zOS

DB2 LUW

DB2 AS400

Generic JDBC

Salesforce

JSON

Kafka

Confluent Cloud

CData Drivers

Delimited Files

Fixed Width Files

COBOL Copybook

Parquet Files

Snowflake

Ingest from Public buckets

Version Details

Version Tested on

Teradata 15.10,
TPT 16.20, TPT 17

Microsoft SQL Server
2016 (RTM-GDR)
(KB3194716) -
13.00.5026 (X64)

Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

Netezza 7.2

Spark Vertica
Connector 3.3.2
 
Vertica JDBC 12.0.0

PostgreSQL 13.10,
AWS RDS 11

MySQL 5.5.68

DB2 11

DB2 SQL110144
LUW on linux

-

Not Applicable

API version 43.0

Apache
Kafka 2.11

Confluent
Cloud 6.0

-

-

-

-

Client Jar

Terajdbc4.jar,
TPT 16.20, TPT 17

mssql-jdbc-12.6.0.jre8.jar

ojdbc8.jar

nzjdbc.jar

spark-vertica-
connector-slim3.3.2.jar

vertica-jdbc-12.0.0-0.jar

postgresql-42.7.3.jar

mariadb-java-client-2.6.2.jar

jcc-11.5.0.0.jar

jcc-11.5.0.0.jar

jt400-10.6

Source specific

force-partner-api 54.0.0

kafka-clients-3.5.1.jar

kafka-clients-3.5.1.jar,kafka_2.13-6.1.3-ce.jar

-

Crawl Mechanisms

Crawl Mechanisms

JDBC, TPT

JDBC, Log based

JDBC, Log based

-

vertica-jdbc-12.0.0-0.jar

JDBC

JDBC

JDBC

JDBC

JDBC

JDBC

REST API,
Bulk API

Sample file

Sample message

Sample messages
(AVRO, JSON),
Schema Registry
(AVRO)

JDBC

Sample file

Sample file

Sample file

Sample file

Spark

-

Schema Crawl

Metadata Crawl

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

History Loads

Full load

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Segmented Load

Yes

Yes

Yes

-

-

Yes

Yes

Yes

Yes

Yes

Yes

-

-

-

-

Yes

-

-

-

-

Yes

-

Incremental Loads

Watermark based incremental load

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Merge (for Delta Storage Format)

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Append Mode

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Rolling Window

Yes

Yes

Yes

-

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

-

-

Yes

-

-

-

-

Yes

-

Datatype Mapping

Datatype Mapping

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Schema Sync

Automatic Schema Sync

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

No

No

No

Yes

-

Data Masking

Pre-ingestion hook

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Server side data masking - Fixed

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

-

Server side data masking - Regex

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

-

Reconciliation

Data Reconciliation

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

-

Split by

Split by

Yes

Yes

Yes

No

No

Yes

Yes

Yes

Yes

Yes

Yes

No

-

No

No

Yes

-

-

-

-

Yes

-

Derived Split by

Yes

Yes

Yes

No

No

Yes

Yes

Yes

Yes

Yes

Yes

No

-

No

No

Yes

-

-

-

-

Yes

-

Partitioning

Derived Partition

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

No

No

No

No

Yes

-

Hierarchical

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Bucketing (secondary partitioning)

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

-

Partitioning

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Storage Format

Parquet Storage format

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Avro

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

ORC

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Delta

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Delimited Text Files

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

UniForm

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Sample

Refresh data sample

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

-

File compression

Zip

-

-

-

-

-

-

-

-

-

-

-

-

No

-

-

-

No

No

No

No

-

-

Gzip

-

-

-

-

-

-

-

-

-

-

-

-

No

-

-

-

Yes

Yes

No

No

-

-

Tar

-

-

-

-

-

-

-

-

-

-

-

-

No

-

-

-

No

No

No

No

-

-

File Ingestion Features

Control file support

-

-

-

-

-

-

-

-

-

-

-

-

No

-

-

-

No

No

No

No

-

-

Multiline files

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

Yes

Yes

Yes

Yes

-

-

File Archival

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

Yes

Yes

No

No

-

-

File name as a column

-

-

-

-

-

-

-

-

-

-

-

-

No

-

-

-

Yes

Yes

Yes

No

-

-

File name derived partition

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

No

No

No

No

-

-

Include / exclude regex

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

Yes

Yes

Yes

No

-

-

Cloud Support

S3

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

Yes

Yes

Yes

Yes

-

No

GCS

-

-

-

-

-

-

-

-

-

-

Yes

Yes

Yes

Yes

Yes

-

No

WASB, ADLS

-

-

-

-

-

-

-

-

-

-

-

-

Yes

-

-

-

Yes

Yes

Yes

Yes

-

Yes

Filesystem Support

DBFS

-

-

-

No

No

-

-

-

-

-

-

-

Yes

No

No

-

Yes

Yes

Yes

Yes

-

-

SFTP

-

-

-

No

No

-

-

-

-

-

-

-

Yes

No

No

-

Yes

Yes

Yes

Yes

-

-

Query Based Ingestion

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

-

-

-

Yes

-

-

-

-

Yes

-

Error Tables

No

No

No

Yes

Yes

No

No

No

No

No

No

Yes

Yes

Yes

Yes

No

Yes

Yes

No

Yes

No

-

History Table

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

-

Data Quality check

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

No

No

Yes

Yes

Yes

No

No

No

-