Onboarding Data

Let's start ingesting the sample CSV source files!

The ingestion process includes the following steps:

• Setting the Source

• Mapping the File

• Configuring the Table

• Onboarding the Data

Setting the Source

• Click the Data Catalog menu and click the Ingest button for the source created.

• In the Source Setup page, enter the source configuration details and click Save.

Sample Values

• Source Base Path: /FileStore/tables/input/getting_started_source/

Mapping the File

• Click the File Mapping tab and click the Add Table button.

• In the File Location page, enter the table details and click Save.

• In the File Mapping page, enter the table mapping details and click Save & Crawl Schema. You can also view the sample data of the table.

Sample Values

• Table Name: customers

• Target Table Name: customers

• Source Path: /customers

• Relative Target Path: /customers

Configuring the Table

• Click the Configure Tables tab.

• In the General Configurations page, enter the table ingestion configuration details and click Save.

Sample Values

• Ingest Type: Full Refresh

• Natural Keys: CUSTOMER_ID

• Storage Format: Read Optimized

• Target Table Name: customers

Onboarding the Data

• Click the Ingest Data tab.

• Select the table and click the Ingest button.

• In the Ingest page, enter the ingestion job details and click Ingest.

• Click OK in the confirmation pop-up window.

The ingestion job will be run and you can see the progress in the Ingestion Jobs page.

You can also view the metrics in the Ingestion Metrics page.

Sample Values

• Job Name: customers

• Max. Parallel Tables: 1

That's it! You have completed ingesting your first source. The next process is to build a data model on the ingested source.