Onboarding Data

Let's start ingesting the sample CSV source files!

The ingestion process includes the following steps:

• Setting the Source
• Mapping the File
• Configuring the Table
• Onboarding the Data

Setting the Source

• Click the Data Catalog menu and click the Ingest button for the source created.
• In the Source Setup page, enter the source configuration details and click Save.

Sample Values

• Source Base Path: /FileStore/tables/input/getting_started_source/

Mapping the File

• Click the File Mapping tab and click the Add Table button.
• In the File Location page, enter the table details and click Save.
• In the File Mapping page, enter the table mapping details and click Save & Crawl Schema. You can also view the sample data of the table.

Sample Values

• Table Name: customers
• Target Table Name: customers
• Source Path: /customers
• Relative Target Path: /customers

Configuring the Table

• Click the Configure Tables tab.
• In the General Configurations page, enter the table ingestion configuration details and click Save.

Sample Values

• Ingest Type: Full Refresh
• Natural Keys: CUSTOMER_ID
• Storage Format: Read Optimized
• Target Table Name: customers

Onboarding the Data

• Click the Ingest Data tab.
• Select the table and click the Ingest button.
• In the Ingest page, enter the ingestion job details and click Ingest.
• Click OK in the confirmation pop-up window.

The ingestion job will be run and you can see the progress in the Ingestion Jobs page.

You can also view the metrics in the Ingestion Metrics page.

Sample Values

• Job Name: customers
• Max. Parallel Tables: 1

That's it! You have completed ingesting your first source. The next process is to build a data model on the ingested source.