Title
Message
Create new category
What is the title of your new category?
Edit page index title
What is the title of the page index?
Edit category
What is the new title of your category?
Edit link
What is the new title and URL of your link?
Incremental ingestion is failing with unable to find the column of the target table error
Copy Markdown
Open in ChatGPT
Open in Claude
Incremental ingestion is failing with "unable to find the column of the target table" error
Description:
Incremental ingestion is failing with unable to find the column of the target table. Error message in the databricks log looks something like below.
Caused by: org.apache.spark.sql.AnalysisException: Unable to find the column 'iwDerivedColumn' of the target table from the INSERT columns: BUSINESS_UNIT, PROJECT_ID, ACTIVITY_ID, RESOURCE_ID, RESOURCE_ID_FROM, BUSINESS_UNIT_GL, JOURNAL_ID, JOURNAL_DATE, UNPOST_SEQ, JOURNAL_LINE, FISCAL_YEAR, ACCOUNTING_PERIOD, ACCOUNT, ALTACCT, DEPTID, OPERATING_UNIT, PRODUCT, FUND_CODE, CLASS_FLD, PROGRAM_CODE, BUDGET_REF, AFFILIATE, AFFILIATE_INTRA1, AFFILIATE_INTRA2, CHARTFIELD1, CHARTFIELD2, CHARTFIELD3, BUS_UNIT_GL_FROM, CURRENCY_CD, STATISTICS_CODE, LEDGER_GROUP, ANALYSIS_TYPE, RESOURCE_TYPE, RESOURCE_CATEGORY, RESOURCE_SUB_CAT, RES_USER1, RES_USER2, RES_USER3, RES_USER4, RES_USER5, TRANS_DT, ACCOUNTING_DT, OPRID, DTTM_STAMP, JRNL_LN_REF, OPEN_ITEM_STATUS, LINE_DESCR, JRNL_LINE_STATUS, JOURNAL_LINE_DATE, FOREIGN_CURRENCY, RT_TYPE, FOREIGN_AMOUNT, RATE_MULT, RATE_DIV, CUR_EFFDT, PROCESS_INSTANCE, PC_DISTRIB_STATUS, GL_DISTRIB_STATUS, PROJ_TRANS_TYPE, PROJ_TRANS_CODE, RESOURCE_STATUS, DESCR, SYSTEM_SOURCE, UNIT_OF_MEASURE, EMPLID, EMPL_RCD, SEQ_NBR, TIME_RPTG_CD, JOBCODE, COMPANY, BUSINESS_UNIT_AP, VENDOR_ID, VOUCHER_ID, VOUCHER_LINE_NUM, APPL_JRNL_ID, PYMNT_CNT, DST_ACCT_TYPE, PO_DISTRIB_STATUS, BUSINESS_UNIT_PO, REQ_ID, REQ_LINE_NBR, REQ_SCHED_NBR, REQ_DISTRIB_NBR, PO_ID, DUE_DATE, LINE_NBR, SCHED_NBR, DISTRIB_LINE_NUM, AM_DISTRIB_STATUS, BUSINESS_UNIT_AM, ASSET_ID, PROFILE_ID, COST_TYPE, BOOK, INCENTIVE_ID, MSTONE_SEQ, CONTRACT_NUM, CONTRACT_LINE_NUM, CONTRACT_PPD_SEQ, BI_DISTRIB_STATUS, BUSINESS_UNIT_BI, BILLING_DATE, INVOICE, REV_DISTRIB_STATUS, BUSINESS_UNIT_AR, CUST_ID, ITEM, ITEM_LINE, ITEM_SEQ_NUM, DST_SEQ_NUM, BUSINESS_UNIT_IN, SCHED_LINE_NO, DEMAND_LINE_NO, INV_ITEM_ID, PAY_END_DT, BUSINESS_UNIT_OM, ORDER_NO, ORDER_INT_LINE_NO, EX_DOC_ID, EX_DOC_TYPE, RESOURCE_QUANTITY, RESOURCE_AMOUNT, BUDGET_HDR_STATUS, KK_AMOUNT_TYPE, KK_TRAN_OVER_FLAG, KK_TRAN_OVER_OPRID, KK_TRAN_OVER_DTTM, BUDGET_OVER_ALLOW, BUDGET_LINE_STATUS, BUDGET_DT, LEDGER, BD_DISTRIB_STATUS, BUSINESS_UNIT_BD, FA_STATUS, ACCOUNTING_DT_Year, ziw_target_timestamp, ziw_is_deleted. INSERT clause must specify value for all the columns of the target table.;Root Cause:
This happens when CDC brings the column which is not there in the already created target table.
Solution:
To resolve this one needs to truncate, recrawl, and reingest the data for that particular table.
Applicable versions:
All versions of Datafoundry.
VariableType to search · ESC to discard
GlossaryType to search · ESC to discard
InsertType to search · ESC to discard
No matches
Last updated on
Was this page helpful?
Next to read:
Incremental ingestion job failing with column modified error for teradata tables when the jdbc driver was upgraded to neFor more details, refer to our Knowledge Base and Best Practices!
For help, contact our support team!
© UNIPHORE TECHNOLOGIES 2025 | Confidential
Discard Changes
Do you want to discard your current changes and overwrite with the template?
Archive Synced Block
Message
Create new Template
What is this template's title?
Delete Template
Message