Title
Message
Create new category
What is the title of your new category?
Edit page index title
What is the title of the page index?
Edit category
What is the new title of your category?
Edit link
What is the new title and URL of your link?
What is the configuration to set the comparator to fetch CDC records
Copy Markdown
Open in ChatGPT
Open in Claude
What is the configuration to set the comparator to fetch CDC records?
In onprem offering(version 2.X-3.X):
The USE_GTE_FOR_CDC configuration allows users to fetch the CDC records based on the use case.
- true: the CDC records will be fetched using the >= comparator. This is the default behavior and must be used for merge use cases to ensure that the data from the last batch is brought again. This is performed when some data of the last batch and timestamp is still being populated in the source system when the ingestion job has finished.
- false: the CDC records will be fetched using the > comparator. This behavior must be used for append mode scenarios where the data for the last batch or timestamp in the source system is fully populated and the user does not want the old data again.
In Databricks offering(version 4.X onwards):
The cdc_comparator_key configuration allows users to fetch the CDC records based on the use case.
- >**=: Here the CDC records will be fetched using the **>= comparator. This is the default value of the cdc_comparator_key config and must be used for merge use cases to ensure that the data from the last batch is brought again. This is performed when some data of the last batch and timestamp is still being populated in the source system when the ingestion job has finished.
- >: Here the CDC records will be fetched using the > comparator. This value must be used for append mode scenarios where the data for the last batch or timestamp in the source system is fully populated and the user does not want the old data again.
NOTE: Data might be lost when the > comparator is used. If records with the same batch ID are being inserted in the source system when the ingestion job is running, all the records that are inserted just after the job is run and with the same batch ID will be missed in the next CDC job.
VariableType to search · ESC to discard
GlossaryType to search · ESC to discard
InsertType to search · ESC to discard
No matches
Last updated on
Was this page helpful?
Next to read:
Workflows are going into pending state in InfoworksFor more details, refer to our Knowledge Base and Best Practices!
For help, contact our support team!
© UNIPHORE TECHNOLOGIES 2025 | Confidential
Discard Changes
Do you want to discard your current changes and overwrite with the template?
Archive Synced Block
Message
Create new Template
What is this template's title?
Delete Template
Message