Query-based connector limitations

Preview

This feature is in Public Preview.

General limitations

• Query-based connectors query the source on a schedule. They don't provide continuous ingestion. If you need lower latency, use a managed CDC database connector.
• The cursor column must be a single column. Composite cursors (multiple columns combined) aren't supported. The column value must be monotonically increasing. Rows with cursor values at or below the stored high-water mark aren't reingested.
• Rows with a NULL cursor column are not ingested.

API-only features

The following features are supported for query-based connectors, but only using the API:

• Row filtering
• Soft-deletion tracking (deletion_condition)
• Hard-deletion tracking (Beta)
• Append-only ingestion (APPEND_ONLY SCD mode)
• Multi-destination catalog and schema