Skip to main content
Last updated on

Marketo connector limitations

Beta

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.

General SaaS connector limitations​

The limitations in this section apply to all software as a service (SaaS) connectors in Lakeflow Connect.

  • When you run a scheduled pipeline, alerts don't trigger immediately. Instead, they trigger when the next update runs.
  • When a source table is deleted, the destination table is not automatically deleted. You must delete the destination table manually. This behavior is not consistent with Spark Declarative Pipelines on Lakeflow behavior.
  • During source maintenance periods, Databricks might not be able to access your data.
  • If a source table name conflicts with an existing destination table name, the pipeline update fails.
  • Multi-destination pipeline support is API-only.
  • You can optionally rename a table that you ingest. If you rename a table in your pipeline, it becomes an API-only pipeline, and you can no longer edit the pipeline in the UI.
  • If you select a column after a pipeline has already started, the connector does not automatically backfill data for the new column. To ingest historical data, manually run a full refresh on the table.
  • Databricks can't ingest two or more tables with the same name in the same pipeline, even if they come from different source schemas.
  • The source system assumes that the cursor columns are monotonically increasing.
  • The connector ingests raw data without transformations. Use downstream Spark Declarative Pipelines on Lakeflow pipelines for transformations.

Connector-specific limitations​

The limitations in this section apply to the Marketo connector.

  • The connector supports four table types: leads, static_lists, activity_<typeId> (one table per activity type), and custom objects (one table per custom object). Other Adobe Marketo Engage entities, such as programs, program members, deleted leads, companies, opportunities, and opportunity roles, are not supported.
  • Adobe Marketo Engage limits Bulk Extract to 10 concurrent jobs, shared across every integration on your subscription. If you ingest many tables at the same time, the pipeline can fail. Keep the number of Bulk Extract tables in a pipeline under 10.
  • Adobe Marketo Engage enforces a daily Bulk Extract export quota, which resets at 12:00 AM CST. The default quota is 500 MB per day; you can purchase a higher quota from Adobe Marketo Engage to improve ingestion performance and avoid daily-quota errors. If you exceed the quota, the pipeline fails until the quota resets.
  • Ingesting leads and custom objects requires the updatedAt filter to be enabled for Bulk Extract in your Adobe Marketo Engage instance.
  • The static_lists table is refreshed with a full snapshot on each pipeline update. It does not support incremental ingestion.
  • For the list of supported tables, see Supported source tables.
On this page