Xero connector limitations

Beta

This feature is in Beta. To use it, a workspace admin must turn on Lakeflow Connect for Xero from the Previews page. See Manage Databricks previews.

The managed Xero connector in Lakeflow Connect ingests one Xero organization per pipeline, detects deletes in only some tables, and reingests 11 tables in full on every run.

General software as a service (SaaS) connector limitations

The limitations in this section apply to all SaaS connectors in Lakeflow Connect.

• When you run a scheduled pipeline, alerts don't trigger immediately. Instead, they trigger when the next update runs.
• When a source table is deleted, the destination table is not automatically deleted. You must delete the destination table manually. This behavior is not consistent with Spark Declarative Pipelines on Lakeflow behavior.
• During source maintenance periods, Databricks might not be able to access your data.
• If a source table name conflicts with an existing destination table name, the pipeline update fails.
• Multi-destination pipeline support is API-only.
• You can optionally rename a table that you ingest. If you rename a table in your pipeline, it becomes an API-only pipeline, and you can no longer edit the pipeline in the UI.
• If you select a column after a pipeline has already started, the connector does not automatically backfill data for the new column. To ingest historical data, manually run a full refresh on the table.
• Databricks can't ingest two or more tables with the same name in the same pipeline, even if they come from different source schemas.
• The source system assumes that the cursor columns are monotonically increasing.
• The connector ingests raw data without transformations. Use downstream Spark Declarative Pipelines on Lakeflow pipelines for transformations.

Connector-specific limitations

The limitations in this section apply to the Xero connector.

Organizations

• Create one pipeline for each Xero organization. Each pipeline ingests only the organization that its tenant_id option names.

Historical data

• Set start_datetime to ingest older records. By default, the first sync of each incremental table ingests only records modified in the last 730 days. See Connector options.

Deleted records

• Records deleted in Xero are deleted from the destination table only for accounts, bank_transactions, credit_notes, invoices, manual_journals, payments, and purchase_orders. The connector removes these rows instead of keeping them, so rows that remain in these tables have IsDeleted set to false.
• Run a full refresh to remove deleted items from the destination table. Xero doesn't report item deletions, so incremental syncs keep deleted items.
• Voided documents stay in the destination table with Status set to VOIDED. Archived contacts and removed users also stay in their tables.

Full-refresh tables

• The bank_transfers, branding_themes, budgets, contact_groups, currencies, linked_transactions, organisation, projects, repeating_invoices, tax_rates, and tracking_categories tables reingest all records on every run. Because these tables reread every record, each run uses more Xero API calls than an incremental sync.