Shopify connector reference
This feature is in Beta. To use it, a workspace admin must turn on Lakeflow Connect for Shopify from the Previews page. See Manage Databricks previews.
This page has reference information for the managed Shopify connector, including supported source tables and destination table schemas.
Supported source tables
The Shopify connector supports the following source tables, all under the default source schema:
Source table | Primary key | Description | Sync mode | Cursor field |
|---|---|---|---|---|
|
| Products in your store, including title, status, vendor, and total inventory. | Incremental |
|
|
| Orders placed in your store, including financial status, fulfillment status, and currency. | Incremental |
|
|
| Customer profiles, including name, contact details, and order count. | Incremental |
|
|
| Draft orders created in your store, including status and completion time. | Incremental |
|
|
| Product collections in your store, including title, handle, and sort order. | Incremental |
|
|
| Manually curated product collections. | Incremental |
|
|
| Automated product collections, including the rule set that determines membership. | Incremental |
|
|
| Blog articles in your online store, including publication status. | Incremental |
|
|
| Online store pages, including publication status. | Incremental |
|
|
| Product variants, including SKU, price, barcode, and inventory quantity. | Incremental |
|
|
| Blogs in your online store. | Incremental |
|
|
| Inventory items, including SKU and whether the item is tracked. | Incremental |
|
|
| Code-based discounts, including the discount configuration. | Incremental |
|
|
| Automatic discounts, including the discount configuration. | Incremental |
|
|
| Store locations, including whether each ships inventory and fulfills online orders. | Full refresh only | |
|
| Store profile, including domain, currency, and timezone. | Full refresh only | |
|
| Tender transactions, including payment method and amount. | Incremental |
|
|
| Associations between products and custom collections. | Full refresh only | |
|
| Payment disputes and chargebacks, including status and evidence deadlines. | Full refresh only | |
|
| Abandoned checkouts, including totals and the recovery URL. | Incremental |
|
|
| Fulfillment orders, including status and assigned location. | Incremental |
|
|
| Image files in your store's media library. | Incremental |
|
|
| All media files in your store's media library, including images, videos, and 3D models. | Incremental |
|
|
| Countries configured for shipping and tax, including provinces. | Full refresh only | |
|
| Shopify Payments balance transactions, including amount, fee, and net. | Incremental |
|
|
| Order payment transactions, including kind, gateway, status, and amount. | Incremental |
|
|
| Refunds issued on orders, including restock and line-item details. | Incremental |
|
|
| Order fulfillments, including tracking and shipment status. | Incremental |
|
|
| Line items included in order refunds, including quantity and restock type. | Incremental |
|
|
| Adjustments applied to order refunds, including amount and reason. | Incremental |
|
|
| Media attached to product variants, including images, videos, and 3D models. | Incremental |
|
|
| Shipping lines on orders, including carrier and pricing. | Incremental |
|
|
| Sales agreements on orders, including type and reason. | Incremental |
|
|
| Fraud risk assessments for orders, including the recommendation. | Incremental |
|
|
| Customer addresses, including city, province, country, and postal code. | Incremental |
|
|
| Line items on fulfillment orders, including total and remaining quantity. | Incremental |
|
|
| Redeem codes for code-based discounts, including usage count. | Incremental |
|
|
| Associations between collections and the products they contain. | Full refresh only | |
|
| Inventory levels per item and location, including available, on-hand, and committed quantities. | Full refresh only |
Connector options
See Examples for usage.
Option | Type | Scope | Required | Applies to | Description |
|---|---|---|---|---|---|
| String ( | Pipeline | No | All incremental tables | Earliest date and time from which to ingest data. Defaults to 365 days before the first sync. Applies only to the first sync of each table; later syncs resume from the stored cursor. |
Destination table schemas
products
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
orders
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
customers
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
draft_orders
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
collections
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
custom_collections
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
smart_collections
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
articles
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
pages
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
product_variants
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
blogs
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
inventory_items
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
discount_codes
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
discount_automatic
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
locations
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
shop
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
tender_transactions
Primary key: id
Cursor field: processedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
collects
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
disputes
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
abandoned_checkouts
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
fulfillment_orders
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
media_image
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
media
Primary key: id
Cursor field: updatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
countries
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
balance_transactions
Primary key: id
Cursor field: transactionDate
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
transactions
Primary key: id
Cursor field: created_at
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
order_refunds
Primary key: id
Cursor field: created_at
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
fulfillments
Primary key: id
Cursor field: updated_at
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
order_line_refund
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
order_adjustment
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
product_variant_media
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
order_shipping_line
Primary key: id
Cursor field: parentUpdatedAt
Removed shipping lines are captured through the isRemoved column.
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
order_agreements
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
order_risks
Primary key: order_id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
customer_address
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
fulfillment_order_line_item
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
discount_redeem_code
Primary key: id
Cursor field: parentUpdatedAt
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
This table doesn't support SCD type 2 because it has VARIANT columns.
collection_product
Primary key: collection_id, product_id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
inventory_level
Primary key: id
Sync mode: Full refresh only
Field | Data type |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Required access scopes
The Shopify app must be authorized for the following Admin API access scopes. The scopes the connector requires depend on which tables you ingest. For more information, see the Shopify access scopes reference.
Access scope | Required for |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- To ingest orders and order-derived tables (such as
orders,transactions,order_refunds, andfulfillments) older than 60 days, the app also needs theread_all_ordersscope, combined withread_orders. Shopify grantsread_all_ordersby approval. Without it,read_ordersreturns only orders from the last 60 days. - The
balance_transactionsanddisputestables require Shopify Payments on the store. - Some
customers,customer_address, and order fields are protected customer data. Shopify returns these fields from non-development stores only after the app meets the protected customer data requirements. See the Shopify access scopes reference.