Skip to main content
Last updated on

Create a Meta Ads ingestion pipeline

Beta

The Meta Ads connector is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.

This article describes how to create a Meta Ads ingestion pipeline using Databricks Lakeflow Connect. The following interfaces are supported:

  • Databricks Asset Bundles
  • Databricks APIs
  • Databricks SDKs
  • Databricks CLI

The Meta Ads connector doesn't currently support UI-based pipeline creation.

Before you begin​

To create an ingestion pipeline, you must meet the following requirements:

  • Your workspace must be enabled for Unity Catalog.

  • Serverless compute must be enabled for your workspace. See Serverless compute requirements.

  • If you plan to create a new connection: You must have CREATE CONNECTION privileges on the metastore.

    If the connector supports UI-based pipeline authoring, an admin can create the connection and the pipeline at the same time by completing the steps on this page. However, if the users who create pipelines use API-based pipeline authoring or are non-admin users, an admin must first create the connection in Catalog Explorer. See Connect to managed ingestion sources.

  • If you plan to use an existing connection: You must have USE CONNECTION privileges or ALL PRIVILEGES on the connection object.

  • You must have USE CATALOG privileges on the target catalog.

  • You must have USE SCHEMA and CREATE TABLE privileges on an existing schema or CREATE SCHEMA privileges on the target catalog.

To ingest from Meta Ads, see Set up Meta Ads as a data source.

Supported objects​

The Meta Ads connector supports ingesting the following objects:

  • ads
  • ad_sets
  • campaigns
  • ad_images
  • ad_insights
  • ad_creatives
  • custom_audiences
  • ad_videos
  • custom_conversions

For the ad_insights object, you can configure breakdowns and action breakdowns to analyze performance data at different granularity levels (account, campaign, ad set, or ad).

Create the ingestion pipeline​

Permissions required: USE CONNECTION or ALL PRIVILEGES on a connection.

This step describes how to create the ingestion pipeline. Each ingested table is written to a streaming table with the same name.

  1. Import the following notebook into your workspace:

    Create a Meta Ads ingestion pipeline

    Open notebook in new tab

  2. Leave the default values in cell 1. Don't modify this cell.

  3. If you want to ingest all objects from your Meta Ads account, modify the schema spec in cell 2. If you only want to ingest specific objects, delete cell 2 and modify the table spec in cell 3 instead.

    Don't modify channel. This must be PREVIEW.

    Cell 2 values to modify:

    • name: A unique name for the pipeline.
    • connection_name: The Unity Catalog connection that stores the authentication details for Meta Ads.
    • source_schema: Your Meta Ads account ID.
    • destination_catalog: A name for the destination catalog that will contain the ingested data.
    • destination_schema: A name for the destination schema that will contain the ingested data.
    • scd_type: The SCD method to use: SCD_TYPE_1 or SCD_TYPE_2. The default is SCD type 1. For more information, see Enable history tracking (SCD type 2).

    Cell 3 values to modify:

    • name: A unique name for the pipeline.
    • connection_name: The Unity Catalog connection that stores the authentication details for Meta Ads.
    • source_schema: Your Meta Ads account ID.
    • source_table: The object name to ingest (for example, ads, campaigns, or ad_insights).
    • destination_catalog: A name for the destination catalog that will contain the ingested data.
    • destination_schema: A name for the destination schema that will contain the ingested data.
    • scd_type: The SCD method to use: SCD_TYPE_1 or SCD_TYPE_2. The default is SCD type 1. For more information, see Enable history tracking (SCD type 2).

    For ad_insights, you can configure additional parameters:

    • breakdown: Optional breakdown dimensions (for example, age, gender, country).
    • action_breakdown: Optional action breakdown dimensions (for example, action_type, action_destination).
    • granularity: Granularity level for insights: account, campaign, ad_set, or ad. Default is ad.

  4. Click Run all.

Next steps​

Additional resources​