Apply tags to managed ingestion pipelines

Preview

This feature is in Public Preview.

Applies to: API-based pipeline authoring SaaS connectors Database connectors

Learn how to apply tags to managed ingestion pipelines to organize resources, track ownership, and attribute costs.

Pipeline tags are different from serverless budget policies. Pipeline tags are metadata on the pipeline resource, whereas serverless budget policies apply tags to serverless compute billing records. See Attribute usage with serverless budget policies.

How pipeline tags work

Pipeline tags are custom key-value pairs that you define in your pipeline specification. Use tags to:

• Group pipelines by environment, project, or team.
• Identify the team or individual responsible for a pipeline.
• Associate pipeline usage with specific cost centers or budgets.
• Filter and automate operations based on tag values.

Tags propagate to system tables and billing records, allowing you to query and analyze pipeline metadata alongside usage data.

Add or update pipeline tags

Add or update pipeline tags with a tags object in your pipeline specification. For example,

JSON

{
  "name": "sales-data-pipeline",
  "catalog": "prod",
  "target": "sales",
  "serverless": true,
  "tags": {
    "environment": "production",
    "owner": "data-engineering-team",
    "costcenter": "engineering-analytics",
    "project": "sales-analytics"
  },
  "ingestion_definition": {
    "connection_name": "salesforce-prod",
    "objects": [
      {
        "table": {
          "source_schema": "salesforce",
          "source_table": "Account",
          "destination_catalog": "prod",
          "destination_schema": "sales",
          "table_configuration": {
            "scd_type": "SCD_TYPE_1"
          }
        }
      }
    ]
  }
}

View pipeline tags

To view tags for a specific pipeline, query the pipelines system table. See Jobs system table reference.

Additional resources

• Monitor managed ingestion pipeline cost