Skip to main content
Last updated on

Access table and column tags from external engines

Beta

This feature is in Beta. To use it, a workspace admin must turn on Tag Exchange via Iceberg REST Catalog from the Previews page. See Manage Databricks previews.

When tag exchange is enabled, Unity Catalog returns table and column tags as Iceberg labels in the LoadTableResponse. External engines with clients that support the Iceberg labels API can read these labels for data discovery, classification, and governance.

How tag exchange works​

Tag exchange is enabled at the workspace level. When tag exchange is turned on, each LoadTableResponse to an Iceberg client includes table and column tags, except when row filters or column masks apply to the querying principal.

Unity Catalog returns governed and ungoverned tags applied to tables and top-level columns. Table tags also include tags inherited from the schema and catalog. For the same key, the table's value takes precedence over the schema's value, which takes precedence over the catalog's value.

Column tags are direct assignments. They don't include inherited table tags.

In the response, object-labels contains table tags. fields contains column tags identified by Iceberg field ID. Resolve each ID against the returned current schema, not the column's ordinal position.

Requirements​

Before you begin, you need the following:

Permissions to read tags​

The user or service principal used by the external client needs the following privileges:

  • USE CATALOG on the parent catalog.
  • USE SCHEMA on the parent schema.
  • SELECT on the table.
  • EXTERNAL USE SCHEMA, explicitly granted on the schema or parent catalog.

The reading principal doesn't need tag assignment privileges.

Permissions to assign tags​

To assign tags in the example, you must own the object or have all of the following privileges: APPLY TAG on the object, USE CATALOG on the parent catalog, and USE SCHEMA on the parent schema. Governed tags also require ASSIGN and an allowed value. See Apply tags to Unity Catalog securable objects.

Example: Read tags from an external engine​

The following example tags the unity.public.customers table with domain=finance and its ssn column with pii=ssn in Unity Catalog, then reads both tags from an external engine.

Apply tags in Unity Catalog​

With the required tagging permissions, run these statements:

  1. Assign the table tag:

    SQL
    ALTER TABLE unity.public.customers
    SET TAGS ('domain' = 'finance');

  2. Assign the column tag:

    SQL
    ALTER TABLE unity.public.customers
    ALTER COLUMN ssn SET TAGS ('pii' = 'ssn');

Access tags from Apache Spark​

This example uses Apache Iceberg 1.12.0 with Spark 4.1.3 and Scala 2.13. Iceberg 1.12.0 includes the Iceberg labels API.

Configure Spark access as described in Use Iceberg tables with Apache Spark. Use org.apache.iceberg:iceberg-spark-runtime-4.1_2.13:1.12.0 and the cloud-specific Iceberg bundle at version 1.12.0. Name the Spark catalog uc and set its warehouse to the Unity Catalog catalog unity.

In the configured Spark Scala shell, load the table and read its labels:

Scala
import scala.jdk.CollectionConverters._
import org.apache.iceberg.SupportsLabels
import org.apache.iceberg.spark.Spark3Util

val table = Spark3Util.loadIcebergTable(spark, "uc.public.customers")
val labels = table.asInstanceOf[SupportsLabels].labels()

labels.objectLabels().asScala.foreach { case (key, value) =>
  println(s"table: $key=$value")
}

labels.fields().asScala.foreach { field =>
  val column = table.schema().findColumnName(field.fieldId())
  field.labels().asScala.foreach { case (key, value) =>
    println(s"column $column: $key=$value")
  }
}

The output includes the tags from the previous step:

Output
table: domain=finance
column ssn: pii=ssn

Access tags using the REST API​

The following curl example loads unity.public.customers using the REST API. It's based on the Iceberg REST API example.

Set DATABRICKS_HOST to your workspace URL, including https:// without a trailing slash, and set OAUTH_TOKEN to a valid OAuth token for the authorized principal.

Bash
curl -X GET \
  -H "Authorization: Bearer $OAUTH_TOKEN" \
  -H "Accept: application/json" \
  "$DATABRICKS_HOST/api/2.1/unity-catalog/iceberg-rest/v1/catalogs/unity/namespaces/public/tables/customers"

The LoadTableResponse includes the tags in labels. This excerpt assumes ssn has Iceberg field ID 3:

JSON
{
  "labels": {
    "object-labels": {
      "domain": "finance"
    },
    "fields": [
      {
        "field-id": 3,
        "labels": {
          "pii": "ssn"
        }
      }
    ]
  }
}

Limitations​

Tags aren't returned to users who are subject to row filters or column masks, including those applied through attribute-based access control (ABAC). Users exempt from these policies can receive tags.

To enforce these policies in supported external engines, see cross-engine ABAC. Cross-engine ABAC enforces policies without returning tags to non-exempt users.

Additional resources​

On this page