Efficient column updates in Delta Lake
This feature is in Beta. To use it, a workspace admin must turn on Efficient Column Updates from the Previews page. See Manage Databricks previews.
Efficient Column Updates (ECU) lets you add and backfill columns in a Delta Lake table without rewriting unchanged columns. Use it to populate new features, embeddings, and other derived values in existing datasets.
Adding a column changes the table schema without rewriting existing data. Populating that column with values can require rewriting the data files containing the affected rows, including columns that have not changed. Efficient Column Updates avoids those unchanged-column rewrites by writing the updated column values to separate Parquet files.
This is particularly useful when you add a small number of columns to a table that contains large payloads or embeddings. Queries continue to return one logical table, combining existing data with the updated column values.
The following diagram compares adding and backfilling a column. Gray blocks represent existing column data, green blocks represent new column data, and red outlines identify rewritten data.

- A: Without ECU, a backfill replaces the original data files (dashed outlines) with files containing both the unchanged columns and the new column.
- B: With ECU, the original files are retained and the new column values are written to separate files.
Requirements
- Use Databricks Runtime 19 or above to read from or write to a table with Efficient Column Updates enabled.
- Use a Unity Catalog managed Delta Lake table with catalog commits enabled.
- Enable row tracking and deletion vectors on the table.
- Use column mapping in
idmode ('delta.columnMapping.mode' = 'id'). This mode must be set when the table is created. - Enable the Beta from the workspace Previews page.
Enable Efficient Column Updates
You can enable Efficient Column Updates when you create a table or on an existing table that meets the requirements. Review the limitations before enabling it.
To create a table with Efficient Column Updates enabled, set the delta.enableColumnUpdates-preview table property together with the properties for the required features:
CREATE TABLE documents (
document_id BIGINT,
description STRING
)
TBLPROPERTIES (
'delta.feature.catalogManaged' = 'supported',
'delta.enableRowTracking' = 'true',
'delta.enableDeletionVectors' = 'true',
'delta.columnMapping.mode' = 'id',
'delta.enableColumnUpdates-preview' = 'true'
);
For an existing table that already uses column mapping in id mode, enable catalog commits, row tracking, and deletion vectors if they are not already enabled:
ALTER TABLE documents SET TBLPROPERTIES (
'delta.feature.catalogManaged' = 'supported',
'delta.enableRowTracking' = 'true',
'delta.enableDeletionVectors' = 'true'
);
Then enable Efficient Column Updates:
ALTER TABLE documents SET TBLPROPERTIES (
'delta.enableColumnUpdates-preview' = 'true'
);
If you encounter issues, you can roll back to standard updates.
Add and backfill a column
The following example uses an existing documents table with a document_id column and a description string column. It adds a column containing the length of each description:
ALTER TABLE documents ADD COLUMN text_length INT;
UPDATE documents
SET text_length = LENGTH(description);
After adding the column, existing rows contain NULL in text_length until the UPDATE populates its values. Efficient Column Updates lets the backfill write the new column values without rewriting the unchanged columns.
Query the table to inspect the populated values:
SELECT document_id, description, text_length
FROM documents
LIMIT 10;
Efficient Column Updates reduces the data rewritten during a backfill. The update still reads the input columns and computes the new values.
Enabling Efficient Column Updates does not guarantee that every UPDATE uses it. In some cases, Databricks falls back to a standard update that rewrites the affected data files. The update still succeeds and produces the same results.
Roll back to standard updates
If you encounter issues with Efficient Column Updates, disable new column updates and consolidate existing column files:
-
Stop writing new column files:
SQLALTER TABLE documents SET TBLPROPERTIES ('delta.enableColumnUpdates-preview' = 'false'); -
Rewrite data files that have column files using REORG TABLE:
SQLREORG TABLE documents APPLY (PURGE);
REORG writes the latest column values into the rewritten data files, creating a new table version without column files. The updated values are preserved. Earlier table versions can still reference column files.
The table feature remains enabled in the table protocol, so reads and writes still require Databricks Runtime 19 or above.
Limitations
- Efficient Column Updates is incompatible with write-time change data feed (CDF). See Legacy change data feed.
- Efficient Column Updates is incompatible with Universal Format (UniForm).
- Apache Iceberg managed tables are not supported.
MERGEoperations do not use Efficient Column Updates.- Removing Efficient Column Updates with
DROP FEATUREis not supported. Databricks recommends that you avoid enabling it on critical tables. To stop using column updates, see Roll back to standard updates. - Lakeflow pipelines are not supported.