Skip to main content

Drop Delta table features (legacy)

important

This documentation has been retired and might not be updated. The products, services, or technologies mentioned in this content are no longer supported. See Drop a Delta Lake table feature and downgrade table protocol.

This article documents the Public Preview behavior for dropping Delta Lake table features available starting in Databricks Runtime 14.3 LTS. Databricks recommends using the generally available functionality in Databricks Runtime 16.3 and above, which replaces the legacy behavior. See Drop a Delta Lake table feature and downgrade table protocol.

Databricks provides limited support for dropping table features. Dropping a table feature involves the following stages:

  • Disable table properties that use the table feature.
  • Remove all traces of the table feature from the data files backing the table.
  • Remove transaction entries that use the table feature from the transaction log.
  • Downgrade the table protocol.
important

All DROP FEATURE operations conflict with all concurrent writes.

Streaming reads fail when they encounter a commit that changes table metadata. If you want the stream to continue you must restart it. For recommended methods, see Production considerations for Structured Streaming.

How can I drop a Delta table feature?​

To remove a Delta table feature, you run an ALTER TABLE <table-name> DROP FEATURE <feature-name> [TRUNCATE HISTORY] command. See ALTER TABLE.

You must use Databricks Runtime 14.3 LTS or above and have MODIFY privileges on the target Delta table.

What Delta table features can be dropped?​

You can drop the following Delta table features:

You cannot drop other Delta table features.

important

Some Delta Lake functionality enables multiple table features. Some table features depend on other table features, and might block dropping dependent table features. Because some table features cannot be dropped, this means that enablement for some Delta Lake features cannot be rolled back.

Databricks recommends always testing dependent workloads and systems for compatibility with new functionality before enabling functionality that upgrades reader or writer protocols for production data.

Enable table features to drop legacy features​

note

This section describes a pattern that is only required in Databricks Runtime 16.0 and below.

The DROP FEATURE command requires protocol versions that support table feature reads and writes. Delta features like columnMapping and checkConstraints were supported in earlier protocol versions. Depending on other features enabled on the table, you might need to upgrade protocol versions before you can drop these features.

You can use the following command to upgrade the table reader and writer versions, which allows you to drop column mapping and downgrade the protocol:

SQL
ALTER TABLE <table-name> SET TBLPROPERTIES (
  'delta.minReaderVersion' = '3',
  'delta.minWriterVersion' = '7'
)

How are Delta table features dropped?​

Because Delta table features affect the way a table is read/written, they must be completely absent from the transaction log for full removal. The specifics of feature removal vary by feature, but the following section provides a general overview.

Dropping a feature occurs in two steps and requires time to elapse before completion.

Step 1: Prepare to drop a table feature​

During the first step, you must prepare to drop the table feature. The following describes what happens during this step:

  1. Run the DROP FEATURE command.
  2. Table properties that specifically enable a table feature have values set to disable the feature.
  3. Table properties that control behaviors associated with the dropped feature have options set to default values before the feature was introduced.
  4. As necessary, data and metadata files are rewritten, respecting the updated table properties.
  5. The command finishes running and returns an error message informing you that you must wait 24 hours to proceed with feature removal.

After first disabling a feature, you can continue writing to the target table before completing the protocol downgrade, but cannot use the table feature you are removing.

note

If you leave the table in this state, operations against the table do not use the table feature, but the protocol still supports the table feature. Until you complete the final downgrade step, the table is not readable by Delta clients that do not understand the table feature.

Step 2: Downgrade the protocol and drop a table feature​

In the second step, you truncate and delete all transaction history associated with the feature. This allows you to drop the table feature and downgrade the protocol.

  1. After at least 24 hours have passed, you execute the DROP FEATURE command again with the TRUNCATE HISTORY clause.
  2. The client confirms that no transactions in the specified retention threshold use the table feature, then truncates the table history to that threshold.
  3. The protocol is downgraded, dropping the table feature.
  4. If the table features that are present in the table can be represented by a legacy protocol version, the minReaderVersion and minWriterVersion for the table are downgraded to the lowest version that supports exactly all remaining features in use by the Delta table.
important

Running ALTER TABLE <table-name> DROP FEATURE <feature-name> TRUNCATE HISTORY removes all transaction log data older than 24 hours. After dropping a Delta table feature, you do not have access to table history or time travel.

See Delta Lake feature compatibility and protocols.

Last updated on