Rewind and replay a pipeline
This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.
A bad transformation, a malformed batch of source records, or an unexpected schema change can leave a pipeline's tables wrong from a known point in time forward. Rewind returns the pipeline to a point before the problem so that you can deploy a fix and reprocess only the affected data.
Rewind restores table versions, streaming source offsets, and operator state together, so replay does not skip records or write duplicates. Three operations reprocess data, and they solve different problems:
- Rewind is for a recoverable pipeline that wrote incorrect data from a known point in time, such as after a bad transformation, malformed input, or a bad code deploy. It restores table data, source offsets, and operator state to a point before the problem and reprocesses only the affected data, keeping operator state. Data before that point is untouched.
- Full refresh rebuilds a table from all available source data and discards its current contents. Use it to recompute everything from scratch, or when a code change is not compatible with the existing state.
- Checkpoint reset recovers a pipeline whose checkpoint is invalid or corrupted, or that is blocked by a checkpoint-incompatible code change. It resets the checkpoint and continues forward while preserving the table's current contents. Rewind cannot recover these cases, because it does not relax Structured Streaming compatibility rules.
Requirements
Requirement | Detail |
|---|---|
Channel | The pipeline must be on the Preview channel. See Configure pipelines. |
Configuration | Set the |
Pipeline mode | Triggered and continuous pipelines. Real-time mode is not supported. |
Sources | Delta tables, streaming tables, Kafka, and Auto Loader. |
Targets | Streaming tables and materialized views. |
Flows | Streaming flows and AUTO change data capture (CDC) flows, including SCD Type 1 and SCD Type 2 targets. Stateful queries such as aggregations, joins, and deduplication are supported. |
Every flow in the pipeline must meet these requirements. While pipelines.rewind.betaEnabled is true, a pipeline containing a flow that does not qualify fails its updates. Confirm that every flow meets the requirements above before you enable it.
A pipeline that has pipelines.rewind.betaEnabled set to true cannot move back to the Current channel until the Current channel is updated to a runtime that supports rewind.
How rewind and replay work
Rewind and replay are separate steps.
Rewind restores each table to the version it held at a rewind point and resets the streaming checkpoints that track how far each flow has read. Your transformations do not run, and no source data is reprocessed.
Replay happens the next time the pipeline runs. It reprocesses from the rewind point using the current pipeline definition, catches up to the present, then resumes normal incremental processing. Rewind does not start the pipeline, so start it yourself when you are ready.
The pipeline generates rewind points automatically, about once an hour, and keeps them for 7 days. A pipeline you just created has nothing to rewind to until it produces its first one.
Rewinding a dataset also rewinds everything downstream of it in the same pipeline. Rewind covers one pipeline. It does not coordinate with other pipelines or with external readers of the same tables, so manage those separately.
Rewind a pipeline using the UI
The UI and Genie are the primary ways to use rewind. The UI lists the available rewind points and shows which datasets each one affects before you commit.
- On your pipeline page, click the
button next to Run pipeline, then click Rewind pipeline. - Select a rewind point, or use a shortcut such as Rewind to yesterday or Rewind to the latest point. Click Next.
- Select which tables to include. Use the Graph view to select datasets in the pipeline graph, or the List view to select them from a table. Leave Reset all checkpoints selected (the default) to restore source offsets and operator state along with the table data, so the pipeline reprocesses from the rewind point. Clear it to restore the table data only, without reprocessing, for example when you want a table's contents restored but do not want to reprocess the affected data. This setting must be the same for a table and its upstreams, and it cannot be cleared for a flow that reads an external source such as Kafka or Auto Loader. Click Next.
- Review the rewind point, the checkpoint setting, and the affected datasets, then click Rewind.
Start the pipeline to replay the data.
You can rewind repeatedly. Each rewind replaces the last, so you can recover from a failed replay by rewinding somewhere else.
After a rewind
A replay failure leaves the pipeline rewound but stopped. Fix the code or the source data and start the pipeline to retry, or rewind to a different point. The pipeline does not roll back on its own, and errors surface through standard pipeline diagnostics and the event log.
Replay only succeeds if the current pipeline definition is compatible with the restored state; rewind does not relax Structured Streaming compatibility rules. For which changes are compatible, see Types of changes in Structured Streaming queries. Materialized views follow batch semantics and tolerate broader schema changes, but still fail if a dependency is incompatible.
A rewind that fails partway can leave the pipeline partially rewound. You have two options:
- Rewind again to the same point or a different one, and the pipeline converges on that point.
- To force the pipeline to start a normal update despite the incomplete rewind, set
pipelines.allowUpdateAfterIncompleteRewindtotrueand restart the pipeline.
How far back you can rewind
Rewind points are kept for 7 days. Within that window, a rewind fails if the data it needs has already been removed. Check for these before you rely on rewind:
VACUUMor a shortdelta.deletedFileRetentionDurationon your tables. See Work with table history.- Source retention shorter than the window you want to rewind across, such as a Kafka topic that keeps one day.
Pipelines with several sources or long dependency chains need more retention, because every table and checkpoint has to reach back to a consistent point.
Limitations
- Rewind cannot restore a pipeline to a point before a full refresh.
- Rewind points are kept for 7 days, and a rewind fails if the table history or source data needed to reach the point has already been removed, for example by
VACUUMor short source retention. See How far back you can rewind. - Real-time mode is not supported.
- Kinesis, Pulsar, Google Pub/Sub, and custom sources built with the DSv2 or Python data source APIs are not supported as sources.
- External sinks and custom sinks are not supported, including sinks defined with
create_sink(). See Use sinks in pipelines. - Streaming tables that use a row filter or a column mask cannot be rewound. See Manually apply row filters and column masks.
- Stateful rewind requires the RocksDB state store, which pipelines use by default. A rewind fails for a flow configured with a different state store.
- Some AUTO CDC streaming tables need a refresh before they can be rewound. The pipeline notifies you when you request the rewind.
- Materialized views might fully recompute rather than incrementally refresh after a rewind. See Incremental refresh for materialized views.
- Rewind covers one pipeline and does not coordinate with external readers or other pipelines reading the same tables.
- Rewind does not restore pipeline code, pipeline configuration, or Unity Catalog object metadata such as tags and grants.