delta.appendOnly
true for this Delta table to be append-only.
If append-only, existing records cannot be deleted, and
existing values cannot be updated.
See Table properties.
Data type: Boolean
Default: false
|
delta.autoOptimize.autoCompact
true for Delta Lake to automatically optimize the
layout of the files for this Delta table.
See Enable Auto Optimize.
Data type: Boolean
Default: (none)
|
delta.autoOptimize.optimizeWrite
true for Delta Lake to automatically optimize the
layout of the files for this Delta table during writes.
See Enable Auto Optimize.
Data type: Boolean
Default: (none)
|
delta.checkpoint.writeStatsAsJson
true for Delta Lake to write file statistics in
checkpoints in JSON format for the stats column.
See Trade-offs with statistics in checkpoints.
Data type: Boolean
Default: true
|
delta.checkpoint.writeStatsAsStruct
true for Delta Lake to write file statistics to checkpoints
in struct format for the stats_parsed column and to write
partition values as a struct for partitionValues_parsed.
See Trade-offs with statistics in checkpoints.
Data type: Boolean
Default: (none)
|
delta.columnMapping.mode
Whether column mapping is enabled for Delta table columns and the corresponding
Parquet columns that use different names.
See Delta column mapping.
Data type: DeltaColumnMappingMode
Default: none
|
delta.compatibility.symlinkFormatManifest.enabled
true for Delta Lake to configure the Delta table so that all write operations on the
table automatically update the manifests.
See Step 3: Update manifests.
Data type: Boolean
Default: false
|
delta.dataSkippingNumIndexedCols
The number of columns for Delta Lake to collect statistics
about for data skipping. A value of -1 means to collect
statistics for all columns. Updating this property does not
automatically collect statistics again; instead, it
redefines the statistics schema of the Delta table. Specifically,
it changes the behavior of future statistics collection
(such as during appends and optimizations) as well as
data skipping (such as ignoring column statistics beyond
this number, even when such statistics exist).
See Data skipping.
Data type: Int
Default: 32
|
delta.deletedFileRetentionDuration
The shortest duration for Delta Lake to keep logically deleted
data files before deleting them physically. This is to
prevent failures in stale readers after compactions or
partition overwrites.
This value should be large enough to ensure that:
It is larger than the longest possible duration of a
job if you run VACUUM when there are concurrent readers
or writers accessing the Delta table. If you run a streaming query that reads from the table,
that query does not stop for longer than this value.
Otherwise, the query may not be able to restart, as it
must still read old files.
See Data retention.
Data type: CalendarInterval
Default: interval 1 week
|
delta.enableChangeDataFeed
true to enable change data feed.
See Enable change data feed.
Data type: Boolean
Default: false
|
delta.isolationLevel
The degree to which a transaction must be isolated from modifications made by concurrent
transactions.
Valid values are Serializable and WriteSerializable.
See Isolation levels.
Data type: String
Default: WriteSerializable
|
delta.logRetentionDuration
How long the history for a Delta table is kept.
Each time a checkpoint is written, Delta Lake automatically cleans up log entries older
than the retention interval. If you set this property to a large enough value, many
log entries are retained. This should not impact performance as operations against the
log are constant time. Operations on history are parallel but will become more expensive
as the log size increases.
See Data retention.
Data type: CalendarInterval
Default: interval 30 days
|
delta.minReaderVersion
The minimum required protocol reader version for a reader that allows to read from this
Delta table.
See Table protocol versioning.
Data type: Int
Default: 1
|
delta.minWriterVersion
The minimum required protocol writer version for a writer that allows to write to this
Delta table.
See Table protocol versioning.
Data type: Int
Default: 2
|
delta.randomizeFilePrefixes
true for Delta Lake to generate a random prefix for a file
path instead of partition information.
For example, this may improve Amazon S3 performance when Delta Lake needs to send very
high volumes of Amazon S3 calls to better partition across S3 servers.
See Table properties.
Data type: Boolean
Default: false
|
delta.randomPrefixLength
When delta.randomizeFilePrefixes is set to true, the
number of characters that Delta Lake generates for random prefixes.
See Table properties.
Data type: Int
Default: 2
|
delta.setTransactionRetentionDuration
The shortest duration within which new snapshots will retain transaction identifiers
(for example, SetTransactions). When a new snapshot sees a transaction identifier older
than or equal to the duration specified by this property, the snapshot considers it
expired and ignores it. The SetTransaction identifier is used when making the writes
idempotent. See Performing stream-static joins
for details.
Data type: CalendarInterval
Default: (none)
|
delta.targetFileSize
The target file size in bytes or higher units for file tuning. For example,
104857600 (bytes) or 100mb.
See Optimize performance with file management.
Data type: String
Default: (none)
|
delta.tuneFileSizesForRewrites
true to always use lower file sizes for all data layout optimization operations on the
Delta table.
false to never tune to lower file sizes, that is, prevent auto-detection from being
activated.
See Optimize performance with file management.
Data type: Boolean
Default: (none)
|