cache

Persists the DataFrame with the default storage level (MEMORY_AND_DISK_DESER).

Syntax

cache()

Returns

DataFrame: Cached DataFrame.

Notes

The default storage level has changed to MEMORY_AND_DISK_DESER to match Scala in 3.0.

On classic compute, cached data is shared across all Spark sessions on the cluster.

On serverless compute, cache() runs the query right away and stores the result for the rest of your session. Only your session can read it, and the result doesn't change when the source tables change. See Cache DataFrames on serverless compute.

Examples

Python

df = spark.range(1)
df.cache()
# DataFrame[id: bigint]

df.explain()
# == Physical Plan ==
# InMemoryTableScan ...