unpersist

Marks the DataFrame as non-persistent, and remove all blocks for it from memory and disk.

Syntax

unpersist(blocking: bool = False)

Parameters

Parameter	Type	Description
blocking	bool	Whether to block until all blocks are deleted.

Parameter	Type	Description
blocking	bool	Whether to block until all blocks are deleted.

Returns

DataFrame: Unpersisted DataFrame.

Notes

blocking default has changed to False to match Scala in 2.0.

Cached data is shared across all Spark sessions on the cluster, so unpersisting it affects all sessions.

Examples

Python

df = spark.range(1)
df.persist()
# DataFrame[id: bigint]
df.unpersist()
# DataFrame[id: bigint]
df = spark.range(1)
df.unpersist(True)
# DataFrame[id: bigint]