REFRESH

Invalidates and refreshes all the cached data (and the associated metadata) in Apache Spark cache for all Datasets that contains the given data source path. Path matching is by prefix, that is, / would invalidate everything that is cached.

Syntax 

REFRESH resource_path

See Delta and Apache Spark caching for the differences between the Delta cache and the Apache Spark cache.

Parameters

  • resource_path

    The path of the resource that is to be refreshed.

Examples 

-- The Path is resolved using the datasource's File Index.
> CREATE TABLE test(ID INT) using parquet;
> INSERT INTO test SELECT 1000;
> CACHE TABLE test;
> INSERT INTO test SELECT 100;
> REFRESH "hdfs://path/to/table";