cacheTable

cacheTable persists a Spark SQL Table in memory.

Syntax:

  • cacheTable(sqlContext, “sqlTable”)

Parameters:

  • sqlContext: SQLContext. This is already created for you in the Databricks notebooks, do not recreate!
  • sqlTable: String, Name of Spark SQL table

Output:

  • NULL
# Create SparkR DataFrame using the faithful dataset from R
df <- createDataFrame(sqlContext, faithful)

# Displays the content of the DataFrame to stdout
head(df)
# Register df as Temporary Table, with table name: tempTable
registerTempTable(df, "tempTable")

# Cache Spark SQL table in memory
cacheTable(sqlContext, "tempTable")
# Perform SQL operations on cached tempTable
query <- sql(sqlContext, "SELECT * FROM tempTable WHERE waiting > 60")
head(query)