getOrCreate (SparkSession.Builder)

Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder.

This method first checks whether there is a valid global default SparkSession and, if so, returns that one. If no valid global default SparkSession exists, the method creates a new SparkSession and assigns it as the global default. When an existing SparkSession is returned, the config options specified in this builder are applied to it. To create a new session without checking, use create.

Syntax

getOrCreate()

Returns

SparkSession

Examples

Python

s1 = SparkSession.builder.config("k1", "v1").getOrCreate()
s1.conf.get("k1") == "v1"
# True

# The configuration of the SparkSession can be changed afterwards.
s1.conf.set("k1", "v1_new")
s1.conf.get("k1") == "v1_new"
# True

# When an existing SparkSession is returned, the config options specified
# in this builder are applied to the existing SparkSession.
s2 = SparkSession.builder.config("k2", "v2").getOrCreate()
s1.conf.get("k1") == s2.conf.get("k1") == "v1_new"
# True
s1.conf.get("k2") == s2.conf.get("k2") == "v2"
# True