Spark Version

View as Markdown

ListGET /api/2.1/clusters/spark-versions

SparkVersion object

keystring

Spark version key, for example "2.1.x-scala2.11". This is the value which should be provided as the "spark_version" when creating a new cluster. Note that the exact Spark version may change over time for a "wildcard" version (i.e., "2.1.x-scala2.11" is a "wildcard" version) with minor bug fixes.

Example: 8.2.x-scala2.12

namestring

A descriptive name for this Spark version, for example "Spark 2.1".

Example: 8.2 (includes Apache Spark 3.1.1, Scala 2.12)

View as Markdown

The SparkVersion object

{
  "key": "8.2.x-scala2.12",
  "name": "8.2 (includes Apache Spark 3.1.1, Scala 2.12)"
}

List GA

GET /api/2.1/clusters/spark-versions

Returns the list of available Spark versions. These versions can be used to launch a cluster.

API scopes: clusters

Response

Returns a list of SparkVersion objects.

View as Markdown

Response

{
  "versions": [
    {
      "key": "8.2.x-scala2.12",
      "name": "8.2 (includes Apache Spark 3.1.1, Scala 2.12)"
    }
  ]
}