element_at

Collection function: Returns element of array at given (1-based) index or value for given key in a map. For arrays, if index is 0, Spark will throw an error. If index < 0, accesses elements from the last to the first. If 'spark.sql.ansi.enabled' is set to true, an exception will be thrown if the index is out of array boundaries instead of returning NULL. For maps, the function always returns NULL if the key is not contained in the map. Supports Spark Connect.

For the corresponding Databricks SQL function, see element_at function.

Syntax

Python

from pyspark.databricks.sql import functions as dbf

dbf.element_at(col=<col>, extraction=<extraction>)

Parameters

Parameter	Type	Description
col	pyspark.sql.Column or str	Name of column containing array or map.
extraction	Any	Index to check for in array or key to check for in map.

Returns

pyspark.sql.Column: value at given position.

Examples

Example 1: Getting the first element of an array

Python

from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
df.select(dbf.element_at(df.data, 1)).show()

Output

+-------------------+
|element_at(data, 1)|
+-------------------+
|                  a|
+-------------------+

Example 2: Getting a value from a map using a key

Python

from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame([({"a": 1.0, "b": 2.0},)], ['data'])
df.select(dbf.element_at(df.data, dbf.lit("a"))).show()

Output

+-------------------+
|element_at(data, a)|
+-------------------+
|                1.0|
+-------------------+