get
Returns the element of an array at the given (0-based) index. If the index points outside of the array boundaries, then this function returns NULL. The position is not 1-based, but 0-based index.
Syntax
Python
from pyspark.sql import functions as sf
sf.get(col, index)
Parameters
Parameter | Type | Description |
|---|---|---|
|
| Name of the column containing the array. |
|
| Index to check for in the array. |
Returns
pyspark.sql.Column: Value at the given position.
Examples
Example 1: Getting an element at a fixed position
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
df.select(sf.get(df.data, 1)).show()
Output
+------------+
|get(data, 1)|
+------------+
| b|
+------------+
Example 2: Getting an element at a position outside the array boundaries
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
df.select(sf.get(df.data, 3)).show()
Output
+------------+
|get(data, 3)|
+------------+
| NULL|
+------------+
Example 3: Getting an element at a position specified by another column
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
df.select(sf.get(df.data, df.index)).show()
Output
+----------------+
|get(data, index)|
+----------------+
| c|
+----------------+
Example 4: Getting an element at a position calculated from another column
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
df.select(sf.get(df.data, df.index - 1)).show()
Output
+----------------------+
|get(data, (index - 1))|
+----------------------+
| b|
+----------------------+
Example 5: Getting an element at a negative position
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([(["a", "b", "c"], )], ['data'])
df.select(sf.get(df.data, -1)).show()
Output
+-------------+
|get(data, -1)|
+-------------+
| NULL|
+-------------+