array_prepend
Returns an array containing the given element as the first element and the rest of the elements from the original array.
Added in Databricks Runtime 13.1
Syntax
Python
from pyspark.sql import functions as sf
sf.array_prepend(col, value)
Parameters
Parameter | Type | Description |
|---|---|---|
|
| Name of column containing array |
| Any | A literal value, or a Column expression. |
Returns
pyspark.sql.Column: an array with the given value prepended.
Examples
Example 1: Prepending a column value to an array column
Python
from pyspark.sql import Row, functions as sf
df = spark.createDataFrame([Row(c1=["b", "a", "c"], c2="c")])
df.select(sf.array_prepend(df.c1, df.c2)).show()
Output
+---------------------+
|array_prepend(c1, c2)|
+---------------------+
| [c, b, a, c]|
+---------------------+
Example 2: Prepending a numeric value to an array column
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3],)], ['data'])
df.select(sf.array_prepend(df.data, 4)).show()
Output
+----------------------+
|array_prepend(data, 4)|
+----------------------+
| [4, 1, 2, 3]|
+----------------------+
Example 3: Prepending a null value to an array column
Python
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3],)], ['data'])
df.select(sf.array_prepend(df.data, None)).show()
Output
+-------------------------+
|array_prepend(data, NULL)|
+-------------------------+
| [NULL, 1, 2, 3]|
+-------------------------+
Example 4: Prepending a value to a NULL array column
Python
from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, IntegerType, StructType, StructField
schema = StructType([
StructField("data", ArrayType(IntegerType()), True)
])
df = spark.createDataFrame([(None,)], schema=schema)
df.select(sf.array_prepend(df.data, 4)).show()
Output
+----------------------+
|array_prepend(data, 4)|
+----------------------+
| NULL|
+----------------------+
Example 5: Prepending a value to an empty array
Python
from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, IntegerType, StructType, StructField
schema = StructType([
StructField("data", ArrayType(IntegerType()), True)
])
df = spark.createDataFrame([([],)], schema=schema)
df.select(sf.array_prepend(df.data, 1)).show()
Output
+----------------------+
|array_prepend(data, 1)|
+----------------------+
| [1]|
+----------------------+