variant_explode

Separates a variant object/array into multiple rows containing its fields/elements. Its result schema is struct<pos int, key string, value variant>. pos is the position of the field/element in its parent object/array, and value is the field/element value. key is the field name when exploding a variant object, or is NULL when exploding a variant array. It ignores any input that is not a variant array/object, including SQL NULL, variant null, and any other variant values.

Syntax

Python

spark.tvf.variant_explode(input)

Parameters

Parameter	Type	Description
input	pyspark.sql.Column	Input column of values to explode.

Examples

Example 1: Using variant_explode with a variant array

Python

from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('["hello", "world"]'))).show()

Output

+---+----+-------+
|pos| key|  value|
+---+----+-------+
|  0|NULL|"hello"|
|  1|NULL|"world"|
+---+----+-------+

Example 2: Using variant_explode with a variant object

Python

from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('{"a": true, "b": 3.14}'))).show()

Output

+---+---+-----+
|pos|key|value|
+---+---+-----+
|  0|  a| true|
|  1|  b| 3.14|
+---+---+-----+

Example 3: Using variant_explode with an empty variant array

Python

from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('[]'))).show()

Output

+---+---+-----+
|pos|key|value|
+---+---+-----+
+---+---+-----+

Example 4: Using variant_explode with an empty variant object

Python

from pyspark.sql import functions as sf
spark.tvf.variant_explode(sf.parse_json(sf.lit('{}'))).show()

Output

+---+---+-----+
|pos|key|value|
+---+---+-----+
+---+---+-----+