Skip to main content
Last updated on

map_from_entries

Transforms an array of key-value pair entries (structs with two fields) into a map. The first field of each entry is used as the key and the second field as the value in the resulting map column.

Syntax​

Python
from pyspark.sql import functions as sf

sf.map_from_entries(col)

Parameters​

Parameter

Type

Description

col

pyspark.sql.Column or str

Name of column or expression

Returns​

pyspark.sql.Column: A map created from the given array of entries.

Examples​

Example 1: Basic usage of map_from_entries

Python
from pyspark.sql import functions as sf
df = spark.sql("SELECT array(struct(1, 'a'), struct(2, 'b')) as data")
df.select(sf.map_from_entries(df.data)).show()
Output
+----------------------+
|map_from_entries(data)|
+----------------------+
|      {1 -> a, 2 -> b}|
+----------------------+

Example 2: map_from_entries with null values

Python
from pyspark.sql import functions as sf
df = spark.sql("SELECT array(struct(1, null), struct(2, 'b')) as data")
df.select(sf.map_from_entries(df.data)).show()
Output
+----------------------+
|map_from_entries(data)|
+----------------------+
|   {1 -> NULL, 2 -> b}|
+----------------------+

Example 3: map_from_entries with a DataFrame

Python
from pyspark.sql import Row, functions as sf
df = spark.createDataFrame([([Row(1, "a"), Row(2, "b")],), ([Row(3, "c")],)], ['data'])
df.select(sf.map_from_entries(df.data)).show()
Output
+----------------------+
|map_from_entries(data)|
+----------------------+
|      {1 -> a, 2 -> b}|
|              {3 -> c}|
+----------------------+

Example 4: map_from_entries with empty array

Python
from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, StringType, IntegerType, StructType, StructField
schema = StructType([
  StructField("data", ArrayType(
    StructType([
      StructField("key", IntegerType()),
      StructField("value", StringType())
    ])
  ), True)
])
df = spark.createDataFrame([([],)], schema=schema)
df.select(sf.map_from_entries(df.data)).show()
Output
+----------------------+
|map_from_entries(data)|
+----------------------+
|                    {}|
+----------------------+