coalesce
Returns the first column that is not null. Supports Spark Connect.
For the corresponding Databricks SQL function, see coalesce function.
Syntax
Python
from pyspark.sql import functions as dbf
dbf.coalesce(*cols)
Parameters
Parameter | Type | Description |
|---|---|---|
|
| List of columns to work on. |
Returns
pyspark.sql.Column: value of the first column that is not null.
Examples
Python
from pyspark.sql import functions as dbf
df = spark.createDataFrame([(None, None), (1, None), (None, 2)], ("a", "b"))
df.select('*', dbf.coalesce("a", df["b"])).show()
Output
+----+----+--------------+
| a| b|coalesce(a, b)|
+----+----+--------------+
|NULL|NULL| NULL|
| 1|NULL| 1|
|NULL| 2| 2|
+----+----+--------------+