fillna

Returns a new DataFrame which null values are filled with new value. DataFrame.fillna and DataFrameNaFunctions.fill are aliases of each other.

Syntax

fillna(value: Union["LiteralType", Dict[str, "LiteralType"]], subset: Optional[Union[str, Tuple[str, ...], List[str]]] = None)

Parameters

Parameter	Type	Description
value	int, float, string, bool or dict	the value to replace null values with. If the value is a dict, then subset is ignored and value must be a mapping from column name (string) to replacement value. The replacement value must be an int, float, boolean, or string.
subset	str, tuple or list, optional	optional list of column names to consider. Columns specified in subset that do not have matching data types are ignored.

Returns

DataFrame: DataFrame with replaced null values.

Examples

Python

df = spark.createDataFrame([
    (10, 80.5, "Alice", None),
    (5, None, "Bob", None),
    (None, None, "Tom", None),
    (None, None, None, True)],
    schema=["age", "height", "name", "bool"])

df.na.fill(50).show()
# +---+------+-----+----+
# |age|height| name|bool|
# +---+------+-----+----+
# | 10|  80.5|Alice|NULL|
# |  5|  50.0|  Bob|NULL|
# | 50|  50.0|  Tom|NULL|
# | 50|  50.0| NULL|true|
# +---+------+-----+----+

df.na.fill(False).show()
# +----+------+-----+-----+
# | age|height| name| bool|
# +----+------+-----+-----+
# |  10|  80.5|Alice|false|
# |   5|  NULL|  Bob|false|
# |NULL|  NULL|  Tom|false|
# |NULL|  NULL| NULL| true|
# +----+------+-----+-----+

df.na.fill({'age': 50, 'name': 'unknown'}).show()
# +---+------+-------+----+
# |age|height|   name|bool|
# +---+------+-------+----+
# | 10|  80.5|  Alice|NULL|
# |  5|  NULL|    Bob|NULL|
# | 50|  NULL|    Tom|NULL|
# | 50|  NULL|unknown|true|
# +---+------+-------+----+