partitionBy (Window)

Creates a WindowSpec with the partitioning defined.

Syntax

Window.partitionBy(*cols)

Parameters

Parameter	Type	Description
cols	str, Column, or list	Names of columns or expressions.

Parameter	Type	Description
cols	str, Column, or list	Names of columns or expressions.

Returns

WindowSpec

Examples

Python

from pyspark.sql import Window, functions as sf

df = spark.createDataFrame(
    [(1, "a"), (1, "a"), (2, "a"), (1, "b"), (2, "b"), (3, "b")], ["id", "category"])

# Show row number ordered by id in each category partition.
window = Window.partitionBy("category").orderBy("id")
df.withColumn("row_number", sf.row_number().over(window)).show()
# +---+--------+----------+
# | id|category|row_number|
# +---+--------+----------+
# |  1|       a|         1|
# |  1|       a|         2|
# |  2|       a|         3|
# |  1|       b|         1|
# |  2|       b|         2|
# |  3|       b|         3|
# +---+--------+----------+