filterΒΆ

filter returns the rows of a SparkDataFrame that matches the given condition.

Syntax:

  • filter(SparkDataFrame, condition)

Parameters:

  • SparkDataFrame: Any SparkDataFrame
  • condition: Condition to filter on. This may either be a Column expression or a string containing a SQL statement

Output:

  • SparkDataFrame
require(SparkR)

# Create SparkDataFrame
df <- createDataFrame(mtcars)
head(df)
# Filter using Column Expression
filtered <- filter(df, df$cyl == 6)
collect(filtered)
# Alternative Syntax
# Filter using SQL statement strings
filtered2 <- filter(df, "cyl = 6")
collect(filtered2)