withColumn

withColumn returns a new DataFrame with an added column, typically after performing a column operation. This is similar to base R’s transform <https://www.rdocumentation.org/packages/base/versions/3.4.3/topics/transform>__.

Syntax:

  • withColumn(df, “newColName”, colExpr)

Parameters:

  • df: Any SparkR DataFrame
  • newColName: String, name of new column to be added
  • colExpr: Column Expression

Output:

  • SparkR DataFrame
# Create SparkR DataFrame
df <- createDataFrame(sqlContext, iris)
head(df)
# Create new DataFrame with additional column: sepalRatio
newdf <- withColumn(df, "sepalRatio", df$Sepal_Length/df$Sepal_Width)
head(newdf)