withColumnΒΆ

withColumn returns a new SparkDataFrame with an added column, typically after performing a column operation. This is similar to base R’s transform <http://stat.ethz.ch/R-manual/R-devel/library/base/html/transform.html>__.

Syntax:

  • withColumn(df, “newColName”, colExpr)

Parameters:

  • df: Any SparkDataFrame
  • newColName: String, name of new column to be added
  • colExpr: Column Expression

Output:

  • SparkDataFrame
require(SparkR)

# Create SparkDataFrame
df <- createDataFrame(iris)
head(df)
# Create new SparkDataFrame with additional column: sepalRatio
newdf <- withColumn(df, "sepalRatio", df$Sepal_Length/df$Sepal_Width)
head(newdf)