The two most commonly used libraries that provide an R interface to Spark are SparkR and sparklyr. Databricks notebooks and jobs support both packages, although you cannot use functions from both SparkR and sparklyr with the same object.

Databricks also provides an integration with RStudio, the popular IDE for R.



For information on using R and RStudio Desktop with Databricks Connect, see R / RStudio.


The articles that appeared in this section have moved to our new Knowledge Base.