Databricks for Scala developers

This section provides a guide to developing notebooks and jobs in Databricks using the Scala language.

Scala API

These links provide an introduction to and reference for the Apache Spark Scala API.

• Introduction to DataFrames
  • Complex and nested data
  • Aggregators
• Introduction to Datasets
• Introduction to Structured Streaming
• Apache Spark API reference

Visualizations

Databricks Scala notebooks supports various types of visualizations using the display function.

• Visualization overview
• Visualization deep dive in Scala

Interoperability

This section describes features that support interoperability between Scala and SQL.

• User-defined aggregate functions - Scala
• User-defined functions - Scala

Tools

In addition to Databricks notebooks, you can also use the following Scala developer tools

• IntelliJ with Databricks Connect

Libraries

Databricks runtimes provide many libraries. To make third-party or locally-built Scala libraries available to notebooks and jobs running on your Databricks clusters, you can install libraries following these instructions:

• Install Scala libraries in a cluster

Resources

• Knowledge Base