Run an R, Scala, or SQL notebook as a job by using the Databricks extension for Visual Studio Code

This article describes how to run an R, Scala, or SQL notebook as a Databricks job by using the Databricks extension for Visual Studio Code. See What is the Databricks extension for Visual Studio Code?.

To run a Python notebook as a Databricks job instead, see Run a Python notebook as a job by using the Databricks extension for Visual Studio Code.

This information assumes that you have already installed and set up the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.

With the extension and your code project opened, do the following:

Note

The following procedure uses the Databricks extension for Visual Studio Code, version 1, which is generally available. To complete this procedure for the Databricks extension for Visual Studio Code, version 2, currently in Private Preview, skip ahead to Run an R, Scala, or SQL notebook as a job by using the Databricks extension for Visual Studio Code, version 2.

In your code project, open the R, Scala, or SQL notebook that you want to run as a job.

Tip

To create an R, Scala, or SQL notebook file in Visual Studio Code, begin by clicking File > New File, select Python File, and save the new file with a .r, .scala, or .sql file extension, respectively.

To turn the .r, .scala, or .sql file into a Databricks notebook, add the special comment Databricks notebook source to the beginning of the file and add the special comment COMMAND ---------- before each cell. Be sure to use the correct comment marker for each language (# for R, // for Scala, and -- for SQL). For more information, see Import a file and convert it to a notebook.

This is similar to the pattern for Python notebooks:
In Run and Debug view (View > Run), select Run on Databricks as Workflow from the drop-down list, and then click the green play arrow (Start Debugging) icon.

Note

If Run on Databricks as Workflow is not available, see Create a custom run configuration for the Databricks extension for Visual Studio Code.

A new editor tab appears, titled Databricks Job Run. The notebook runs as a job in the workspace. The notebook and its output are displayed in the new editor tab’s Output area.

To view information about the job run, click the Task run ID link in the Databricks Job Run editor tab. Your workspace opens and the job run’s details are displayed in the workspace.