This article describes how to run an R, Scala, or SQL notebook as a Databricks job by using the Databricks extension for Visual Studio Code. See What is the Databricks extension for Visual Studio Code?.
To run a Python notebook as a Databricks job instead, see Run a Python notebook as a job by using the Databricks extension for Visual Studio Code.
This information assumes that you have already installed and set up the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.
With the extension and your code project opened, do the following:
In your code project, open the R, Scala, or SQL notebook that you want to run as a job.
To create an R, Scala, or SQL notebook file in Visual Studio Code, begin by clicking File > New File, select Python File, and save the new file with a
.sqlfile extension, respectively.
To turn the
.sqlfile into a Databricks notebook, add the special comment
Databricks notebook sourceto the beginning of the file and add the special comment
COMMAND ----------before each cell. Be sure to use the correct comment marker for each language (
//for Scala, and
--for SQL). For more information, see Import a file and convert it to a notebook.
This is similar to the pattern for Python notebooks:
In Run and Debug view (View > Run), select Run on Databricks as Workflow from the drop-down list, and then click the green play arrow (Start Debugging) icon.
If Run on Databricks as Workflow is not available, see Create a custom run configuration for the Databricks extension for Visual Studio Code.
A new editor tab appears, titled Databricks Job Run. The notebook runs as a job in the workspace. The notebook and its output are displayed in the new editor tab’s Output area.
To view information about the job run, click the Task run ID link in the Databricks Job Run editor tab. Your workspace opens and the job run’s details are displayed in the workspace.