Run a Python notebook as a job by using the Databricks extension for Visual Studio Code

This article describes how to run a Python notebook as a Databricks job by using the Databricks extension for Visual Studio Code. See What is the Databricks extension for Visual Studio Code?.

To run a Python file as a Databricks job instead, see Run a Python file as a job by using the Databricks extension for Visual Studio Code. To run an R, Scala, or SQL notebook as a Databricks job instead, see Run an R, Scala, or SQL notebook as a job by using the Databricks extension for Visual Studio Code.

This information assumes that you have already installed and set up the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.

With the extension and your code project opened, do the following:

Note

The following procedure uses the Databricks extension for Visual Studio Code, version 1, which is generally available. To complete this procedure for the Databricks extension for Visual Studio Code, version 2, currently in Private Preview, skip ahead to Run a Python file as a job by using the Databricks extension for Visual Studio Code, version 2.

  1. In your code project, open the Python notebook that you want to run as a job.

    Tip

    To create a Python notebook file in Visual Studio Code, begin by clicking File > New File, select Python File, and save the new file with a .py file extension.

    To turn the .py file into a Databricks notebook, add the special comment # Databricks notebook source to the beginning of the file, and add the special comment # COMMAND ---------- before each cell. For more information, see Import a file and convert it to a notebook.

    A Python code file formatted as a Databricks notebook1

  2. Do one of the following:

    • In Explorer view (View > Explorer), right-click the notebook file, and then select Run File as Workflow on Databricks from the context menu.

      Run File as Workflow on Databricks context menu command 1

    • In the notebook file editor’s title bar, click the drop-down arrow next to the play (Run or Debug) icon. Then in the drop-down list, click Run File as Workflow on Databricks.

      Run File as Workflow on Databricks editor command 2

A new editor tab appears, titled Databricks Job Run. The notebook runs as a job in the workspace, and the notebook and its output are displayed in the new editor tab’s Output area.

To view information about the job run, click the Task run ID link in the Databricks Job Run editor tab. Your workspace opens and the job run’s details are displayed in the workspace.

Run a Python file as a job by using the Databricks extension for Visual Studio Code, version 2

Note

The Databricks extension for Visual Studio Code, version 2 is in Private Preview.

This procedure assumes that have already installed and set up the Databricks extension for Visual Studio Code, version 2. See Install and open the Databricks extension for Visual Studio Code, version 2.

  1. In your code project, open the Python notebook that you want to run as a job.

    Tip

    To create a Python notebook file in Visual Studio Code, begin by clicking File > New File, select Python File, and save the new file with a .py file extension.

    To turn the .py file into a Databricks notebook, add the special comment # Databricks notebook source to the beginning of the file, and add the special comment # COMMAND ---------- before each cell. For more information, see Import a file and convert it to a notebook.

    A Python code file formatted as a Databricks notebook1

  2. Do one of the following:

    • In Explorer view (View > Explorer), right-click the notebook file, and then select Run on Databricks > Run File as Workflow from the context menu.

    • In the notebook file editor’s title bar, click the Run on Databricks icon, and then click Run File as Workflow.

A new editor tab appears, titled Databricks Job Run. The notebook runs as a job in the workspace, and the notebook and its output are displayed in the new editor tab’s Output area.

To view information about the job run, click the Task run ID link in the Databricks Job Run editor tab. Your workspace opens and the job run’s details are displayed in the workspace.