Skip to main content

Local development tools

Databricks provides an ecosystem of tools to help you develop applications and solutions that integrate with Databricks and programmatically manage Databricks resources and data.

This page provides recommendations for the best tools for common developer scenarios. For a complete overview of developer tools, see Develop on Databricks.

Tool

When to use

Databricks extension for Visual Studio Code

PyCharm Databricks plugin

For other IDEs, use the Databricks CLI with Databricks Connect

  • Interactive development and debugging from a local IDE

Databricks CLI

  • Direct interaction with Databricks from the command line
  • Shell scripting
  • Experimentation
  • Invoke the REST API directly
  • Manage local authentication profiles
  • Sync code from the IDE to the Databricks workspace

Databricks Asset Bundles (a feature of the CLI)

  • Manage workflows and deploy projects to Databricks
  • Apply CI/CD best practices
  • Co-version, co-author, and co-deploy your resources and assets as one unit
  • Supports the most common resources

Databricks Terraform provider

  • Infrastructure as code and CI/CD
  • Administer and create workspaces, catalogs, and metastores
  • Enforce permissions
  • Guarantee environment portability and disaster recovery
  • Many supported resources

Databricks Python SDK

Databricks Java SDK

Databricks Go SDK

Databricks R SDK

  • Application development
  • Integrate with existing deployment systems
  • Create custom Databricks workflows and web services

SQL drivers

  • Run SQL commands and scripts from client applications

Databricks REST API

  • Automate processes where an SDK in your preferred programming language is not available
  • Access to almost all Databricks resources
  • Advanced scenarios only