Databricks reference documentation

This page contains links to comprehensive reference documentation, including links to reference for Databricks APIs, SQL, CLIs, SDKs, and other resources.

REST API reference

• • Databricks REST API
  • REST API reference for Databricks services.
• • MLflow REST API
  • REST API reference for MLflow machine learning lifecycle management and model tracking.
• • Account SCIM v2.1 API
  • REST API reference for SCIM API for user and group management across Databricks accounts.
• • Jobs v2.0 API
  • REST API reference for version 2.0 version of the Jobs REST API. Databricks recommends that you use the latest Databricks REST API for new and existing clients and scripts.

Python reference

• • PySpark API
  • Reference documentation for PySpark.
• • Delta Lake Python API
  • Python API reference for Delta Lake operations and versioning.
• • Lakeflow pipelines Python API
  • Python API reference specific to Lakeflow pipelines development and transformations.
• • Data Quality Monitoring API
  • Python API reference for Data Quality Monitoring.

AI and ML Python APIs

• • Agent Evaluation Python API
  • Reference for agent evaluation, including dataset management, review apps, labeling sessions, and built-in judges.
• • Custom Agents Python API
  • Reference for the databricks-agents package for deploying, managing permissions, and configuring agent deployments.
• • AutoML Python API
  • Reference for AutoML functionality and automated machine learning workflows.
• • Databricks AI Bridge Python API
  • Reference for the databricks-ai-bridge package, including the Genie client and shared utilities for AI Search tools.
• • databricks-langchain Python API
  • Reference for the databricks-langchain package for integrating Databricks models, embeddings, AI Search, and MCP servers with LangChain.
• • databricks-mcp Python API
  • Reference for the databricks-mcp package for connecting to MCP servers on Databricks.
• • databricks-openai Python API
  • Reference for the databricks-openai package for using Databricks-hosted models with the OpenAI SDK, including MCP server and AI Search support.
• • Feature engineering Python API
  • Reference for feature engineering and feature store operations.
• • Feature store Python API (Deprecated)
  • Deprecated. Use the Feature engineering Python API instead.
• • Lakehouse Monitoring for GenAI Python API
  • Reference for monitoring generative AI applications with Lakehouse Monitoring.
• • MLflow Python API
  • Python API reference for MLflow.
• • AI Search
  • Reference for the Python API for managing endpoints and indexes in the AI Search service.
• • Serverless GPU Python API
  • Python API reference for distributed training on AI Runtime and model inference.

Scala reference

• • Delta Lake Scala API
  • Scala API reference for Delta Lake operations and versioning.
• • Scala Spark API
  • Scala Spark API reference.

SQL reference

• • Databricks SQL
  • SQL reference for Databricks SQL, including syntax, functions, and operators.
• • Lakeflow pipelines SQL
  • SQL reference specific to Lakeflow pipelines development and transformations.

Developer tools reference

• • Databricks CLI commands
  • Command reference for the Databricks CLI.
• • Declarative Automation Bundles configuration
  • Reference for Declarative Automation Bundles configuration YAML.
• • Databricks Terraform provider
  • Reference documentation for the Databricks Terraform provider.
• • Databricks SDK for Python
  • Reference for the Python SDK for integration with Databricks and automation in Python applications.
• • Databricks SDK for R
  • Reference for the R SDK for data science workflows and statistical computing with Databricks.
• • Databricks SDK for Java
  • Reference for the Java SDK for enterprise applications and JVM-based integrations.
• • Databricks SDK for Go
  • Reference for the Go SDK for high-performance applications and cloud-native integrations.

Error reference

• • SQL error codes
  • Complete reference for SQL error codes and their meanings in Databricks.
• • Error classes
  • Error class documentation for categorized error types and resolution guidance.

Additional resources

• • Apache Spark APIs
  • Reference documentation overview for Apache Spark APIs.
• • Delta Lake APIs
  • Reference links for Delta Lake operations and versioning (Delta Spark).
• • Python development
  • Overview of Python development on Databricks.
• • Scala development
  • Overview of Scala development on Databricks.
• • R documentation
  • Overview of R-based data processing and analytics on Databricks.