Databricks documentation archive

important

This documentation has been retired and might not be updated. The products, services, or technologies mentioned in this content are no longer supported.

In this archive, you can find earlier versions of documentation for Databricks products, features, APIs, and workflows.

Administration

• End-of-life for legacy workspaces
• Workspace-level SSO
• Administration guide (legacy)
• Workspace-level SCIM provisioning

Compute

• Create cluster UI (legacy)
• Cluster UI preview
• Install a library with an init script (legacy)
• Cluster-named init scripts (legacy)
• Global init scripts (legacy)
• Compute policies best practices

Dev tools

• Drop Delta table features (legacy)
• Manage libraries with %conda commands (legacy)
• Explore and create tables in DBFS
• Transactional writes to cloud storage with DBIO
• Browse files in DBFS
• FileStore
• Hive table (legacy)
• Koalas
• Skew join optimization using skew hints
• Legacy UniForm IcebergCompatV1
• Read Databricks tables from Apache Iceberg clients (legacy)
• Workspace libraries (legacy)
• Legacy Databricks CLI
• What is dbx by Databricks Labs?
• dbutils.library
• Migrate to Spark 3.x
• VSCode with Git folders
• VSCode workspace directory
• Pulumi Databricks resource provider
• Basic authentication (End of life)

Governance

• External metastores (legacy)
• Credential passthrough (legacy)

Machine learning and AI

• Load data using Petastorm
• Share feature tables across workspaces (legacy)
• MLeap ML model export
• Train a PySpark model and save in MLeap format
• Deploy MLeap model on SageMaker
• Distributed training with TensorFlow 2
• Horovod
• Model inference using Hugging Face Transformers for NLP
• Model serving (legacy)
• Optimized LLM serving
• Migrate optimized LLM serving endpoints to provisioned throughput
• Set up and considerations for ai_generate_text()
• Analyze customer reviews with ai_generate_text() and OpenAI
• Apache Spark MLlib and automated MLflow tracking

MLflow

• Run MLflow Projects on Databricks
• Quickstart R
• Quickstart Java and Scala

Notebooks

• No-code EDA with bamboolib

Release notes

• Databricks light
• Databricks runtime release notes (end-of-support)
• Unity Catalog GA release note

Resources

• Office hours

Security

• Audit log schemas for security monitoring
• Create and verify a cluster for legacy HIPAA support

Storage

• End-of-life for legacy workspaces
• Amazon S3 source with Amazon SQS (legacy)
• Azure Blob storage file source with Azure Queue Storage (legacy)
• Azure Cosmos DB
• Structured Streaming writes to Azure Synapse
• Connecting Databricks and Azure Synapse with PolyBase (legacy)
• Neo4j
• Read and write XML data using the spark-xml library
• Connect to external systems
• Query databases using JDBC
• Query PostgreSQL with Databricks
• Query MySQL with Databricks
• Query MariaDB with Databricks
• Query SQL Server with Databricks
• Use the Databricks connector to connect to another Databricks workspace
• Query Amazon Redshift using Databricks
• Amazon S3 Select
• MongoDB
• Cassandra
• Couchbase
• ElasticSearch
• Google BigQuery
• Read and write data from Snowflake
• Query data in Azure Synapse Analytics
• Connect to Azure Synapse Analytics dedicated pool
• Configure Delta storage credentials
• Connect to Azure Blob Storage with WASB (legacy)