Databricks documentation
Databricks documentation provides how-to guidance and reference information for data analysts, data scientists, and data engineers working in the Databricks Data Science & Engineering, Databricks Machine Learning, and Databricks SQL environments. The Databricks Lakehouse Platform, from the original creators of Apache Spark, enables data teams to collaborate to solve some of the world’s toughest problems.
Databricks excels at enabling data scientists, data engineers, and data analysts to work together on use cases like:
Applying advanced analytics for machine learning and graph processing at scale
Using deep learning for harnessing the power of unstructured data such for AI, image interpretation, automatic translation, natural language processing, and more
Making data warehousing fast, simple, and scalable
Proactively detecting threats with data science and AI
Analyzing high-velocity sensor and time-series IoT data in real-time
Making GDPR data subject requests easy to execute
Getting started
User guides
- Data Science & Engineering
- Machine Learning
- Quickstart
- 10-minute tutorials
- Databricks Machine Learning home page
- Prepare data
- Environment setup
- Databricks AutoML
- Train models
- Track model development
- Manage models
- Deploy models for inference
- Export and import models
- Reference solutions
- MLflow guide
- GraphFrames
- Machine learning workflows and Unity Catalog
- MLOps
- Databricks Feature Store
- Experiments
- Models
- Databricks SQL
- Data lakehouse
- Data discovery
- Data ingestion
- Delta Lake
- Introduction
- Delta Lake quickstart
- Introductory notebooks
- Ingest data into Delta Lake
- Table batch reads and writes
- Table streaming reads and writes
- Table deletes, updates, and merges
- Change data feed
- Table utility commands
- Constraints
- Table protocol versioning
- Delta column mapping
- Unity Catalog
- Delta Lake APIs
- Concurrency control
- Access Delta tables from external data processing engines
- Migration guide
- Best practices: Delta Lake
- Frequently asked questions (FAQ)
- Delta Lake resources
- Optimizations
- Delta table properties reference
- Developer tools
- Integrations
Administration guides
- Account and workspace administration
- Security and compliance
- Security overview
- Access control
- Secret management
- Credential passthrough
- Credential redaction
- Customer-managed keys for encryption
- Encrypt traffic between cluster worker nodes
- Secure cluster connectivity
- IP access lists
- Configure domain name firewall rules
- Best practices: GDPR and CCPA compliance using Delta Lake
- Configure access to Azure storage with an Azure Active Directory service principal
- Enable the compliance security profile
- FedRAMP Moderate compliance controls
- HIPAA compliance features
- Create and verify a cluster for legacy HIPAA support
- PCI-DSS compliance controls
- Enhanced Security Monitoring
- Audit log schemas for monitoring agents Capsule8 and ClamAV
- Data governance
- Data sharing
Reference guides