Databricks platform release notes

• March 2025
  • Install custom dependencies in Unity Catalog Python UDFs (Public Preview)
  • Web terminal activity is now considered compute activity
  • Budget policies renamed to serverless budget policies
  • Delta Sharing supports renaming tables
  • OIDC federation for Databricks-to-open Delta Sharing (Public Preview)
  • The compute system tables are enabled by default
  • Hive metastore federation is generally available
  • Compute logs can now be delivered to volumes (Public Preview)
  • Download as Excel is now supported in notebooks connected to serverless compute
  • Bind storage credentials, service credentials, and external locations to specific workspaces (GA)
  • Databricks Runtime 16.3 (Beta)
  • Databricks JDBC Driver (OSS) is GA
  • Monitor and revoke personal access tokens in your account (Public Preview)
  • Genie Conversation API is now in Public Preview
  • Perform batch inference using AI Functions (Public Preview)
  • ServiceNow connector (Public Preview)
  • External models now supports OpenAI-compatible LLMs
  • Agent Framework monitoring is in Beta
  • Mosaic AI Agent Framework is generally available
  • Delta Sharing support for query optimizer statistics is GA
  • AI Gateway now supports fallbacks for external models
  • Control the location of Databricks Assistant with the new docking feature
• February 2025
  • Databricks ODBC driver 2.9.1
  • Publish pipeline event logs as Unity Catalog tables
  • FedRAMP High and Department of Defense IL5 Authorization is now authorized
  • Databricks Assistant provides table insights and lineage
  • Import open Delta Sharing shares directly into Databricks using the Import provider dialog
  • Delta Sharing from open sharing providers to Databricks using OAuth
  • Streaming reads on Unity Catalog views is GA
  • Event time ordering for initial snapshot stream processing is GA
  • Asynchronous progress tracking is GA
  • New filter panel in Databricks Marketplace
  • OAuth secret lifetime now configurable
  • Preview files in volumes
  • Model serving billing records are now logged at five-minute intervals
  • Automatic liquid clustering (Public Preview)
  • Databricks Runtime 14.1 series support ends
  • Serverless compute can now use instance profiles for data access
  • Databricks Runtime 16.2 is GA
  • Support of Scala streaming foreach, foreachBatch, and flatMapGroupsWithState on standard access mode compute (formerly shared access mode)
  • Unity Catalog-governed access to external cloud services using service credentials is now GA
  • Download as Excel is now supported in notebooks connected to SQL warehouses
  • Write data from pipelines to external services with DLT sinks (Public Preview)
• January 2025
  • State store reader is now GA
  • Predictive optimization can now be enabled at the catalog or schema level
  • Filtering full datasets for large tables is now supported
  • Databricks on AWS GovCloud DoD is now GA
  • The Meta Llama 3.1 405B model family is retired on Foundation Model Fine-tuning
  • Clean Rooms is GA
  • Connect AI agent tools to external services (Public Preview)
  • Unified login UI update
  • DLT now supports publishing to tables in multiple schemas and catalogs
  • Databricks Runtime 16.2 (Beta)
  • Comments now support email notifications and @ mentions
  • Shortcut to adjust font size
  • OAuth token federation is now available in Public Preview
  • Custom Python AI agents now support AI Gateway and streaming output
  • Import workspace files with drag-and-drop
  • Meta Llama 3.3 now powers AI Functions that use Foundation Model APIs
  • Notebook output improvements
  • Faster notebook load times
  • Notebooks are now supported as workspace files
  • Failed tasks in continuous jobs are now automatically retried
  • Notebooks: Databricks Assistant chat history available only to user who initiates it
  • Stats collection is now automated by predictive optimization
  • Update to Databricks Marketplace and Partner Connect UI
  • EXPLAIN now shows stats used in query planning
  • Databricks JDBC driver 2.7.1
  • Lakehouse Federation supports Teradata (Public Preview)
  • databricks-agents SDK 0.14.0 release: Custom evaluation metrics
  • AI Gateway now supports provisioned throughput (Public Preview)
  • Databricks Runtime 15.2 series support ends
  • Databricks Runtime 15.3 series support ends
  • Meta Llama 2, 3 and Code Llama model family retirements on Foundation Model Fine-tuning
• December 2024
  • Databricks Runtime 16.1 is GA
  • The default format for new notebooks is now IPYNB (Jupyter) format
  • Python code executor for AI agents (Public Preview)
  • databricks-agents SDK 0.13.0 release
  • External groups are now labeled and immutable
  • vector_search now supports vector embedding inputs
  • Specify response formats for ai_query
  • Test clean rooms with collaborators within the same metastore
  • Assign compute resources to groups (Public Preview)
  • Version 2.2 of the Jobs API is released
  • Unity Catalog MANAGE privilege (Public Preview)
  • Meta Llama 3.3 is now available for provisioned throughput workloads
  • View streaming workload metrics for your job runs (Public Preview)
  • View streaming workload metrics for your DLT pipeline updates (Public Preview)
  • Lakehouse Federation supports Oracle (Public Preview)
  • Unified login is now enabled on accounts without single sign-on
  • Databricks Runtime 16.1 (Beta)
  • Manage serverless outbound network connections with serverless egress control
  • Network access events system table is now available (Public Preview)
  • Monitor and revoke personal access tokens in your account (Private Preview)
  • Unity Catalog can federate to Hive metastores and AWS Glue
  • Remove metastore-level storage to enforce catalog-level storage isolation
  • Meta Llama 3.3 70B Instruct is now available on Model Serving
  • bamboolib is now deprecated
  • Streamline AI Agent Evaluation using synthetic evaluation sets
  • Improve Databricks-to-Databricks Delta Sharing table read performance with history sharing (Public Preview)
  • Maximum personal access token lifetime now 730 days (two years)
  • Mosaic AI Model Training - serverless forecasting (Public Preview)
  • Add serverless budget policies to model serving endpoints
• November 2024
  • Additional metadata for the table tooltip in notebooks
  • Databricks Assistant can use Databricks-hosted models
  • FedRAMP Moderate and PCI-DSS now support serverless compute in us-west-2
  • Emergency access for workspaces without unified login
  • Automatic cluster update and enhanced security and compliance UI are generally available
  • Expanded support for Python debugger in notebooks
  • Improved web terminal experience
  • Query history and query profile support for DLT
  • Mosaic AI Model Training rebrand
  • Reuse SQL outputs as _sqldf in subsequent SQL cells
  • Warehouses system table is now available (Public Preview)
  • Cross-platform view sharing (Public Preview)
  • Clean Rooms public preview is ungated, with new features
  • Databricks Runtime 16.0 is GA
  • Breaking change: Hosted RStudio is end-of-life
  • Predictive optimization enabled for all new Databricks accounts
  • Updated and unified read-only code blocks
  • Increased Quick Fix recommendations
  • Foundation Model APIs pay-per-token available in all Model Serving supported regions
  • Unity Catalog-governed access to external cloud services using service credentials (Public Preview)
  • Databricks Runtime 14.2 series support ends
• October 2024
  • Enable cost attribution of fine-grained access control on dedicated compute (formerly single user compute)
  • Databricks Assistant helps optimize SQL queries
  • Databricks Runtime 15.1 series support ends
  • Llama 2 70B Chat model retirement in Foundation Model APIs pay-per-token
  • Enable a metastore to be assigned automatically to new workspaces
  • Configure an encryption algorithm on an external location
  • Structured outputs supported on Mosaic AI Model Serving (Public Preview)
  • Fine-grained access control on dedicated compute (formerly single user compute) is generally available
  • AI Functions now available in EU regions
  • Attribute serverless usage with serverless budget policies (Public Preview)
  • Serverless compute automatically enabled on eligible accounts
  • The billable usage system table now attributes networking costs
  • Databricks Assistant Autocomplete is now generally available
  • Filters on notebook results tables now persist
  • Improvements to comments in notebooks
  • Databricks Runtime 16.0 (Beta)
  • Mosaic AI Model Serving now supports batch inference
  • Create secure data and AI apps for your users with Databricks Apps (Public Preview)
  • More reliable reloading of modified Python modules with improvements to autoreload
  • Salesforce ingestion connector simplification
  • Compliance security profile with serverless compute support
  • Serverless compute is now available in the eu-west-2 region
• September 2024
  • Meta Llama 3.1 405B Instruct now supported on Databricks Function Calling
  • vector_search() now available to all customers (Public Preview)
  • External models now supports instance profiles for Amazon Bedrock
  • Emergency access no longer requires single sign-on
  • Meta Llama 3.2 3B and 1B models are supported in Model Serving
  • Meta Llama 3.2 3B and 1B models are supported in Foundation Model Fine-tuning
  • Publish to Power BI is now generally available
  • Use @ to reference tables in Databricks Assistant prompts
  • Prototype and export tool-calling generative AI agents in AI Playground
  • Control external access to data in Unity Catalog using the new EXTERNAL USE SCHEMA privilege
  • GTE v1.5 (English) embedding models are now supported in Foundation Model API provisioned throughput
  • Databricks Assistant Quick Fix debugs code inline
  • AI Gateway is now Public Preview
  • Meta Llama 3.1 70B and 8B models supported in Foundation Model Fine-tuning
  • Extended AI-generated comments support
  • Monitor clean room usage in the billable usage table
  • Databricks extension for Visual Studio Code is GA
  • Function calling is now supported on Foundation Model APIs provisioned throughput
  • Foundation Model APIs pay-per-token available in eu-central-1 and eu-west-1
  • System tables are now generally available
• August 2024
  • Monitor Unity Catalog object usage against quotas using the new Resource Quotas APIs
  • MPT 7B Instruct and MPT 30B Instruct model retirements
  • AskSupport replaced by Databricks assistant
  • Lakeflow system tables are extend with additional columns
  • Python code formatting error highlights
  • Databricks on AWS GovCloud is now GA
  • Delta Sharing: More Delta Lake features now supported by the Python and Power BI connectors
  • Delta Sharing adds support for TimestampNTZ
  • The Databricks Jobs For each task is GA
  • Databricks Runtime 15.4 LTS is GA
  • Personalized notebook autocomplete
  • Symlinks are now supported in workspace files
  • Configure your workspace’s default access mode for jobs compute
  • New slash commands for Databricks Assistant
  • Workspace search now supports volumes
  • Databricks JDBC driver 2.6.40
  • Databricks personal access tokens revoked if unused after 90 days
  • Serverless SQL warehouses with the compliance security profile is now GA
  • Clusters API now supports partial configuration updates
  • Wrap lines in notebook cells
  • Specify columns to sync for Mosaic AI Vector Search
  • Files can no longer have identical names in workspace folders
  • Compute policy enforcement now available
  • Foundation Model APIs pay-per-token is now GA
  • Collaborate on data projects securely and privately using Databricks Clean Rooms (Public Preview)
  • Mosaic AI Vector Search is now HIPAA-compliant
  • Format columns in notebook and query results tables
  • Row filters and column masks are now GA, with improvements
  • Canadian Centre for Cybersecurity (CCCS) Medium (Protected B) compliance controls
  • Lakehouse Federation is generally available (GA)
  • Meta Llama 3.1 405B models supported in Foundation Model Fine-tuning
  • Create a new workspace with enhanced security and compliance features
• July 2024
  • Increased limit for simultaneous tasks
  • Embed and drag & drop images in notebooks
  • Command palette available in notebooks
  • Workflow system schema renamed to lakeflow
  • Lakeflow Connect (gated Public Preview)
  • Support for Cloudflare R2 storage is GA
  • Monitor Databricks Assistant activities with system tables (Public Preview)
  • Sharing schemas using Delta Sharing is now GA
  • Mosaic AI Agent Framework is available in eu-central-1
  • Databricks Assistant can diagnose issues with jobs (Public Preview)
  • Updates to Databricks Git folders authentication and sharing behaviors
  • Cluster library installation timeout
  • Compute plane outbound IP addresses must be added to a workspace IP allow list
  • Databricks Runtime 9.1 series support extended
  • Single sign-on (SSO) is supported in Lakehouse Federation for SQL Server
  • Enable cross-Geo processing
  • Model sharing using Delta Sharing is now generally available
  • Share comments and primary key constraints using Delta Sharing
  • New Databricks JDBC Driver (OSS)
  • Databricks Runtime 15.4 LTS (Beta)
  • Scala is GA on Unity Catalog standard compute
  • Dedicated compute (formerly single user) supports fine-grained access control, materialized views, and streaming tables
  • Node timeline system table is now available (Public Preview)
  • Meta Llama 3.1 is now supported in Model Serving
  • Unity Catalog will soon drop support for storage credentials that use non-self-assuming IAM roles
  • Notebooks: toggle more visible cell titles
  • / in workspace asset names is deprecated
  • Delta Sharing lets you share tables that use liquid clustering
  • Query history system table is now available (Public Preview)
  • Vulnerability scan reports are now emailed to admins
  • Partition metadata logging for Unity Catalog external tables
  • Serverless compute for workflows is GA
  • Serverless compute for notebooks is GA
  • Databricks Connect for Python now supports serverless compute
  • Filter data outputs using natural language prompts
  • Plaintext secrets support for external models
  • Forecast time series data using ai_forecast()
  • SQL File task support for files with multi-statement SQL queries is GA
  • Lakehouse Federation supports Salesforce Data Cloud (Public Preview)
  • Databricks Assistant system table now available (Public Preview)
  • Account SCIM API v2.1 (Public Preview)
  • Foundation Model Fine-tuning available to all us-west-2 customers (Public Preview)
  • UK Cyber Essentials Plus compliance controls
  • End of life for Databricks-managed passwords
  • Sign-in with one-time passcodes and external accounts
  • Resource quota increase for tables per Unity Catalog metastore
  • Databricks Assistant can diagnose notebook errors automatically
  • Support for the :param syntax with the SQL file task is GA
  • OAuth in Databricks on AWS GovCloud
• June 2024
  • AI-generated comments (GA)
  • Notebooks - out of order syntax highlighting
  • Notebooks Assistant automatically diagnoses errors
  • Catalog Explorer improvements
  • Serve multiple external models from a single model serving endpoint
  • Databricks Assistant is now GA
  • Databricks Assistant now gives titles to threads
  • Serverless notebooks environment manager
  • Notebooks debugging console
  • Quickly add a periodic schedule to a Databricks workflow
  • Emergency access for single sign-on GA
  • Databricks Runtime 15.3 is GA
  • Databricks Geos is GA
  • New improved catalog creation UI
  • The billing system tables are enabled by default
  • Lakehouse Monitoring is GA
  • Volume sharing using Delta Sharing is now generally available
  • Create budgets to monitor account spending (Public Preview)
  • Importable cost management dashboard added to account console (Public Preview)
  • Unified login is now generally available
  • Customer managed keys (CMK) for Databricks Vector Search are now GA
  • Mosaic AI Agent Framework (Public Preview)
  • Predictive optimization is now GA
  • Route optimization is available for serving endpoints
  • Column mapping is now GA
  • Workflow system tables are now available (Public Preview)
  • Function calling is Public Preview
  • Query a vector search index using vector_search()
  • Mosaic AI Vector Search now supports hybrid search
  • GTE is now supported in Model Serving
  • Route optimized serving endpoints enforce network restrictions
  • Databricks Assistant: Threads & queries experience enabled by default
  • New entry points for in-product help
  • Improved notebook Markdown editor
  • Quickly access Catalog Explorer tables from notebooks
  • Databricks ODBC driver 2.8.2
  • The new Databricks Notebook UI is now generally available
• May 2024
  • Serverless firewall configuration now supports more compute types
  • Databricks Runtime 15.0 series support ends
  • Databricks Runtime 15.3 (Beta)
  • The compute metrics UI is now available on all Databricks Runtime versions
  • Improved search and filtering in notebook and SQL editor results tables
  • New dashboard helps Databricks Marketplace providers monitor listing usage
  • View system-generated federated queries in Query Profile
  • Compute plane outbound IP addresses must be added to a workspace IP allow list
  • OAuth is supported in Lakehouse Federation for Snowflake
  • Bulk move and delete workspace objects from the workspace browser
  • Unity Catalog objects are available in recents and favorites
  • New dbt-databricks connector 1.8.0 adopts decoupled dbt architecture
  • New compliance and security settings APIs (Public Preview)
  • Databricks Runtime 15.2 is GA
  • New Tableau connector for Delta Sharing
  • New deep learning recommendation model examples
  • Bind storage credentials and external locations to specific workspaces (Public Preview)
  • Git folders are GA
  • Pre-trained models in Unity Catalog (Public Preview)
  • Mosaic AI Vector Search is GA
  • The compliance security profile now supports AWS Graviton instance types
  • Databricks Assistant autocomplete (Public Preview)
  • Meta Llama 3 support in Foundation Model Training
  • New changes to Git folder UI
  • Compute now uses EBS GP3 volumes for autoscaling local storage
  • Unified Login now supported with AWS PrivateLink (Private Preview)
  • Foundation Model Training (Public Preview)
  • Allow users to copy data to the clipboard from results table
  • Attribute tag values for Unity Catalog objects can now be 1000 characters long (Public Preview)
  • New Previews page
  • New capabilities for Mosaic AI Vector Search
  • Credential passthrough and Hive metastore table access controls are deprecated
  • Databricks JDBC driver 2.6.38
  • Unified Login now supported with AWS PrivateLink (Public Preview)
  • Databricks Runtime 15.2 (Beta)
  • Notebooks now detect and auto-complete column names for Spark Connect DataFrames
• April 2024
  • Databricks Runtime 15.1 is GA
  • Databricks Assistant: Threads & history
  • Cancel pending serving endpoint updates in Model Serving
  • Data lineage now captures reads on tables with column masks and row-level security
  • Meta Llama 3 is supported in Model Serving for AWS
  • Notebooks now automatically detect SQL
  • New columns added to the billable usage system table (Public Preview)
  • Delta Sharing supports tables that use column mapping (Public Preview)
  • Get serving endpoint schemas (Public Preview)
  • Creation and installation of workspace libraries is no longer available
  • Jobs created through the UI are now queued by default
  • Configuring access to resources from serving endpoints is GA
  • Serverless compute for workflows is in public preview
  • Lakehouse Federation supports foreign tables with case-sensitive identifiers
  • Compute cloning now clones any libraries installed on the original compute
  • Route optimization is available for serving endpoints
  • DLT notebook developer experience improvements (Public Preview)
  • Databricks on AWS GovCloud (Public Preview)
• March 2024
  • DBRX Base and DBRX Instruct are now available in Model Serving
  • Model Serving is HIPAA compliant in all regions
  • Provisioned throughput in Foundation Model APIs is GA and HIPAA compliant
  • MLflow now enforces quota limits for experiments and runs
  • The Jobs UI is updated to better manage jobs deployed by Databricks Asset Bundles
  • Google Cloud Vertex AI supported as model provider for external models
  • Accessing resources from serving endpoints using instance profiles is GA
  • Interactive notebook debugging
  • Self-service sign-up for private exchange providers in Marketplace
  • Databricks Runtime 15.0 is GA
  • Databricks Repos changed to Git folders
  • Databricks Runtime 14.1 and 14.2 series support extended
  • Databricks ODBC driver 2.8.0
  • SQL warehouses for notebooks is GA
  • Delegate the ability to view an object’s metadata in Unity Catalog (Public Preview)
  • Databricks Runtime 15.0 (Beta)
  • Databricks Runtime 14.0 series support ends
  • New computation for sys.path and CWD in Repos
  • Feature Serving is GA
  • Predictive optimization available in more regions
• February 2024
  • Use DLT in Feature Engineering (Public Preview)
  • Restrict creating a personal access token for a service principal
  • Restrict changing a job owner and the run as setting
  • Automatic cluster update is enabled if the compliance security profile is enabled (GA)
  • Account admins can enable enhanced security and compliance features (Public Preview)
  • Support for Cloudflare R2 storage to avoid cross-region egress fees (Public Preview)
  • Notebooks for monitoring and managing Delta Sharing egress costs are now available
  • Add data UI supports XML file format
  • Support for cloud storage firewall from serverless compute (Public Preview)
  • Use AI Functions to invoke a generative AI model from Foundation Model APIs
  • Unity Catalog volumes are GA
  • Full-page AI-powered search
  • Run SQL notebook jobs on a SQL warehouse
  • File arrival triggers in Databricks Jobs is GA
  • Search for machine learning models in Unity Catalog using global workspace search
  • Databricks Git server proxy is GA
  • Databricks Git server proxy no longer requires CAN_ATTACH_TO permissions
  • Workspace file support for the dbt and SQL file tasks is GA
  • Databricks Connect is GA for Scala
  • Create tables from files in volumes
  • Databricks Runtime 14.3 LTS is GA
  • Delta Sharing supports tables that use deletion vectors (Public Preview)
• January 2024
  • Native XML file format support (Public Preview)
  • Share AI models using Databricks Marketplace (Public Preview)
  • Workspace path update
  • Streamlined creation of Databricks jobs
  • Monitor GPU model serving workloads using inference tables
  • URI path-based access to Unity Catalog external volumes
  • Access controls lists can be enabled on upgraded workspaces
  • Marketplace listing events system table now available (Public Preview)
  • Updated UI for notebook cells (Public Preview)
  • Quick Fix help with syntax errors in the notebook
  • Databricks Runtime 14.3 LTS (Beta)
  • Share AI models using Delta Sharing (Public Preview)
  • Databricks Marketplace supports volume sharing
  • Create widgets from the Databricks UI
  • Warehouse events system table is now available (Public Preview)
  • UI experience for OAuth app registration
  • Reuse subnets across workspaces for customer-managed VPCs
  • Workspace file size limit is now 500MB
  • Feature removal notice for legacy Git integration in Databricks
  • Databricks ODBC driver 2.7.7
  • Databricks Runtime 13.2 series support ends
• December 2023
  • Share dynamic views using Delta Sharing (Public Preview)
  • Share volumes using Delta Sharing (Public Preview)
  • Entity Relationship Diagram for primary keys and foreign keys
  • Unity Catalog volume file upload size limit increase
  • New notebook cell results rendering available (Public Preview)
  • Notebook editor themes available
  • External models support in Model Serving is in Public Preview
  • Databricks Online Tables is Public Preview
  • Repos & Git Integration Settings UI now correctly notes support for GitHub Enterprise Server
  • Databricks JDBC driver 2.6.36
  • Support for referencing workspace files from init scripts
  • Feature & Function Serving is Public Preview
  • Foundation Model APIs is Public Preview
  • New unified admin settings UI
  • Init scripts on DBFS are end-of-life
  • Legacy global and cluster-named init scripts are end-of-life
• November 2023
  • Mosaic AI Vector Search is Public Preview
  • IAM policies for storage credentials now require an external ID
  • Access controls lists can no longer be disabled
  • AI assistive features are enabled by default
  • New behaviors and actions in Catalog Explorer for volumes
  • Databricks Runtime 14.2 is GA
  • Databricks SQL Connector for Python version 3.0.0
  • Libraries in workspace files supported on no-isolation shared clusters
  • Deprecation of workspace libraries
  • Delegate the ability to create a storage credential in Unity Catalog
  • Search for Databricks Marketplace listings using global workspace search
  • Consume data products in Databricks Marketplace using external platforms
  • Automatic enablement of Unity Catalog for new workspaces
  • Authentication using OAuth is GA
  • Databricks Runtime 14.2 (beta)
  • Databricks Marketplace includes Databricks Solution Accelerators
  • Lakehouse Federation adds support for Google BigQuery
• October 2023
  • Published partner OAuth applications are easier to use by default
  • View the YAML source for a Databricks job
  • Add conditional logic to your Databricks workflows
  • Configure parameters on a Databricks job that can be referenced by all job tasks
  • Support for new GPU instance types
  • Auto-enable deletion vectors
  • Unity Catalog support for UNDROP TABLE is GA
  • Partner Connect supports Dataiku
  • AutoML generated notebooks are now saved as MLflow artifacts
  • Predictive optimization (Public Preview)
  • Compute system tables are now available (Public Preview)
  • On-demand feature computation is GA
  • Feature Engineering in Unity Catalog is GA
  • AI-generated table comments (Public Preview)
  • Compliance security profile works with serverless SQL warehouses in ap-southeast-2 region (Public Preview)
  • Models in Unity Catalog is GA
  • Libraries now supported in compute policies (Public Preview)
  • Partner Connect supports Monte Carlo
  • Semantic search (Public Preview)
  • Enable Databricks Assistant at the workspace level
  • IP access lists for the account console is GA
  • New Photon defaults
  • Databricks Runtime 14.1 is GA
  • Developer tools release notes have moved
  • Databricks extension for Visual Studio Code updated to version 1.1.5
  • Predictive I/O for updates is GA
  • Deletion vectors are GA
  • Automatic enablement of Unity Catalog for new workspaces
  • Infosec Registered Assessors Program (IRAP) compliance controls
  • Partner Connect supports RudderStack
  • Databricks CLI updated to version 0.207.0 (Public Preview)
  • Run selected cells in a notebook
  • Use workspace-catalog binding to give read-only access to a catalog
  • New in-product Help experience (Public Preview)
  • Databricks extension for Visual Studio Code updated to version 1.1.4
  • Databricks SDK for Python updated to version 0.10.0 (Beta)
  • Databricks SDK for Go updated to version 0.22.0 (Beta)
• September 2023
  • Lakehouse Federation available on single user clusters
  • Databricks Runtime 14.1 (Beta)
  • GPU model serving optimized for LLMs in Public Preview
  • Prevent runs of a job from being skipped when you reach concurrency limits
  • Databricks Terraform provider updated to version 1.27.0
  • Databricks CLI updated to version 0.206.0 (Public Preview)
  • Databricks SDK for Go updated to version 0.21.0 (Beta)
  • Inference tables for model serving endpoints is Public Preview
  • Databricks ODBC driver 2.7.5
  • Databricks extension for Visual Studio Code updated to version 1.1.3
  • Running jobs as a service principal is GA
  • Databricks CLI updated to version 0.205.2 (Public Preview)
  • Databricks CLI updated to version 0.205.1 (Public Preview)
  • Databricks SDK for Go updated to version 0.20.0 (Beta)
  • Databricks SDK for Python updated to version 0.9.0 (Beta)
  • Databricks SDK for Python updated to version 0.7.1 (Beta)
  • Databricks Terraform provider updated to version 1.26.0
  • Databricks Connect for Databricks Runtime 14.0
  • Configure Tableau and Power BI OAuth with SAML SSO
  • Lakeview dashboards in Public Preview
  • Fleet clusters now support Graviton instances
  • Databricks Connect V2 is Public Preview for Scala
  • GPU model serving in Public Preview
  • On-demand feature computation now available in Unity Catalog (Public Preview)
  • Unified login for accounts created before June 21, 2023 (Public Preview)
  • Databricks Runtime 14.0 is GA
  • Account email notifications upgrade
  • Databricks Terraform provider updated to version 1.25.1
  • Databricks SDK for Go updated to version 0.19.2 (Beta)
  • Databricks CLI updated to version 0.205.0 (Public Preview)
  • Reserved location for /Workspace
  • Databricks Asset Bundles (Public Preview)
  • Databricks Terraform provider updated to version 1.25.0
  • Databricks extension for Visual Studio Code updated to version 1.1.2
  • Partner Connect supports Snowplow
  • Databricks CLI updated to version 0.204.1 (Public Preview)
  • Filter sensitive data with row filters and column masks (Public Preview)
  • System tables now include Marketplace schema (Public Preview)
  • Pricing system table is now available (Public Preview)
  • Databricks SDK for Go updated to version 0.19.1 (Beta)
  • Databricks ODBC driver 2.7.3
  • Databricks CLI updated to version 0.204.0 (Public Preview)
  • Data Explorer is now Catalog Explorer
  • Databricks SDK for Go updated to version 0.19.0 (Beta)
  • Databricks SDK for Python updated to version 0.8.0 (Beta)
  • Delegate allowlist privileges in Unity Catalog
  • GitHub apps integration in Repos is GA
• August 2023
  • Tables now appear in navigational search
  • Databricks CLI updated to version 0.203.3 (Public Preview)
  • Databricks JDBC driver 2.6.34
  • Databricks SDK for Go updated to version 0.18.0 (Beta)
  • Databricks SDK for Python updated to version 0.7.0 (Beta)
  • Databricks Terraform provider updated to version 1.24.1
  • Init scripts on DBFS end of life extended to Dec 1, 2023
  • Databricks Runtime 14.0 (Beta)
  • Custom workspace tags
  • Unified navigation experience is GA
  • Allow additional ports from your classic compute plane to control plane by January 31, 2024
  • Databricks Terraform provider updated to version 1.24.0
  • Standard access mode (formerly shared access mode) clusters can now connect to public networks outside of Databricks VPC
  • Databricks Runtime for Genomics setting removed from the workspace admin settings page
  • Container Services setting removed from the workspace admin settings page
  • Databricks CLI updated to version 0.203.2 (Public Preview)
  • Go to definition for functions and variables in Python notebooks
  • Databricks Runtime 13.3 LTS is GA
  • Introducing tags with Unity Catalog
  • Allowlist for init scripts, JARs, and Maven coordinates on Unity Catalog shared clusters is in Public Preview
  • Volumes support for init scripts and JARs is in Public Preview
  • Easier Databricks Repos .ipynb file output commits
  • IPYNB notebook support in Databricks Repos is GA
  • Databricks SDK for Go updated to version 0.17.0 (Beta)
  • Databricks SDK for Python updated to version 0.6.0 (Beta)
  • Databricks CLI updated to version 0.203.1 (Public Preview)
  • Unified schema browser is now GA
  • Databricks SDK for Go updated to version 0.16.0 (Beta)
  • Partner Connect supports Census
  • Databricks SDK for Python updated to version 0.5.0 (Beta)
  • Programmatic write support for workspace files
  • Access resources from serving endpoints with instance profiles (Public Preview)
  • Databricks CLI updated to version 0.203.0 (Public Preview)
  • Databricks Terraform provider updated to version 1.23.0
  • Groups can now be renamed
  • Databricks SDK for Go updated to version 0.15.0 (Beta)
  • Databricks SDK for Python updated to version 0.4.0 (Beta)
  • Third-party iFraming prevention configuration setting was removed
  • Databricks extension for Visual Studio Code updated to version 1.1.1
  • LangChain available in 13.1 and above
  • Feature Engineering in Unity Catalog is Public Preview
  • Improved error handling for repeated continuous job failures
  • Share schemas using Delta Sharing (Public Preview)
  • Run tasks conditionally in your Databricks jobs
  • Compliance security profile works with serverless SQL warehouses in some regions (Public Preview)
  • Databricks Terraform provider updated to version 1.22.0
  • Lakehouse Monitoring is Public Preview
  • Databricks Runtime 13.3 LTS (Beta)
  • New Git operations are generally available: Merge branches, rebase and pull with conflict resolution
• July 2023
  • Email addresses in Databricks are now case insensitive
  • Workspace admins can now create account groups
  • Group manager role is in Public Preview
  • Databricks CLI updated to version 0.202.0 (Public Preview)
  • Databricks SDK for Python updated to version 0.3.0 (Beta)
  • Databricks SDK for Go updated to version 0.14.1 (Beta)
  • Databricks SDK for Go updated to version 0.14.0 (Beta)
  • Run another job as a task in a Databricks job
  • All users can access data products in Databricks Marketplace by default
  • Classic keyboard shortcuts mode
  • Lakehouse Federation lets you run queries against external database providers (Public Preview)
  • Move to trash enabled for Repos
  • Create alerts for slow-running or stuck jobs
  • Databricks SDK for Go updated to version 0.13.0 (Beta)
  • Databricks SDK for Python updated to version 0.2.0 (Beta)
  • Databricks CLI updated to version 0.201.0 (Public Preview)
  • Databricks SDK for Python updated to version 0.2.1 (Beta)
  • Databricks Assistant is in Public Preview
  • Deactivate users and service principals from your account
  • Account-level SCIM provisioning now deactivates users when they are deactivated in the identity provider
  • Trash directory admin access
  • Prevention of MIME type sniffing and XSS attack page rendering are now always enabled
  • Unity Catalog volumes are in Public Preview
  • Simplified experience for submitting product feedback from the workspace
  • Databricks extension for Visual Studio Code updated to version 1.1.0
  • Functions now displayed in Catalog Explorer (Public Preview)
  • Databricks Terraform provider updated to version 1.21.0
  • The maximum offset for the List all jobs and List job runs API requests is now limited
  • Databricks Runtime 13.2 is GA
  • Delta Sharing and Databricks Marketplace support view sharing (Public Preview)
  • Init scripts on DBFS reach end of life on Sept 1, 2023
• June 2023
  • Databricks Terraform provider updated to version 1.20.0
  • Databricks CLI updated to version 0.200.1 (Public Preview)
  • Databricks SDK for Go updated to version 0.12.0 (Beta)
  • Databricks SDK for Go updated to version 0.11.0 (Beta)
  • Databricks SDK for Python (Beta)
  • Databricks SDK for Go (Beta)
  • Access audit log, billable usage, and lineage system tables (Public Preview)
  • Models in Unity Catalog (Public Preview)
  • Databricks Marketplace is now GA
  • Databricks Runtime 13.2 (Beta)
  • Databricks extension for Visual Studio Code (General Availability)
  • See a visual overview of completed job runs in the Databricks Jobs UI
  • Databricks CLI (Public Preview)
  • Unified Login GA for new accounts
  • Test single sign-on
  • Improved pagination of results from List all jobs and List job runs API requests
  • Full-page workspace browser includes Repos
  • Databricks Terraform provider updated to version 1.19.0
  • Run jobs as a service principal (Public Preview)
  • New service principal UI provides better management experience
  • Home folders restored when users are re-added to workspaces
  • Databricks Marketplace: Private exchanges are now available
  • Databricks Marketplace: Consumers can uninstall data products using the UI
  • Databricks Marketplace: providers can create their own profiles
  • Databricks Connect V2 is GA for Python
  • Databricks Terraform provider updated to version 1.18.0
  • New Databricks Marketplace providers
  • Databricks Runtime 13.1 is GA
  • Partner Connect supports Hunters
  • View data from notebooks, SQL editor, and Catalog Explorer with the unified schema browser (Public Preview)
• May 2023
  • Create or modify table UI supports Avro, Parquet, and text file uploads
  • Use the add data UI to load data using Unity Catalog external locations (Public Preview)
  • Run Databricks notebooks on SQL warehouses (Public Preview)
  • Prevent Enter key from accepting autocomplete suggestion
  • Databricks Terraform provider updated to version 1.17.0
  • Upload data UI supports new column data types
  • Configure your workspace to use IMDS v2 (GA)
  • M7g and R7g Graviton3 instances are now supported on Databricks
  • Bind Unity Catalog catalogs to specific workspaces
  • All users can connect to Fivetran using Partner Connect (Public Preview)
  • Authentication using OAuth tokens for service principals (Public Preview)
  • Schedule automatic cluster update (Public Preview)
  • Databricks Terraform provider updated to version 1.16.1
  • Databricks JDBC driver 2.6.33
  • Partner Connect supports Alation
  • New default theme for editor
  • Databricks Terraform provider updated to version 1.16.0
  • New region: Europe (Paris)
  • Compliance security profile now supports more EC2 instance types
  • Databricks Runtime 13.1 (Beta)
  • Run file-based SQL queries in a Databricks workflow
  • Databricks Terraform provider updated to version 1.15.0
  • Account nicknames now available in the account console
  • Share notebooks using Delta Sharing
  • Deprecation of cluster-scoped init scripts on DBFS
  • New region: South America (São Paulo)
  • Unified navigation (Public Preview)
  • AWS fleet instance types now available
• April 2023
  • Databricks Marketplace (Public Preview): an open marketplace for data, analytics, and AI
  • Configure the Python formatter
  • Workspace files are GA
  • Cluster-scoped init scripts can now be stored in workspace files
  • New Delta Sharing privileges enable delegation of share, recipient, and provider management tasks
  • Databricks Connect V2 (Public Preview)
  • New cluster metrics UI
  • Databricks Runtime 13.0 is GA
  • Control access to the account console by IP address ranges (Public Preview)
  • Databricks Terraform provider updated to version 1.14.3
  • Audit log entries for changed admin settings for workspaces and accounts
  • Workspaces with security profile or ESM include audit logs rows for system and monitor logs
  • Databricks Terraform provider updated to versions 1.14.1 and 1.14.2
  • Combined SQL user settings and general user settings
  • Legacy notebook visualizations deprecated
  • Create or modify table from file upload page supports JSON file uploads
• March 2023
  • Databricks Terraform provider updated to version 1.14.0
  • Databricks Runtime 7.3 LTS ML support ends
  • C7g Graviton 3 instances are now supported on Databricks
  • Distributed training with TorchDistributor
  • Databricks Runtime 13.0 (Beta)
  • Improved file editor
  • Databricks no longer creates a serverless starter SQL warehouse
  • In SQL Warehouses API, enabling serverless compute now must be explicit
  • Changes for workspace settings for serverless SQL warehouses
  • Changes for serverless compute settings for accounts and workspaces
  • Databricks SQL Serverless is GA
  • .ipynb (Jupyter) notebook support in Repos (preview)
  • Support for reload4j
  • Execute SQL cells in the notebook in parallel
  • Create job tasks using Python code stored in a Git repo
  • Databricks Terraform provider updated to version 1.13.0
  • Databricks Terraform provider updated to version 1.12.0
  • SQL admin console and workspace admin console combined
  • Model Serving is GA
  • Automatic feature lookup is GA
  • New Catalog Explorer availability
  • View frequent queries and users of a table using the Insights tab
  • Exact match search is available in global search
  • View lineage information for your Databricks jobs
  • Databricks Runtime 12.2 LTS and Databricks Runtime 12.2 LTS ML are GA
  • Workspace files are now in Public Preview
• February 2023
  • SAML single sign-on (SSO) in the account console is generally available
  • Ray on Databricks (Public Preview)
  • Notebook cell output results limit increased
  • Databricks Jobs now supports running continuous jobs
  • Trigger your Databricks job when new files arrive
  • Databricks Terraform provider updated to version 1.10.0
  • Legacy global init scripts and cluster-named init scripts disabled
  • Improvements to the MLflow experiment UI
  • Databricks Runtime 12.2 (Beta)
  • Databricks extension for Visual Studio Code (Public Preview)
  • Serverless Real-Time Inference Public Preview now available to all customers
  • Databricks Terraform provider updated to version 1.9.2
  • Variable explorer in Databricks notebooks
• January 2023
  • Authenticate to Power BI and Tableau using OAuth
  • Audit logs include entries for OAuth SSO authentication to the account console (Public Preview)
  • Account SCIM is now GA
  • Easier creation and editing of Databricks jobs in the UI
  • Improvements to the Databricks Jobs UI when viewing job runs
  • REST API Reference is now available for browsing API documentation
  • New account console home screen provides better account management experience
  • Databricks Terraform provider updated to version 1.9.1
  • Account users can update email preferences in the account console
  • Region support consolidated onto one page
  • Databricks Runtime 12.1 and Databricks Runtime 12.1 ML are GA
  • Cluster policies now support limiting the max number of clusters per user
  • Databricks Terraform provider updated to version 1.9.0
  • Partner Connect supports connecting to Privacera
  • Databricks Terraform provider updated to version 1.8.0
  • Databricks Runtime 12.1 (Beta)
  • Partner Connect supports Sigma
  • New left and right sidebars in Databricks notebooks
• December 2022
  • Databricks SQL Driver for Go is Generally Available
  • Prevent concurrent workspace updates
  • Databricks Terraform provider updated to version 1.7.0
  • Databricks Runtime 12.0 and 12.0 ML are GA
  • Jobs are now available in global search
  • Billable usage graphs can now aggregate by individual tags
  • Use SQL to specify schema- and catalog-level storage locations for Unity Catalog managed tables
  • Capturing lineage data with Unity Catalog is now generally available
  • Databricks ODBC driver 2.6.29
  • Databricks JDBC driver 2.6.32
  • Partner Connect supports connecting to AtScale
  • Improved serverless SQL warehouse support for customer-managed keys
• November 2022
  • Enhanced notifications for your Databricks jobs (Public Preview)
  • Databricks Runtime 12.0 (Beta)
  • Upload data UI can now be disabled via admin settings
  • Partner Connect support for Unity Catalog is GA
  • Work with large repositories with Sparse Checkout
  • Databricks Terraform provider updated to version 1.6.5
  • Databricks Terraform provider updated to versions 1.6.3 and 1.6.4
  • Specify a cloud storage location for Unity Catalog managed tables at the catalog and schema levels
  • Access recent objects from the search field in the top bar of your workspace
  • Create or modify table from file upload page now supports multiple files
  • Create or modify table from file upload page now supports overwrite
  • Search for jobs by name with the Jobs API 2.1
  • Databricks Terraform provider updated to version 1.6.2
  • Search for tables in Unity Catalog is GA
• October 2022
  • GA: Repos support for non-notebook files
  • Deploy models for streaming inference with DLT notebooks
  • Connect to Fivetran from the add data UI
  • Databricks SQL Driver for Node.js is Generally Available
  • Partner Connect supports connecting to erwin Data Modeler by Quest
  • Enforce user isolation cluster types on a workspace
  • Databricks Runtime 11.3 LTS and 11.3 LTS ML are GA
  • Format Python code in notebooks (Public Preview)
  • IP access lists no longer block PrivateLink traffic
  • AWS PrivateLink support is now generally available
  • Improvements to AWS PrivateLink support for updating workspaces
  • Update a failed workspace with Databricks-managed VPC to use a customer-managed VPC
  • Personal Compute cluster policy is available by default to all users
  • Add data UI provides a central UI for loading data to Databricks
  • Create or modify table from file upload page unifies experience for small file upload to Delta Lake
  • Partner Connect supports connecting to Hevo Data
  • Enable admin protection for No Isolation Shared clusters
  • SQL persona integrated with new search experience
  • Databricks is a FedRAMP® Authorized Cloud Service Offering (CSO) at the moderate impact Level
  • Serverless SQL warehouses are available in regions eu-central-1 and us-east-2
  • Privilege inheritance in now supported in Unity Catalog
  • Top navigation bar in the UI
  • Databricks Runtime 11.3 (Beta)
  • The account console is available in multiple languages
• September 2022
  • New reference solution for natural language processing
  • More regions for Unity Catalog
  • Protect and control access to some types of encrypted data with customer-managed keys (GA)
  • Compliance security profile workspaces support i4i instance types
  • Audit logs now include events for web terminal
  • Audit logs now include events for managing credentials for Git repos
  • Select cluster policies directly in the DLT UI
  • New data trasformation card on workspace landing pages
  • Orchestrate Databricks SQL tasks in your Databricks workflows (Public Preview)
  • Delta cache is now disk cache
  • Capture and view lineage data with Unity Catalog (Public Preview)
  • Search for tables using Catalog Explorer (Public Preview)
  • View and organize assets in the workspace browser across personas
  • Databricks Runtime 11.2 and 11.2 ML are GA
  • Support for AWS Graviton instances is GA
• August 2022
  • Account users can access the account console
  • Databricks ODBC driver 2.6.26
  • Databricks JDBC driver 2.6.29
  • Databricks Feature Store client now available on PyPI
  • Unity Catalog is GA
  • Delta Sharing is GA
  • Databricks Runtime 11.2 (Beta)
  • Reduced message volume in the DLT UI for continuous pipelines
  • Easier cluster configuration for your DLT pipelines
  • Orchestrate dbt tasks in your Databricks workflows (Public Preview)
  • Users can be members of multiple Databricks accounts
  • Identity federation is GA
  • Partner Connect supports connecting to Stardog
  • Databricks Feature Store integration with Serverless Real-Time Inference
  • Additional data type support for Databricks Feature Store automatic feature lookup
  • Bring your own key: Encrypt Git credentials
  • Cluster UI preview and access mode replaces security mode
  • Unity Catalog limitations (Public Preview)
  • Serverless Real-Time Inference in Public Preview
  • Serverless SQL warehouses improvements
  • Share VPC endpoints among Databricks accounts
  • AWS PrivateLink private access level ANY is deprecated
  • Improvements to AWS PrivateLink connectivity
  • Improved workspace search is now GA
  • Use generated columns when you create DLT datasets
  • Improved editing for notebooks with Monaco-based editor (Experimental)
  • Compliance controls FedRAMP Moderate, PCI-DSS, and HIPAA (GA)
  • Add security controls with the compliance security profile (GA)
  • Add image hardening and monitoring agents with enhanced security monitoring (GA)
  • Databricks Runtime 10.3 series support ends
  • DLT now supports refreshing only selected tables in pipeline updates
  • Job execution now waits for cluster libraries to finish installing
• July 2022
  • Databricks Runtime 11.1 and 11.1 ML are GA
  • Photon is GA
  • Notification upon notebook completion
  • Increased limit for the number of jobs in your Databricks workspaces
  • Verbose audit logs now record when Databricks SQL queries are run
  • Databricks SQL Serverless supports instance profiles whose name does not match its associated role
  • Configure your workspace to use IMDS v2 (Public Preview)
  • Databricks JDBC driver 2.6.27
  • Databricks ODBC driver 2.6.25
  • Databricks Runtime 11.1 (Beta)
  • Improved notebook visualizations
• June 2022
  • ALTER TABLE permission changes for Unity Catalog
  • Databricks Runtime 6.4 Extended Support reaches end of support
  • Databricks Runtime 10.2 series support ends
  • Databricks ODBC driver 2.6.24
  • Databricks Terraform provider is now GA
  • Serverless SQL warehouses available for E2 workspaces (Public Preview)
  • Enable enhanced security controls with a security profile (Public Preview)
  • PCI-DSS compliance controls (Public Preview)
  • HIPAA compliance controls for E2 (Public Preview)
  • Enhanced security monitoring (Public Preview)
  • Databricks Runtime 11.0 and 11.0 ML are GA; 11.0 Photon is Public Preview
  • Change to Repos default working directory in Databricks Runtime 11.0
  • Databricks Runtime 10.1 series support ends
  • Audit logs can now record when a notebook command is run
  • DLT now supports SCD type 2
  • Create DLT pipelines directly in the Databricks UI
  • Select the DLT channel when you create or edit a pipeline
  • Communicate between tasks in your Databricks jobs with task values
  • Enable account switching in the Databricks UI
  • Updating the AWS Region for a failed workspace is no longer supported
• May 2022
  • Copy and paste notebook cells between tabs and windows
  • Additional data type support for Databricks Feature Store automatic feature lookup
  • Databricks Runtime 11.0 (Beta)
  • Improved workspace search (Public Preview)
  • Explore SQL cell results in Python notebooks natively using Python
  • Databricks Repos: Support for more files in a repo
  • Databricks Repos: Fix to issue with MLflow experiment data loss
  • Upgrade wizard makes it easier to copy databases and multiple tables to Unity Catalog (Public Preview)
  • Power BI Desktop system-wide HTTP proxy support
  • Streamline billing and account management by signing up for Databricks using AWS Marketplace
  • Databricks Runtime 10.5 and 10.5 ML are GA; 10.5 Photon is Public Preview
  • Authenticate to the account console using SAML 2.0 (Public Preview)
  • Databricks JDBC driver 2.6.25
  • See the user a pipeline runs as in the DLT UI
• April 2022
  • Use tags to better manage your Databricks jobs
  • Databricks Runtime 10.0 series support ends
  • Get a visual overview of your job runs with the new jobs matrix view
  • Save time and resources when your Databricks job runs are unsuccessful
  • View the run history for job tasks
  • Assign a new cluster in the jobs UI when the Single User access no longer exists
  • Databricks Runtime 10.5 (Beta)
  • Feature Store now supports publishing features to AWS DynamoDB
  • The DLT UI is enhanced to disable unauthorized actions
  • AutoML is generally available
  • Use datasets from Unity Catalog with AutoML
  • DLT is GA on AWS and Azure, and in Public Preview on GCP
  • DLT SQL interface: non-breaking change to table names
• March 2022
  • Better performance and cost for your DLT pipelines with Databricks enhanced autoscaling
  • Files in Repos enabled by default in new workspaces
  • Databricks Feature Store is generally available
  • Share an experiment from the experiment page
  • RStudio Workbench bug fix
  • New workspace language options
  • Databricks Runtime 10.4 LTS and 10.4 LTS ML are GA; 10.4 Photon is Public Preview
  • Unity Catalog is available in Public Preview
  • Delta Sharing is available in Public Preview
  • Enhanced access control for DLT pipelines
  • Test DLT preview functionality with the new channel setting (Public Preview)
  • Improved error handling for DLT Python functions (Public Preview)
  • Improvements to Databricks Repos
  • Audit logging for cluster policy changes
  • Databricks Runtime 10.4 (Beta)
• February 2022
  • Easier scheduling for your DLT pipelines (Public Preview)
  • Easily browse the history of your DLT pipeline updates (Public Preview)
  • Ensure job idempotency for the Jobs API Run now request
  • Jobs service stability and scalability improvements
  • Compare MLflow runs from different experiments
  • Improvements to MLflow compare runs display
  • Improved visibility into job run owners in the clusters UI
  • Drop dataset columns in AutoML
  • Experiments page is GA
  • Support for temporary tables in the DLT Python interface
  • User interface improvements for DLT (Public Preview)
  • Databricks Runtime 9.0 series support ends
  • Data Science & Engineering landing page updates
  • Databricks Repos now supports AWS CodeCommit for Git integration
  • Improved visualization for your DLT pipelines (Public Preview)
  • Updated Markdown parser
  • DLT now supports change data capture processing (Public Preview)
  • Select algorithm frameworks to use with AutoML
  • Customer-managed VPCs are now available in ap-northeast-2
  • Databricks hosted MLflow models can now look up features from online stores
  • Databricks Runtime 10.3 and 10.3 ML are GA; 10.3 Photon is Public Preview
• January 2022
  • MLflow Model Registry Webhooks on Databricks (Public Preview)
  • Breaking change: cluster idempotency token cleared on cluster termination
  • Databricks Runtime 10.3 (Beta)
  • View information on recent job runs
  • Use Markdown in Databricks Repos file editor
  • Improved cluster management for jobs that orchestrate multiple tasks
  • Add or rotate the customer-managed key for managed services on a running workspace
  • Delta Sharing Private Preview adds functionality and new terms
  • Address AWS GuardDuty alerts related to Databricks access to your S3 bucket
  • Databricks Runtime 8.3 and Databricks Runtime 8.4 series support ends
  • Databricks JDBC driver 2.6.22
  • Support for G5 family of GPU-accelerated EC2 instances (Public Preview)
  • New Share button replaces Permissions icon in notebooks
  • New workspace language options
• December 2021
  • Databricks Runtime 6.4 Extended Support series end-of-support date extended
  • Databricks Runtime 5.5 Extended Support series reaches end of support
  • Databricks JDBC driver 2.6.21
  • Databricks Connector for Tableau 2021.4
  • Databricks Runtime 10.2 and 10.2 ML are GA; 10.2 Photon is Public Preview
  • Workspaces in the ap-southeast-1 region now support AWS PrivateLink
  • Updated Markdown parser
  • User interface improvements for DLT
  • Databricks Runtime 8.3 series support extended
  • Databricks Runtime 10.2 (Beta)
  • Revert of recent breaking change that removed escaping and quotes from $ in environment variable values for cluster creation
  • serverless SQL warehouses are available in region eu-west-1
• November 2021
  • Create tags for feature tables (Public Preview)
  • Syntax highlighting and autocomplete for SQL commands in Python cells
  • Rename, delete, and change permissions for MLflow experiments from experiment page (Public Preview)
  • New data profiles in notebooks: tabular and graphic summaries of your data (Public Preview)
  • Improved logging when schemas evolve while running a DLT pipeline
  • Databricks Partner Connect GA
  • Breaking change: remove escaping and quotes from $ in environment variable values for cluster creation
  • Ease of use improvements for Files in Repos
  • Support for legacy SQL widgets ends on January 15, 2022
  • User interface improvements for Databricks jobs
  • Delta Sharing Connector for Power BI
  • Databricks ODBC driver 2.6.19
  • Databricks Runtime 10.1 and 10.1 ML are GA; 10.1 Photon is Public Preview
  • Databricks Runtime 10.1 (Beta)
  • Rename and delete MLflow experiments (Public Preview)
  • Photon support for additional cluster instance families
  • You can now create a cluster policy by cloning an existing policy
  • Single sign-on (SSO) in the account console is Generally Available
  • Change the default language of notebooks and notebook cells more easily
  • Use Files in Repos from the web terminal
• October 2021
  • Databricks Runtime 8.2 series support ends
  • Databricks Runtime 10.0 and 10.0 ML are GA; 10.0 Photon is Public Preview
  • Limit the set of VPC endpoints your workspace can use for AWS PrivateLink connections (Public Preview)
  • User interface improvements for DLT (Public Preview)
  • Specify a fixed-size cluster when you create a new pipeline in DLT (Public Preview)
  • View data quality metrics for tables in DLT triggered pipelines (Public Preview)
  • Jobs orchestration is now GA
  • Databricks Connector for Power BI
  • Repos now supports arbitrary file types
  • More detailed job run output with the Jobs API
  • Improved readability of notebook paths in the Jobs UI
  • Open your DLT pipeline in a new tab or window
  • New escape sequence for $ in legacy input widgets in SQL
  • Faster model deployment with automatically generated batch inference notebook
• September 2021
  • Databricks Runtime 10.0 (Beta)
  • Databricks ODBC driver 2.6.18
  • Customer control of workspace login by Databricks staff
  • Databricks Runtime 9.1 LTS and 9.1 LTS ML are GA; 9.1 LTS Photon is Public Preview
  • Databricks Runtime 8.1 series support ends
  • Databricks JDBC driver 2.6.19
  • Share feature tables across workspaces
  • Security and usability improvements when resetting passwords
  • Repos now supports .gitignore
  • Enhanced jobs UI is now standard for all workspaces
  • Databricks now available in region ap-southeast-1
  • PrivateLink supported in all availability zones within the supported regions
  • Databricks Runtime 9.1 (Beta)
  • Streamlined management of settings for Databricks jobs
  • Databricks SQL Public Preview available in all workspaces
  • Delete feature tables from Feature Store
  • Grant view pipeline permissions in the DLT UI (Public Preview)
  • Reduce cluster resource usage with DLT (Public Preview)
  • Development optimizations for DLT (Public Preview)
  • Use MLflow models in your DLT pipelines (Public Preview)
  • Find DLT pipelines by name (Public Preview)
  • PyTorch TorchScript and other third-party libraries are now supported in Databricks jobs
  • Databricks Runtime 8.0 series support ends
• August 2021
  • Databricks Repos GA
  • Serverless SQL provides instant compute, minimal management, and cost optimization for SQL queries (Public Preview)
  • User interface improvements for DLT (Public Preview)
  • More control over how tables are materialized in DLT pipelines (Public Preview)
  • Increased timeout for long-running notebook jobs
  • Jobs service stability and scalability improvements
  • User entitlements granted by group membership are displayed in the admin console
  • Manage MLflow experiment permissions (Public Preview)
  • Improved job creation from notebooks
  • Improved support for collapsing notebook headings
  • Databricks Runtime 9.0 and 9.0 ML are GA; 9.0 Photon is Public Preview
  • Low-latency delivery of audit logs is generally available
  • Databricks Runtime 9.0 (Beta)
  • Manage repos programmatically with the Databricks CLI (Public Preview)
  • Manage repos programmatically with the Databricks REST API (Public Preview)
  • Databricks Runtime 7.6 series support ends
  • Log delivery APIs now report delivery status
  • Use the AWS EBS SSD gp3 volume type for all clusters in a workspace
  • Audit events are logged when you interact with Databricks Repos
  • Improved job creation and management workflow
  • Simplified instructions for setting Git credentials (Public Preview)
  • Import multiple notebooks in .html format
  • Usability improvements for DLT
  • Configure Databricks for SSO with Microsoft Entra ID in your Azure tenant
• July 2021
  • Manage MLflow experiment permissions with the Databricks REST API
  • Databricks web interface is localized in Portuguese and French (Public Preview)
  • Databricks Runtime 5.5 LTS for Machine Learning support ends, replaced by Extended Support version
  • Databricks Light 2.4 support ends September 5, replaced by Extended Support version
  • Reduced permissions for cross-account IAM roles
  • Feature freshness information available in Databricks Feature Store UI (Public Preview)
  • Display up to 10,000 result rows
  • Bulk import and export notebooks in a folder as source files
  • Autocomplete in SQL notebooks now uses all-caps for SQL keywords
  • Reorderable and resizable widgets in notebooks
  • Databricks UI usability fixes
  • Quickly define pipeline settings when you create a new DLT pipeline
  • Databricks Runtime 8.4 and 8.4 ML are GA; 8.4 Photon is Public Preview
  • Use Spark SQL with the DLT Python API
  • Enhanced data processing and analysis with Databricks jobs (Public Preview)
  • Reduced cost for DLT default clusters (Public Preview)
  • Sort pipelines by name in the DLT UI (Public Preview)
  • Changes to Compute page
  • Databricks Runtime 5.5 LTS support ends, replaced by Databricks Runtime 5.5 Extended Support through the end of 2021
  • Repos API (Public Preview)
  • Databricks Runtime 8.4 (Beta)
• June 2021
  • Correction: Repos for Git is enabled by default in new and existing workspaces in some regions
  • Change to Feature Store permissions
  • Improved access to results in the MLflow runs table
  • Better cost visibility for DLT
  • Enhanced data quality constraints for DLT
  • API changes for updating and replacing IP address lists
  • Databricks ODBC driver 2.6.17
  • Use an API to download usage data directly
  • Databricks Runtime 7.5 series support ends
  • Optimize performance and control costs by using different pools for the driver node and worker nodes
  • Photon runtimes now support i3.xlarge instances (Public Preview)
  • Registry-wide permissions for Model Registry
  • A user’s home directory is no longer protected when you delete a user using the SCIM API
  • Accelerate SQL workloads with Photon (Public Preview)
  • Databricks Runtime 8.3 and 8.3 ML are GA; 8.3 Photon is Public Preview
  • Python and SQL table access control (GA)
  • Jobs UI and API now show the owner of a job run
  • Protect sensitive Spark configuration properties and environment variables using secrets (Public Preview)
  • Repos for Git is enabled by default in new and existing workspaces in some regions
  • Redesigned Workspace Settings UI
  • Updates to ListTokens and ListAllTokens database queries expired tokens
  • Confirmation now required when granting or revoking Admin permissions
  • Changes to keyboard shortcuts in the web UI
• May 2021
  • Databricks Mosaic AI: a data-native and collaborative solution for the full ML lifecycle
  • SQL Analytics is renamed to Databricks SQL
  • Create and manage ETL pipelines using DLT (Public Preview)
  • Reduced scope of required egress rules for customer-managed VPCs
  • Workspaces in the eu-west-2 region now support AWS PrivateLink
  • Encrypt Databricks SQL queries and query history using your own key (Public Preview)
  • Increased limit for the number of terminated all-purpose clusters
  • Increased limit for the number of pinned clusters
  • Manage where notebook results are stored (Public Preview)
  • The new improved account console is GA
  • Customer-managed keys for workspace storage (Public Preview)
  • Changes to the Account API for customer-managed keys
  • Google Cloud Storage connector (GA)
  • Databricks Runtime 7.4 series support ends
  • Better governance with enhanced audit logging
  • Use SSO to authenticate to the account console (Public Preview)
  • Repos users can now integrate with Azure DevOps using personal access tokens
  • Jobs service stability and scalability improvements (Public Preview)
  • Service principals provide API-only access to Databricks resources (Public Preview)
• April 2021
  • Databricks Runtime 8.2 (GA)
  • AWS PrivateLink for Databricks workspaces (Public Preview)
  • Update running workspaces with new credentials or network configurations
  • Databricks can now send in-product messages and product tours directly to your workspace (Public Preview)
  • Easier job management with the enhanced jobs user interface
  • Cluster policy changes are applied automatically to existing clusters at restart and edit
  • Track retries in your job tasks when task attempts fail
  • Quickly view cluster details when you create a new cluster
  • MLflow sidebar reflects the most recent experiment
  • Change to default channel for conda.yaml files in MLflow
  • New free trial and pay-as-you-go customers are now on the E2 version of the platform
  • Databricks Runtime 8.2 (Beta)
  • User and group limits
  • Easier monitoring of job run status
  • Better governance with enhanced audit logging
  • Global init scripts no longer run on model serving clusters
  • Databricks Runtime 6.4 series support ends
• March 2021
  • Databricks now supports dark mode for viewing notebooks
  • Databricks Runtime 8.1 (GA)
  • Easier job creation and management with the enhanced jobs user interface (Public Preview)
  • Track job retry attempts with a new sequential value returned for each job run attempt
  • Increased limit for the number of saved jobs in Premium and Enterprise workspaces
  • Easier way to connect to Databricks from your favorite BI tools and SQL clients
  • Databricks Repos let you use Git repositories to integrate Databricks with CI/CD systems
  • Automatic retries for failed job clusters reverted
  • Databricks Runtime 8.1 (Beta)
  • Limit username and password authentication with password ACLs (GA)
  • Receive email notification about activity in Model Registry
  • Model Serving now supports additional model types
  • New options for searching Model Registry
  • Increased limit for the number of terminated all-purpose clusters
  • Increased limit for the number of pinned clusters in a workspace
  • Databricks Runtime 8.0 (GA)
• February 2021
  • New Databricks Power BI connector (GA)
  • Add and manage account admins using the SCIM API (E2 accounts, Public Preview)
  • Added modification_time to the DBFS REST API get-status and list responses
  • Easily copy long experiment names in MLflow
  • Adjust memory size and number of cores for serving clusters
  • Web terminal is now GA
  • Auto-AZ: automatic selection of availability zone (AZ) when you launch clusters, available on all deployment types
  • Delegate account management beyond the account owner (E2 accounts, Public Preview)
  • Separate account-level and workspace-level audit logging configurations help you monitor account and workspace activity more effectively
  • Databricks Runtime 7.2 series support ends
  • Databricks Runtime 7.6 GA
  • Databricks Runtime 8.0 (Beta)
  • Databricks Runtime for Genomics now deprecated
  • View more readable JSON in the MLflow run artifact display
  • Provide comments in the Model Registry using REST API
  • Easily specify default cluster values in API calls
  • Tune cluster worker configuration according to current worker allocation
  • Pass context specific information to a job’s task with task parameter variables
  • Error messages from job failures no longer contain possibly sensitive information
  • Download usage data from the new account console for E2 accounts
• January 2021
  • Databricks Runtime 7.1 series support ends
  • Start clusters faster with Docker images preloaded into instance pools
  • Notebook find and replace now supports changing all occurrences of a match
  • Single Node clusters (GA)
  • Free form cluster policy type renamed to Unrestricted
  • Cluster policy field not shown if a user only has access to one policy
  • G4 family of GPU-accelerated EC2 instances GA
  • Databricks Runtime 7.0 series support ends
  • Billable usage and audit log S3 bucket policy and object ACL changes
  • E2 platform comes to the Asia Pacific region
• December 2020
  • Databricks Runtime 7.5 GA
  • Existing Databricks accounts migrate to E2 platform today
  • Jobs API now supports updating existing jobs
  • New global init script framework is GA
  • New account console enables customers on the E2 platform to create and manage multiple workspaces (Public Preview)
  • Databricks Runtime 7.5 (Beta)
  • Auto-AZ: automatic selection of availability zone (AZ) when you launch clusters
  • Jobs API end_time field now uses epoch time
  • Find DBFS files using new visual browser
  • Visibility controls for jobs, clusters, notebooks, and other workspace objects are now enabled by default on new workspaces
  • Improved display of nested runs in MLflow
  • Admins can now lock user accounts (Public Preview)
  • Updated NVIDIA driver
  • Use your own keys to secure notebooks (Public Preview)
• November 2020
  • Databricks Runtime 6.6 series support ends
  • MLflow Model Registry GA
  • Filter experiment runs based on whether a registered model is associated
  • Partner integrations gallery now available through the Data tab
  • Cluster policies now use allowlist and blocklist as policy type names
  • Automatic retries when the creation of a job cluster fails
  • Navigate notebooks using the table of contents
  • Databricks SQL (Public Preview)
  • Web terminal available on Databricks Community Edition
  • Single Node clusters now support Databricks Container Services
  • Databricks Runtime 7.4 GA
  • Databricks JDBC driver update
  • Databricks Connect 7.3 (Beta)
• October 2020
  • New Databricks Power BI connector available in the online Power BI service (Public Preview)
  • Databricks Runtime 7.4 (Beta)
  • Expanded experiment access control (ACLs)
  • High fidelity import and export of Jupyter notebook (ipynb) files
  • SCIM API improvement: both indirect and direct groups returned in user record response
  • Databricks Runtime 6.5 series support ends
  • Self-service, low-latency audit log configuration (Public Preview)
  • SCIM API improvement: $ref field response
  • Databricks Runtime 7.3, 7.3 ML, and 7.3 Genomics declared Long Term Support (LTS)
  • Render images at higher resolution using matplotlib
• September 2020
  • Databricks Runtime 7.3, 7.3 ML, and 7.3 Genomics are now GA
  • Debugging hints for SAML credential passthrough misconfigurations
  • Single Node clusters (Public Preview)
  • DBFS REST API rate limiting
  • New sidebar icons
  • Running jobs limit increase
  • Artifact access control lists (ACLs) in MLflow
  • MLflow usability improvements
  • New Databricks Power BI connector (Public Preview)
  • New JDBC and ODBC drivers bring faster and lower latency BI
  • MLflow Model Serving (Public Preview)
  • Clusters UI improvements
  • Visibility controls for jobs, clusters, notebooks, and other workspace objects
  • Ability to create tokens no longer permitted by default
  • Support for c5.24xlarge instances
  • MLflow Model Registry supports sharing of models across workspaces
  • Databricks Runtime 7.3 (Beta)
  • E2 architecture—now GA—provides better security, scalability, and management tools
  • Account API is generally available on the E2 version of the platform
  • Secure cluster connectivity (no public IPs) is now the default on the E2 version of the platform
• August 2020
  • Token Management API is GA and admins can use the Admin Console to grant and revoke user access to tokens
  • Message size limits for Shiny apps increased
  • Improved instructions for setting up a cluster in local mode
  • View version of notebook associated with a run
  • Databricks Runtime 7.2 GA
  • Databricks Runtime 7.2 ML GA
  • Databricks Runtime 7.2 Genomics GA
  • Permissions API (Public Preview)
  • Databricks Connect 7.1 (GA)
  • Repeatable installation order for cluster libraries
  • Customer-managed VPC is GA
  • Secure cluster connectivity (no public IPs) is GA
  • Multi-workspace API (Account API) adds pricing tier
  • Create model from MLflow registered models page (Public Preview)
  • Databricks Container Services supports GPU images
• July 2020
  • Web terminal (Public Preview)
  • New, more secure global init script framework (Public Preview)
  • IP access lists now GA
  • New file upload dialog
  • SCIM API filter and sort improvements
  • Databricks Runtime 7.1 GA
  • Databricks Runtime 7.1 ML GA
  • Databricks Runtime 7.1 Genomics GA
  • Databricks Connect 7.1 (Public Preview)
  • IP Access List API updates
  • Python notebooks now support multiple outputs per cell
  • View notebook code and results cells side by side
  • Pause job schedules
  • Jobs API endpoints validate run ID
  • Format SQL in notebooks automatically
  • Support for r5.8xlarge and r5.16xlarge instances
  • Use password access control to configure which users are required to log in using SSO or authenticate using tokens (Public Preview)
  • Reproducible order of installation for Maven and CRAN libraries
  • Take control of your users’ personal access tokens with the Token Management API (Public Preview)
  • Customer-managed VPC deployments (Public Preview) can now use regional VPC endpoints
  • Encrypt traffic between cluster worker nodes (Public Preview)
  • Table access control supported on all accounts with the Premium plan (Public Preview)
  • IAM credential passthrough supported on all accounts with the Premium plan (Public Preview)
  • Restore cut notebook cells
  • Assign jobs CAN MANAGE permission to non-admin users
  • Non-admin Databricks users can view and filter by username using the SCIM API
  • Link to view cluster specification when you view job run details
• June 2020
  • Billable usage logs delivered to your own S3 bucket (Public Preview)
  • Databricks Connect now supports Databricks Runtime 6.6
  • Databricks Runtime 7.0 ML GA
  • Databricks Runtime 7.0 GA, powered by Apache Spark 3.0
  • Databricks Runtime 7.0 for Genomics GA
  • Stage-dependent access controls for MLflow models
  • Notebooks now support disabling auto-scroll
  • Skipping instance profile validation now available in the UI
  • Account ID is displayed in account console
  • Internet Explorer 11 support ends on August 15
  • Databricks Runtime 6.2 series support ends
  • Simplify and control cluster creation using cluster policies (Public Preview)
  • SCIM Me endpoint now returns SCIM compliant response
  • G4 family of GPU-accelerated EC2 instances now available for machine learning application deployments (Beta)
  • Deploy multiple workspaces in your Databricks account (Public Preview)
  • Deploy Databricks workspaces in your own VPC (Public Preview)
  • Secure cluster connectivity with no open ports on your VPCs and no public IP addresses on Databricks workers (Public Preview)
  • Restrict access to Databricks using IP access lists (Public Preview)
  • Encrypt locally attached disks (Public Preview)
• May 2020
  • Databricks Runtime 6.6 for Genomics GA
  • Databricks Runtime 6.6 ML GA
  • Databricks Runtime 6.6 GA
  • Easily view large numbers of MLflow registered models
  • Libraries configured to be installed on all clusters are not installed on clusters running Databricks Runtime 7.0 and above
  • Databricks Runtime 7.0 for Genomics (Beta)
  • Databricks Runtime 7.0 ML (Beta)
  • Databricks Runtime 6.6 for Genomics (Beta)
  • Databricks Runtime 6.6 ML (Beta)
  • Databricks Runtime 6.6 (Beta)
  • Job clusters now tagged with job name and ID
  • DBFS REST API delete endpoint size limit
  • Restore deleted notebook cells
  • Jobs pending queue limit
• April 2020
  • MLflow tracking UI enhancement
  • Notebook usability improvements
  • Databricks Connect now supports Databricks Runtime 6.5
  • Databricks Runtime 6.1 and 6.1 ML support ends
  • Databricks Runtime 6.5 GA
  • Databricks Runtime 6.5 for Machine Learning GA
  • Databricks Runtime 6.5 for Genomics GA
  • Authenticate to S3 buckets automatically using your IAM credentials (Public Preview)
  • IAM role renamed to instance profile
  • Easier notebook title changes
  • Cluster termination reporting enhancement
  • DBFS REST API delete endpoint size limit
  • Databricks Runtime 6.0 and 6.0 ML support ends
• March 2020
  • Managed MLflow Model Registry collaborative hub available (Public Preview)
  • Load data from hundreds of data sources into Delta Lake using Stitch
  • Databricks Runtime 7.0 (Beta) previews Apache Spark 3.0
  • Databricks Runtime 6.5 ML (Beta)
  • Databricks Runtime 6.5 (Beta)
  • Optimized autoscaling on all-purpose clusters running Databricks Runtime 6.4 and above
  • Single-sign-on (SSO) now available on all pricing plans
  • Develop and test Shiny applications inside RStudio Server
  • Change the default language of a notebook
  • Databricks to add anonymized usage analytics
  • Databricks Connect now supports Databricks Runtime 6.4
  • Databricks Connect now supports Databricks Runtime 6.3
• February 2020
  • Databricks Runtime 6.4 for Genomics GA
  • Databricks Runtime 6.4 ML GA
  • Databricks Runtime 6.4 GA
  • The Clusters and Jobs UIs now reflect new cluster terminology and cluster pricing
  • New interactive charts offer rich client-side interactions
  • New data ingestion network adds partner integrations with Delta Lake (Public Preview)
  • Flags to manage workspace security and notebook features now available
• January 2020
  • All cluster and pool tags now propagate to usage reports
  • Cluster and pool tag propagation to EC2 instances is more accurate
  • Databricks Runtime 6.3 for Genomics GA
  • Databricks Runtime 6.3 ML GA
  • Databricks Runtime 6.3 GA
  • Cluster worker machine images now use chrony for NTP
  • Cluster standard autoscaling step is now configurable
  • SCIM API supports pagination for Get Users and Get Groups (Public Preview)
  • File browser swimlane widths increased to 240px
  • Databricks Runtime 3.5 LTS support ends
• December 2019
  • Databricks Connect now supports Databricks Runtime 6.2
  • Databricks Runtime 6.2 for Genomics GA
  • Azure Databricks SCIM provisioning connector available in the app gallery
  • Databricks Runtime 5.3 and 5.4 support ends
  • Databricks Runtime 6.2 ML GA
  • Databricks Runtime 6.2 GA
  • Databricks Connect now supports Databricks Runtime 6.1
• November 2019
  • Databricks Runtime 6.2 ML Beta
  • Databricks Runtime 6.2 Beta
  • Configure clusters with your own container image using Databricks Container Services
  • Cluster detail now shows only cluster ID in the HTTP path
  • Secrets referenced by Spark configuration properties and environment variables (Public Preview)
• October 2019
  • Databricks Runtime 6.1 for Genomics GA
  • Databricks Runtime 6.1 for Machine Learning GA
  • MLflow API calls are now rate limited
  • Pools of instances for quick cluster launch generally available
  • New instance types (Beta)
  • Databricks Runtime 6.1 GA
  • Databricks Runtime 6.0 for Genomics GA
  • Non-admin Databricks users can read user and group names and IDs using SCIM API
  • Workspace API returns notebook and folder object IDs
  • Databricks Runtime 6.0 ML GA
  • Databricks Runtime 6.0 GA
  • Account usage reports now show usage by user name
• September 2019
  • Databricks Runtime 5.2 support ends
  • Launch pool-backed automated clusters that use Databricks Light (Public Preview)
  • Beta support for m5a and r5a instances
  • pandas DataFrames now render in notebooks without scaling
  • Python version selector display now dynamic
  • Databricks Runtime 6.0 Beta
• August 2019
  • Workspace library installation enhancement
  • Clusters UI now reflects more consistent interactive and automated cluster terminology
  • Databricks Runtime 5.5 and Databricks Runtime 5.5 ML are LTS
  • Instance allocation notifications for pools
  • New cluster events
  • MLflow updates
• July 2019
  • Coming soon: Databricks 6.0 will not support Python 2
  • Ideas Portal
  • Preload the Databricks Runtime version on pool idle instances
  • Custom cluster tags and pool tags play better together
  • MLflow 1.1 brings several UI and API improvements
  • pandas DataFrame display renders like it does in Jupyter
  • New regions
  • Databricks Runtime 5.5 with Conda (Beta)
  • Set permissions on pools (Public Preview)
  • Databricks Runtime 5.5 for Machine Learning
  • Databricks Runtime 5.5
  • Keep a pool of instances on standby for quick cluster launch (Public Preview)
  • Global series color
• June 2019
  • Account usage chart updated to display usage grouped by workload type
  • RStudio integration no longer limited to high concurrency clusters
  • MLflow 1.0
  • Databricks Runtime 5.4 with Conda (Beta)
  • Databricks Runtime 5.4 for Machine Learning
  • Databricks Runtime 5.4
• May 2019
  • Cluster event log filtering
  • JDBC/ODBC connectivity available without Premium plan or above
• April 2019
  • MLflow on Databricks (GA)
  • Delta Lake on Databricks
  • MLflow runs sidebar
  • C5d series Amazon EC2 instance types (Beta)
  • Databricks Runtime 5.3 (GA)
  • Databricks Runtime 5.3 ML (GA)
• March 2019
  • Purge deleted MLflow experiments and runs
  • Databricks Light generally available
  • Searchable cluster selector
  • Upcoming usage display changes
  • Manage groups from the Admin Console
  • Notebooks automatically have associated MLflow experiment
  • Z1d series Amazon EC2 instance types (Beta)
  • Two private IP addresses per node
  • Databricks Delta public community
• February 2019
  • Managed MLflow on Databricks Public Preview
  • Azure Data Lake Storage connector is generally available
  • Python 3 now the default when you create clusters
  • Additional cluster instance types
  • Delta Lake generally available
• January 2019
  • Upcoming change: Python 3 to become the default when you create clusters
  • Databricks Runtime 5.2 for Machine Learning (Beta) release
  • Databricks Runtime 5.2 release
  • Cluster configuration JSON view
  • Library UI
  • Cluster Events
  • Cluster UI
• December 2018
  • Databricks Runtime 5.1 for Machine Learning (Beta) release
  • Databricks Runtime 5.1 release
  • Library UI
• November 2018
  • Library UI
  • Custom Spark heap memory settings enabled
  • Jobs and idle execution context eviction
  • Databricks Runtime 5.0 for Machine Learning (Beta) release
  • Databricks Runtime 5.0 release
  • displayHTML support for unrestricted loading of third-party content
• October 2018
  • SCIM provisioning using OneLogin
  • Copy notebook file path without opening notebook
• September 2018
  • SCIM provisioning using Okta and Microsoft Entra ID (Preview)
  • EBS leaked volumes deletion
  • Support for r5 instances
  • SCIM API for provisioning users and groups (Preview)
• August 2018
  • Workspace sidebar redesign
  • New environment variables in init scripts
  • EBS leaked volumes logging and deletion
  • AWS r3 and c3 instance types now deprecated
  • Audit logging for ACL changes
  • Cluster-scoped init scripts
  • Collapsible headings
• July 2018
  • Libraries API supports Python wheel files
  • IPython notebook export
  • New instance types (beta)
  • Cluster mode and High Concurrency clusters
  • Table access control
  • RStudio integration
  • R Markdown support
  • Home page redesign, with ability to drop files to import data
  • Widget default behavior
  • Table creation UI
  • Multi-line JSON data import
• June 2018
  • Cluster log purge
  • Trash folder
  • Reduced log retention period
  • Gzipped API responses
  • Table import UI
• May 2018
  • General Data Protection Regulation (GDPR)
  • HorovodEstimator
  • MLeap ML Model Export
  • Notebook cells: hide and show
  • Doc site search
  • Databricks Runtime 4.1 for Machine Learning (Beta)
  • New GPU cluster types
  • Secret management
  • Cluster pinning
  • Cluster autostart
  • Workspace purging
  • Databricks CLI 0.7.1
  • Display() support for image data types
  • Databricks Delta update
  • S3 Select connector
• April 2018
  • AWS account updates
  • Spark error tips
  • Databricks CLI 0.7.0
  • Increase init script output truncation limit
  • Clusters API: added UPSIZE_COMPLETED event type
  • Command autocomplete
  • Serverless pools upgraded to Databricks Runtime 4.0
• March 2018
  • Command execution details
  • Databricks CLI 0.6.1 supports --profile
  • ACLs enabled by default for new Operational Security customers
  • New doc site theme
  • Cluster event log
  • Databricks CLI: 0.6.0 release
  • Job run management
  • Edit cluster permissions now requires edit mode
  • Databricks ML Model Export
• February 2018
  • New line chart supports time-series data
  • More visualization improvements
  • Delete job runs using Job API
  • Bring your own S3 bucket
  • KaTeX math rendering library updated
  • Databricks CLI: 0.5.0 release
  • DBUtils API library
  • Filter for your jobs only
  • Spark-submit from the Create Job page
  • Select Python 3 from the Create Cluster page
  • Workspace UI improvements
  • Autocomplete for SQL commands and database names
  • Serverless pools now support R
  • Distributed TensorFlow and Keras Libraries Support
  • XGBoost available as a Spark Package
  • Table access control for SQL and Python (Beta)
• January 2018
  • Mount points for Azure Blob storage containers and Data Lake Stores
  • Table Access Control for SQL and Python (Private Preview)
  • Exporting notebook job run results via API
  • Apache Airflow 1.9.0 includes Databricks integration