Pular para o conteúdo principal

Databricks on AWS GovCloud release notes 2025

The following platform features, improvements, and fixes were released on Databricks on AWS GovCloud in 2025.

May 2025​

The following features and updates were released on Databricks on AWS GovCloud in May 2025.

Billing and audit system tables are now available in AWS GovCloud​

May 9, 2025

The system.billing schema and the system.access.audit table are now supported on AWS GovCloud. System tables provide a a Databricks-hosted analytical store of your account's operational data, accessible in the system catalog. System tables in schemas other than billing and access.audit are not available on AWS GovCloud. System tables are not available in AWS GovCloud DoD. For more information, see Monitor account activity with system tables.

You can now create views in ETL pipelines​

May 8, 2025

The CREATE VIEW SQL command is now available in ETL pipelines. You can create a dynamic view of your data. See CREATE VIEW (DLT).

Configure Python syntax highlighting in Databricks notebooks​

May 8, 2025

You can now configure Python syntax highlighting in notebooks by placing a pyproject.toml file in the notebook's ancestor path or your home folder. Through the pyproject.toml file, you can configure ruff, pylint, pyright, and flake8 linters, as well as disable Databricks-specific rules. This configuration is supported for clusters running Databricks Runtime 16.4 or above, or Client 3.0 or above.

See Configure Python syntax highlighting.

Jobs and pipelines now share a single, unified view (Public Preview)​

May 7, 2025

You can now view all workflows, including jobs, ETL pipelines, and ingestion pipelines, in a single unified list. See View jobs and pipelines.

File events for external locations improve file notifications in Auto Loader and file arrival triggers in jobs (Public Preview)​

May 5, 2025

You can now enable file events on external locations that are defined in Unity Catalog. This makes file arrival triggers in jobs and file notifications in Auto Loader more scalable and efficient.

This feature is in Public Preview. Auto Loader support for file events requires enablement by a Databricks representative. For access, reach out to your Databricks account team.

For details, see the following:

April 2025​

The following features and updates were released on Databricks on AWS GovCloud in April 2025.

Deletion vectors on DLT tables now follow workspace settings​

April 28, 2025

New streaming tables and materialized views will follow the workspace settings for deletion vectors. See Auto-enable deletion vectors and What are deletion vectors?.

Share streaming tables and materialized views using Delta Sharing (Public Preview)​

April 23, 2025

You can now use Delta Sharing to share streaming tables and materialized views.

See Create and manage shares for Delta Sharing and Read shared streaming tables and materialized views.

Strict enforcement of row-level security and column masking policies in Delta Sharing​

April 21, 2025

Delta Sharing now consistently enforces row-level security and column masking policies applied to tables a shared data asset is dependent on, whether those policies were applied before or after the data asset was shared. Recipients may experience differences in query behavior when accessing shared data that depends on tables with row-level security or column masking policies. This ensures that data access aligns with the provider's intended security controls at all times.

See Filter sensitive table data using row filters and column masks.

Run a subset of tasks within a job​

April 21, 2025

You can now run a subset of the tasks when manually triggering a job. See Run a job with different settings.

Python type error highlighting​

April 14, 2025

Python code in notebooks and file editors can highlight type errors for non-existent attributes, missing arguments, and mismatched arguments. See Python error highlighting.

Reference SQL output in downstream tasks of a job​

April 14, 2025

You can now use dynamic values to reference the output of a SQL task in downstream tasks in the same job. For each tasks can iterate over the rows of data in the output.

See What is a dynamic value reference?.

Batch Unity Catalog Python UDFs (Public Preview)​

April 14, 2025

Unity Catalog Batch Python UDFs extend the capabilities of Unity Catalog UDFs by allowing you to write Python code to operate on batches of data, significantly improving efficiency by reducing the overhead associated with row-by-row UDFs. Batch Python UDFs support service credentials to access external cloud services. See Batch Python User-defined functions (UDFs) in Unity Catalog.

Access UDF context information using TaskContext​

April 14, 2025

The TaskContext PySpark API now allows you to retrieve context information—such as user identity and cluster tags—while running Batch Unity Catalog Python UDFs or PySpark UDFs. This feature lets you pass user-specific details, like identity, to authenticate external services within UDFs. See Get task context in a UDF.

The BROWSE privilege is GA​

April 1, 2025

The BROWSE privilege is now generally available. The BROWSE privilege allows you to grant users, service principals, and account groups permission to view a Unity Catalog object's metadata. This enables users to discover data without having read access to the data. A user can view an object's metadata using Catalog Explorer, the schema browser, search results, the lineage graph, information_schema, and the REST API.

See BROWSE.

Última atualização em