AI Functions on Databricks

Preview

This article describes Databricks AI Functions, built-in SQL functions that allow you to apply AI on your data directly from SQL.

SQL is crucial for data analysis due to its versatility, efficiency, and widespread use. Its simplicity enables swift retrieval, manipulation, and management of large datasets. Incorporating AI functions into SQL for data analysis enhances efficiency, which enables businesses to swiftly extract insights.

Integrating AI into analysis workflows provides access to information previously inaccessible to analysts, and empowers them to make more informed decisions, manage risks, and sustain a competitive advantage through data-driven innovation and efficiency.

AI functions using Databricks Foundation Model APIs

Note

For Databricks Runtime 15.0 and above, these functions are supported in notebook environments including Databricks notebooks and workflows.

These functions invoke a state-of-the-art generative AI model from Databricks Foundation Model APIs to perform tasks like, sentiment analysis, classification and translation. See Analyze customer reviews using AI Functions.

ai_query

Note

For Databricks Runtime 14.2 and above, this function is supported in notebook environments including Databricks notebooks and workflows.
For Databricks Runtime 14.1 and below, this function is not supported in notebook environments, including Databricks notebooks.

The ai_query() function allows you to serve your machine learning models and large language models using Databricks Model Serving and query them using SQL. To do so, this function invokes an existing Databricks Model Serving endpoint and parses and returns its response. You can use ai_query() to query endpoints that serve custom models, foundation models made available using Foundation Model APIs, and external models.