Skip to main content

What is Databricks Assistant?

Databricks Assistant is an AI-based pair-programmer and a support agent that makes you more efficient as you create notebooks, queries, dashboards, and files. It can help you rapidly answer questions by generating, optimizing, completing, explaining, and fixing code and queries.

This page provides general information about the Assistant. For questions about privacy and security, see Privacy and security.

How can Databricks Assistant help?​

Databricks Assistant can help with the following tasks:

Assistant uses Unity Catalog metadata to understand your tables, columns, descriptions, and popular data assets across your company to provide personalized responses.

Code generated by the Databricks Assistant is intended to run in a Databricks compute environment. It is optimized to create code in Databricks-supported programming languages, frameworks, and dialects. It is not intended to be a general-purpose programming assistant. The Assistant often uses information from Databricks resources, such as the Databricks Documentation website or Knowledge Base, to better answer user queries. It performs best when the user question is related to questions that can be answered with knowledge from Databricks documentation, Unity Catalog, and user code in the Workspace.

Get coding help from Databricks Assistant​

To access Databricks Assistant, click the In-product help assistant icon - color Assistant icon in the left sidebar of the notebook, the file editor, the SQL Editor, or the dashboard Data tab.

Databricks assistant icon and panel location

The Assistant pane can open on the left or right side of the screen.

Users should always review any code generated by the Assistant before running it because it can sometimes make mistakes.

For more information on how the Assistant can help you with your code, see Get coding help from Databricks Assistant.

Create data visualizations using the Databricks Assistant​

You can use the Databricks Assistant when drafting dashboards. As you create visualizations on an existing dashboard dataset, prompt the Assistant with questions to receive responses in the form of generated charts. To use the Assistant in a dashboard, first create one or more datasets, then add a visualization widget to the Canvas. The visualization widget includes a prompt to describe your new chart. Type a description of the chart you want to see, and the assistant will generate it. You can approve or reject the chart, or modify the description to generate something new.

For details and examples of using the Assistant with dashboards, see Create visualizations with Databricks Assistant.

Services used by Databricks Assistant​

Databricks Assistant might use third-party services to provide responses, including Azure OpenAI operated by Microsoft.

These services are subject to their respective data management policies. Data sent to these services is not used for any model training. For details, see Azure data management policy.

For Azure OpenAI, Databricks has opted out of Abuse Monitoring so no prompts or responses are stored with Azure OpenAI.

Tips for improving the accuracy of results​

  • Use the prompt “Find Tables” for better responses. Before you ask questions about data in a table, ask the Assistant to find related tables by subject matter or other characteristics. Example: Find tables related to NFL games.

  • Specify the structure of the response you want. The structure and detail that Databricks Assistant provides varies, even for the same prompt. Databricks Assistant knows about your table and column schema and metadata, so you can use natural language to ask your question. Example: List active and retired NFL quarterbacks' passing completion rate, for those who had over 500 attempts in a season. Assistant answers using data from columns such as s.player_id and s.attempts.

  • Provide examples of your row-level data values. Databricks Assistant doesn’t have access to row-level data, thus for more accurate answers provide examples of the data. Example: List the average height for each position in inches. This returns an error because the data set shows height in feet and inches, as in 6-2.

  • Test code snippets by running them in the Assistant pane. Use the Assistant pane as a scratchpad that saves iterations of your queries and assistant answers. You can run code and edit it in the pane until you are ready to add it to a notebook.

    Testing code snippets by running them in the Assistant pane.

  • Use cell actions in a notebook. Cell actions include shortcuts to common tasks, such as documenting (commenting), fixing, and explaining code.

    /doc cell action prompts Assistant to comment the code.

For fully illustrated examples, see 5 tips for Databricks Assistant.

Databricks Assistant considers the history of the conversation so you can refine your questions as you go.

What is the pricing for Databricks Assistant?​

All current Databricks Assistant capabilities are available at no additional cost for all customers.

Users pay only for the compute that they use to run their notebooks, queries, jobs, and so on. There are fair usage limits in place to prevent abuse. Most users are not impacted by these limits.

Give feedback​

The best way to send feedback is to use the Provide Feedback links in the notebook and SQL editor. You can also send an email to assistant-feedback@databricks.com or to your account team.

Share product improvement suggestions and user experience issues rather than feedback about prompt accuracy. If you receive an unhelpful suggestion from the Assistant, click the “Not useful” Thumb down icon button.

Geo availability of Assistant features​

Databricks Assistant is a Designated Service that uses Geos to manage data residency when processing customer content. To learn more about the availability of Assistant features in different geographic areas, see Databricks Designated Services.

tip

If you can’t enable Databricks Assistant, you might need to disable Enforce data processing within workspace Geography for AI features. See Enable cross-Geo processing.

Privacy and security​

Q: What data is being sent to the models?​

Databricks Assistant sends code and metadata to the models on each API request. This helps return more relevant results for your data. Examples include:

  • Code/queries in the current notebook cell or SQL Editor tab
  • Table and Column names and descriptions
  • Previous questions
  • Favorite tables

Q: Does the metadata sent to the models respect the user’s Unity Catalog permissions?​

Yes, all of the data sent to the model respects the user’s Unity Catalog permissions, so it does not send metadata relating to tables that the user does not have permission to see.

Q: If I execute a query with results, and then ask a question, do the results of my query get sent to the model?​

No, only the code contents in cells, metadata about tables, and the user-entered text is shared with the model. For the “fix error” feature, Databricks also shares the stack trace from the error output.

Q: If I share my notebook or query with another internal user, can they see my chat history?​

No. Interactions with Assistant are visible only to the user who initiated them.

Q: Will Databricks Assistant execute dangerous code?​

No. Databricks Assistant does not automatically run code on your behalf. AI models can make mistakes, misunderstand intent, and hallucinate or give incorrect answers. Review and test AI- generated code before you run it.

Q: Has Databricks done any assessment to evaluate the accuracy and appropriateness of the Assistant responses?​

Yes. Databricks has mitigations to prevent the Assistant from generating harmful responses such as hate speech, insecure code, prompt jailbreaks, and third-party copyright content. Databricks has done extensive testing of all our AI assistive features with thousands of simulated user inputs to assess the robustness of mitigations. These assessments focused on the expected use cases for the Assistant such as code generation in the Python, Databricks SQL, R, and Scala languages.

Q: Can I use Databricks Assistant with tables that process regulated data (PHI, PCI, IRAP, FedRAMP)?​

Yes. To do so, you must comply with requirements, such as enabling the compliance security profile, and add the relevant compliance standard as part of the compliance security profile configuration.

Last updated on