Customize and improve Databricks Assistant responses

This page provides tips and best practices to help Databricks Assistant provide better responses.

Customize Assistant responses by adding instructions

You can customize how Databricks Assistant responds by adding custom instructions. The assistant considers these instructions when it generates new responses.

For example, instructions can include:

• Information about your preferences, such as which libraries you prefer to use when writing code.
• Context about who you are, such as your name and role.
• Coding conventions to follow.
• Guidelines to follow.
• Directions on how the Assistant should respond, such as using a casual tone.

With the exception of Quick Fix and Autocomplete, Databricks Assistant applies instructions to every response it generates, including Inline Assistant, General Chat, Suggest Fix, and Edit mode.

Databricks Assistant supports two types of instructions: user instructions and workspace instructions. Unless otherwise instructed, the assistant generally prioritizes workspace instructions over user instructions.

Add user instructions

User instructions let you provide system-level instructions to the Assistant. They're a persistent way to share context, preferences, or preferred authoring methods. These instructions are specific to you and are only applied to your interactions with the Assistant.

To add user instructions:

1. Open the Assistant pane by clicking in the upper-right corner of your workspace.

2. In the Assistant pane, click to open the Assistant settings.

   

3. Under User instructions, click Add instructions file. This creates a .assistant_instructions.md file in your default user workspace directory (/Users/<your-username-or-email>) and opens it in a new file tab.

   If you’ve already created an instructions file, click Open instructions file to open the file.

4. Edit the .assistant_instructions.md file to add your custom instructions. Databricks recommends adding your instructions in plain English text, separating each instruction as a new bullet point or paragraph. See Best practices for Assistant instructions.

The Assistant automatically picks up the instructions the next time you interact with it.

You can also use the Assistant to add instructions for you. From the Assistant panel or an inline Assistant input box, use the /addInstructions command.

Add and view workspace instructions

note

Only Workspace admins can add or edit workspace instructions.

Workspace instructions are configured by your workspace admin and provide more context to the Assistant to help it follow guidelines and operate more efficiently in your workspace.

If your workspace does not have workspace instructions yet, a workspace adminstrator can create the instructions file by following these steps:

1. In the Workspace/ directory of the workspace, create a new file named .assistant_workspace_instructions.md.
2. Edit the file to add instructions for the Assistant that should apply workspace-wide. Databricks recommends adding your instructions in plain English text, separating each instruction as a new bullet point or paragraph. See Best practices for Assistant instructions.

The Assistant automatically picks up the new workspace instructions the next time a user interacts with it.

To view or edit workspace instructions:

1. Open the Assistant pane by clicking in the upper-right corner of your workspace.

2. In the Assistant pane, click to open the Assistant settings.

3. Under Workspace instructions, click View file. This opens the .assistant_workspace_instructions.md file in a new tab.

4. If you’re a workspace administrator, you can edit the file and add instructions that apply to the Assistant’s behavior throughout the workspace. See Best practices for Assistant instructions for more information on adding instructions.

   If you’re not an admin, you can still view the file to understand what instructions the Assistant is following in addition to any user instructions you may have provided.

   

Best practices for Assistant instructions

It’s important to consider the potential limitations of the Assistant when providing instructions. Here are some general guidelines and tips for writing instructions to optimize your experience and get the most out of this feature.

• Write clear, specific instructions: Be explicit and unambiguous when writing your instructions.
• Keep instructions under 4000 characters: Instruction files are limited to 4000 characters. Any instructions past this limit will not be used by the Assistant. Be concise and prioritize important instructions.
• Use headings and delimiters for structure: Instructions are in Markdown files. Use Markdown formatting in your instructions to provide structure. Use bullet points as delimiters to separate out distinct instructions. Use headings to group together a set of instructions. For example, you might have a heading for "Python code conventions" that includes a bulleted list of Python-specific guidelines that the Assistant should follow when working with Python code.
• Scope: The Assistant applies instructions to every response it generates when you interact with it using Inline Assistant, General Chat, Suggest Fix, Edit mode, etc. However, the Assistant does not consider instructions for Quick Fix and Autocomplete. Keep instructions broadly relevant to different Assistant functions to avoid unintended behavior.
• Provide context and references: While the Assistant uses surrounding context like notebook code and table metadata, it does not yet proactively fetch additional information based on the instructions. To ensure helpful responses, include any key details directly in the instructions, such as when to use a particular table or function.

Tips to improve Assistant responses

Providing instructions allows the Assistant to generate responses customized to your preferences. In addition to this, there are other general things you can do when interacting with the Assistant to help it return better responses.

Databricks Assistant considers the history of the conversation so you can refine your questions as you go.

Be specific

The structure and detail that Databricks Assistant provides varies from time to time, even for the same prompt. Try to provide the Assistant as much guidance as you can to help it return the information you want in the desired format, level of detail, and so on. Consider doing the following:

• Specify the level of detail you're looking for. For example, "Explain this code in a couple sentences" or "Explain this code line-by-line."
• Specify which library you want the Assistant to use. For example, "Create a visualization using Matplotlib" or "Create a visualization using Seaborn."
• Specify the structure of the response you want. For example, "Provide instructions in numbered steps" or "Provide instructions with the steps as bullet points with brief explanations."
• Be specific in your prompts. Databricks Assistant knows about your table and column schema and metadata, so you can use natural language to ask your question. For example, "List active and retired NFL quarterbacks' passing completion rate, for those who had over 500 attempts in a season." Databricks Assistant answers using data from columns such as s.player_id and s.attempts.

Databricks Assistant uses context to provide better answers

Databricks Assistant has access to table and column schemas and metadata. This allows you to use natural language and generate more accurate queries. For example, if a table has a column called State, you can ask Databricks Assistant to generate a list of users who live in Michigan.

Databricks Assistant uses the following context:

• Code or queries in the current notebook cell or Databricks SQL editor tab.
• Table and Column names and descriptions.
• Previous prompt questions.
• Favorite and active tables.
• For the diagnose error feature, the stack trace from the error output.

Because Databricks Assistant uses your conversation history to provide better and more accurate answers, you can ask Databricks Assistant to alter the output of a previous response without having to rewrite the entire prompt. Use Assistant's chat history to iteratively clean, explore, filter, and slice DataFrames in the Assistant pane.

Help the Assistant work with datasets

The Assistant can access context about the tables you're working with. Here are some things you can do to help it respond better when working with datasets:

• When searching for a table, use the prompt "Find tables" or the command /findTables for better responses. For example, "Find tables related to NFL games."

• Use @<table-name> to reference a specific table in a prompt. The Assistant will tailor its responses to use that table.

• When selecting columns from a DataFrame, you can get more accurate results by providing a starting query. For example, provide a statement like SELECT * FROM <table_name>. This allows Databricks Assistant to get the column names and not have to guess.

• If you need to use column data type conversions to run an operation, you might need to provide details. For example: "Convert this code from pandas to PySpark, including the code needed to convert the pandas DataFrame to a PySpark DataFrame and changing the data type of column churn from boolean to integer."

• Document tables and columns in Unity Catalog. See Add AI-generated comments to Unity Catalog objects. This gives the Assistant more context on the data. Use table or column comments in Catalog Explorer to add a line of sample data. For example, suppose your height column is in the format feet-inches. To help Assistant interpret the data, add a comment such as "The height column is in string format and is separated by a hyphen. Example: '6-2'." For information about table and column comments, see Add comments to data and AI assets.

Edit and run code in Databricks Assistant chat pane

Run code in the Assistant pane to validate it or use it as a scratchpad. To run code, click in the upper-left corner of the code box in the Assistant pane.

The tabs below show examples for Python and SQL code:

Python

SQL

When you run code in the Assistant pane, output is displayed and the variables become usable in the notebook.

You can also edit the code that Databricks Assistant generates directly in the Assistant chat box before moving the code to the notebook.

Use Shift + Enter to add a new line in the chat text box

Use Shift + Enter to add a new line in the Assistant chat text box. This makes it easy to format and organize your messages to Databricks Assistant.

Use cell actions in a notebook

Cell actions include shortcuts to common tasks, such as documenting (commenting), fixing, and explaining code.

Additional resources

For more tips, see:

• Databricks Assistant Tips and Tricks for Data Analysts
• Databricks Assistant Tips and Tricks for Data Engineers