Set up and manage an AI/BI Genie space

This article explains how to set up and manage a Genie space, a chat interface for business users to ask natural-language questions about their data.

Technical requirements and limits

The following requirements and limits apply when using Genie spaces:

• Unity Catalog: The data for the Genie space must be registered to Unity Catalog. You can add up to 25 tables or views to a Genie space.
• Compute: Genie spaces require a pro or serverless SQL warehouse. You must have at least CAN USE permission on the selected warehouse. If this permission is missing, space creation fails.
• Throughput: Each workspace can handle up to 20 questions per minute across all Genie spaces.
• Capacity: Each Genie space can support up to 10,000 conversations, and each conversation can include up to 10,000 messages.

Required permissions

The following permissions are required to create, edit, and use Genie spaces:

• Compute permissions: Genie space creators, editors, and users require at least CAN USE access to the default SQL warehouse designated for the space.
• Data access permissions: Privileges granted in Unity Catalog control which users can access specific data objects. Any user who interacts with the space needs at least SELECT privileges on the data used in that space.
• Genie space permissions: Users need CAN RUN permissions on the Genie space to interact with Genie. See Genie space ACLs for a complete mapping of privileges and abilities for a Genie space.

note

Configuring data and compute access requires elevated permissions generally restricted to an administrator. See Create a SQL warehouse and Manage privileges in Unity Catalog.

Manage Genie access

Genie uses partner-powered AI assistive features that must be enabled at the account level. To learn how to manage these features for your account, see Enable and try Databricks Assistant.

note

You must be an account administrator to manage access to this feature. If you disable partner-powered assistive AI features, users with the SQL entitlement can still click the Genie icon in the sidebar, but they are not able to access any Genie spaces.

Create a Genie space

Genie spaces allow data analysts to create a domain-specific Genie space experience for business users by taking the following steps:

1. Click Genie in the sidebar.

2. Click New in the upper-right corner of the screen.

3. Choose the data sources that you want to include in your Genie space. Then, click Create.

   

Your new space is created. Type a question in the chat to start interacting with the space.

Configure settings

Customize your Genie space by configuring addtional settings. Click Configure and then Settings to access the following settings:

• Title: The title appears in the workspace browser with other workspace objects. Choose a title that will help end users discover your Genie space.
• Default warehouse: This compute resource powers the SQL statements generated in the Genie spaces. End users need access to this warehouse to interact with the space. A Genie space can use a pro or serverless SQL warehouse. For optimal performance, Databricks recommends using a serverless SQL warehouse.
• Description: Users see the description when they open the Genie space. Use this text area to describe the spaces’s purpose. The description field supports Markdown formatting so that you can style your text and supply links to helpful context and references. For more information on markdown syntax, see the Markdown Guide cheat sheet.
• Sample questions: Sample questions are optional. They appear in chats to provide examples of the types of questions users can ask. They can be added during creation or later from the space’s Settings.

Expect to iterate on your Genie space based on testing and usage. For guidance on best practices for creating and iterating on a space, see Curate an effective Genie space.

Add context

Genie uses Unity Catalog metadata, Unity Catalog functions, instructions, and sample SQL queries to help generate responses. Click Configure to open the editor. The Context tab includes tabs that show available Data, Instructions, and SQL Queries. Use the SQL Queries tab to manage queries and Unity Catalog functions.

You can add up to 100 instructions in total for a Genie space. Instruction types contribute to the count in the following ways:

• The entire block of text in General instructions counts as one.
• Each example SQL query counts as one.
• Each SQL function counts as one.

Manage data

The Data tab shows the data tables used to provide answers. Click the Add button to add more tables. Click the Trash icon to the right of the table name to remove a table from the space.

Provide instructions

Click the Instructions tab to write plain text instructions that help Genie understand how to respond to specific questions about your business. You can format the instructions as a single comprehensive note or categorize them by topic for better organization.

Instructions help guide Genie’s responses so that it can process the unique jargon, logic, and concepts in a given domain. Comprehensive instructions are critical to a seamless, intuitive Genie space experience.

The following example includes information you could include in general instructions:

• Company-specific business information:
  • “Our fiscal year starts in February”
• Values, aliases, or common filters:
  • “Always convert to lowercase and use a like operator when applying filters.”
  • “Use abbreviations for states in filter values.”
• Specify functions to answer questions and provide instructions for return values:
  • “For quarters use the adventureworks.oneb.get_quarter(date) UDF. The output of get_quarter() is the quarter and is either 1,2,3, or 4. Use this to filter the data as needed.

Add example SQL queries and functions

Use the SQL Queries tab to add the following:

• Example queries (Recommended): Example SQL queries help Genie generate the correct SQL to answer user questions. Queries can be static or parameterized. For each example SQL queries, you provide a sample question and the matching SQL that returns the expected result. Genie can directly use the example for matching questions, or learn from it to answer related questions. When a parameterized query is used to generate a response, it is marked as Trusted. Users with CAN EDIT privileges in the space can view the query used to generate the response, which can help with troubleshooting and refinement.

• SQL functions:: For questions that cannot be answered with a static or parameterized SQL query, you can register a custom function to Unity Catalog. Functions can be shared across your teams and used by Genie to answer specific questions. Responses that are answered using a SQL function are marked as Trusted. To learn more about using SQL functions in your Genie space, see Use trusted assets in AI/BI Genie spaces.

How Genie uses example queries

Example queries show Genie how to use the available data to answer questions. Enter a sample question into the text field, and then enter a SQL query that answers that question. When Genie receives a matching question, it can use the example query directly to provide an answer. When Genie gets a similar question, it uses clues from the example query to learn and structure the SQL provided in the response. Focus on providing samples that highlight logic that is unique to your organization and data, as in the following example:

SQL

  -- Return our current total open pipeline by region.
  -- Opportunities are only considered pipelines if they are tagged as such.
  SELECT
    a.region__c AS `Region`,
    sum(o.amount) AS `Open Pipeline`
  FROM
    sales.crm.opportunity o
    JOIN sales.crm.accounts a ON o.accountid = a.id
  WHERE
    o.forecastcategory = 'Pipeline' AND
    o.stagename NOT ILIKE '%closed%'
  GROUP BY ALL;

Parameterized example queries allow space users to specify a certain value to be inserted in the query at runtime. To learn more about working with parameterized queries, see Use parameters in SQL queries.

How does Genie use SQL functions?

SQL functions are useful when a question involves complex logic that cannot be captured with a static or parameterized query. They are stored in Unity Catalog and can be called by Genie using user-supplied parameters. Genie cannot view or modify the SQL used in the function, making this approach well-suited for logic that should not be surfaced or changed. For guidance on registering a function in Unity Catalog, see Create a SQL table function and User-defined functions (UDFs) in Unity Catalog.

Test your Genie space

Most user interactions take place in the chat window. The best way to learn if your space is working as you want it to is to test it with realistic questions that you expect your business users to ask.

Sample questions configured in the space settings appear in the chat window. Genie can also generate sample questions based on the space's context to help users begin exploring the data. Users can click a sample question or enter their own questions in the text field at the bottom of the screen.

Responses appear above the text field. After a user enters a question, it is saved to the chat history.

To start a new conversation:

1. Click New chat to start a new chat. Click to open a previous conversation.
2. Type your question into the Ask your question… text entry field.

Review responses

Responses are typically delivered as a natural language answer to the questions and a table showing the relevant result set. When Genie detects that a visualization could improve response clarity, it also returns a visualization. The precise response structure varies based on the question. If a SQL query was generated to answer the question, it is included in the response.

note

Genie, like other large language models (LLMs), can exhibit non-deterministic behaviors. This means that you might occasionally receive different outputs when submitting the same prompt multiple times. Providing example SQL queries that Genie can learn from can help make Genie more consistent. See Add example SQL queries and functions.

Response feedback

Each response prompts the user to answer Is this correct?. Users can respond in one of the following ways:

• Yes: Confirms the response appears accurate.
• Fix it: Flags the response as incorrect. Users can select from a list of common issues or enter their own explanation. They can then:
  • Click Submit and try again to regenerate the response using the provided feedback.
  • Click Submit to send the feedback without regenerating the response.
• Request review: Flags the response for manual review. Users can add an optional comment to give you additional context.

As an editor, you can view feedback and flagged responses in the Genie interface. Use this feedback to identify improvement opportunities or respond directly to user questions. Databricks recommends encouraging users to provide feedback on the space using this mechanism.

Business users can view updates to the questions they’ve marked for review on their Monitoring page. Users with at least CAN MANAGE permission on the Genie space can review the specific exchange, comment on the request, and confirm or correct the response. They can access feedback and review requests on the monitoring page. Then you can use that feedback to tune responses and iterate on your space. See Monitor the space.

Suggested follow-up questions

Genie uses space instructions and conversation context to suggest follow-up questions. These questions can help users explore the data from different angles or glean new insights.

Other response actions

For responses that include generated SQL, additional options allow you to interact with the returned data.

• Download all rows: Space users can download up to approximately 1GB of results data as a CSV. The final file download size might be slightly more or less than 1GB, as the 1GB limit is applied to an earlier step than the final file download. To download results, click the download icon in the response.
• Add as instruction: For interactions that might be useful for teaching Genie how to answer similar questions, click Add as instruction. This opens the UI for saving example SQL queries, populated with the question and generated SQL. You can leave the example as written, or edit and save to make changes. See Add example SQL queries and functions.
• Show code: Click Show code to view the generated query. This can be useful for troubleshooting unreliable responses. See Edit and save queries.
• The kebab menu: Users can copy the response CSV to their clipboard or refresh the data by running the previously generated query.

To edit a visualization, click the edit icon in the upper-right corner of the visualization. You can also generate a visualization by describing it in words after getting a query result. See Generate visualizations.

Format results

To format a column in a table within a response, click the kebab menu in the column header, then click Format to view the available options. Formatting options depend on the column’s data type.

Edit and save queries

You can review and edit the generated SQL in any response. Genie space authors typically know the domain and data that allows them to recognize when Genie is generating an incorrect answer. Often, errors can be fixed with a small amount of manual tuning to the generated SQL query. To view the generated SQL for any response, click Show generated code to inspect the query.

You can edit the generated SQL statement to correct it if you have CAN EDIT or greater privileges on the Genie space. After you’ve made your corrections, run the query. Then, you can save it as an instruction to teach Genie how to answer in the future. To save your edited query, click Add as instruction.

Generate visualizations

Users can ask for a specific visuallization to be generated as part of a response. For many questions, Genie includes a visualization as part of a response. To hide an automatically generated visualization, click the collapse icon. To show it again, click Show visualization.

You can edit visualizations in the chat window.

The following chart types are supported:

• Area
• Bar
• Line
• Pie
• Scatter

To edit a visualization:

• In the upper-right corner of the visualization, click the edit icon. An editing pane opens on the right side of the page.
• Use the Visualization drop-down menu to choose a different chart type.
• For most chart types, use the X axis and Y axis drop-down menus to choose the values represented on each axis. For Pie charts, use the Angle drop-down menu to choose the column you want to represent.
• (Optional) To customize the color scheme, click the + in the Color category.
• (Optional) To select data to show in the tooltip when you hover over the visualization, click the + in the Tooltip section.

Monitor the space

You can think of a Genie space as a long-term collaboration tool between data teams and business users. It accumulates knowledge over time rather than serving as a one-time deployment. As users ask new questions, you can refine the space to improve coverage and accuracy.

Use the Monitoring tab to review individual questions and responses, view user feedback, and identify responses flagged for review.

The monitoring tab shows all of the questions and answers that have been asked in the space. You can filter questions by time, rating, user, or status. By monitoring the space, Genie space editors can proactively understand the queries raised by business users and how the Genie space responded.

By identifying the questions that Genie struggles with, you can update the Genie space with specific instructions to improve its responses. Click a question to open the question and response text, and view the complete chat thread.

Share a Genie space

important

Genie space users must use their own credentials to access data. Any question about data they can’t access generates an empty response.

Genie space users must have CAN USE permissions on the warehouse attached to a Genie space and access permissions on the Unity Catalog objects surfaced in the space. See What data should I use?.

New Genie spaces are saved to your user folder by default. Like other workspace objects, they inherit permissions from their enclosing folder. You can use your workspace folder structure to share them with other users. See Organize workspace objects into folders.

You can also specify certain users or groups to share with at a given permission level: CAN MANAGE, CAN EDIT, CAN RUN, and CAN VIEW.

To share with specific users or groups:

1. Click Share.
2. Enter users or groups you want to share the space with. Then, click Add and set appropriate permission levels. Individual users and members of small groups receive an email notification confirming that the space has been shared.
3. Use the Copy link button at the bottom of the Share dialog to get a shareable link to the Genie space. Privileged users can click the link to open the Genie space in a new tab and ask questions.

Clone a Genie space

Cloning a Genie space creates a copy that includes all setup context and instructions. Cloning is useful when you want to test changes in a separate space or reuse the original context in a new one. After a space is cloned, the new space is independent of the original. You can make edits and adjustments without impacting the original.

The following elements are copied to a cloned Genie space:

• Tables and settings
• General instructions
• Example SQL queries
• SQL functions

Existing chat threads and Monitoring data are not copied to the new space.

To clone a Genie space, do the following:

1. Open the Genie space you want to clone.
2. Click the Kabab menu kebab menu in the upper-right corner of the space.
3. Click Clone.
4. (Optional) In the Clone dialog, specify a new name and workspace folder location for the cloned Genie space.
5. Click Clone to create the new space.

Best practices for space preparation

For guidance about best practices and help with troubleshooting issues, see Curate an effective Genie space.

Monitor usage

You can use audit logs to monitor usage of your Genie space. See AI/BI Genie events.

Use the Genie conversation API

Preview

This feature is in Public Preview.

The Genie Conversation API exposes an endpoint for an existing Genie space where you can ask questions, retrieve generated SQL and query results, or get the details associated with a Genie space. The throughput rates for the Conversations API are best-effort and depend on system capacity. Under normal or low-traffic conditions, requests are limited to 5 queries per minute per workspace. During peak usage periods, actual throughput can be lower as requests are processed based on available capacity.

To learn how to create a multi-agent system using Mosaic AI Agent Framework, see Use Genie in multi-agent systems.