What is a Genie space?

Preview

This feature is in Public Preview.

A Genie space is a no-code interface powered by DatabricksIQ where business users can interact with the Databricks Assistant to analyze data using natural language. Domain experts, like data analysts, configure Genie spaces with datasets, sample queries, and text guidelines to help the Assistant translate business questions into analytical queries. After set up, business users can ask questions and generate visualizations to understand operational data.

See DatabricksIQ-powered features.

Data analysts can prepare a domain-specific Genie space experience for business users by doing the following:

  • Selecting relevant tables from Unity Catalog and exposing their metadata (table and column descriptions) in the Genie space.

  • Adding instructions that transfer organization-specific information (business logic and metadata) into the Genie space.

Example use cases

You can create different Genie spaces to serve a variety of different non-technical audiences. The following scenarios describe two possible use cases.

Get status with visualization

A sales manager wants to get the current status of open and closed opportunities by stage in their sales pipeline. They can interact with the Genie space using natural language, and automatically generate a visualization.

The following gif shows this interaction:

Gif with sample question, response, and auto-generated visualization

Tracking logistics

A logistics company wants to use Genie spaces to help business users from different departments track operational and financial details. They set up a Genie space for their shipment facility managers to track shipments and another for their financial executives to understand their financial health.

Technical requirements

  • Genie spaces use data registered to Unity Catalog.

  • Genie spaces require a Pro or Serverless warehouse.

  • Creating Genie spaces with the Databricks Assistant requires enabling Partner-powered AI assistive features. For details on enabling Databricks Assistant, see What is Databricks Assistant?. For questions about privacy and security, see Privacy and security.

How are Genie space responses generated?

Genie spaces generate responses to natural language questions using table and column names and descriptions. The actual data in the tables remains hidden from the Assistant.

The Assistant uses the names and descriptions to convert natural language questions to an equivalent SQL query. Then, it provides a response that includes the results of that query as a table. Genie space authors and end users can inspect the generated SQL query that produces each response.

When creating visualizations, the first row of query results is shared with the Assistant. This preserves data privacy while leveraging database annotations to inform responses.

Required permissions

You must have at least CAN USE privileges on a SQL warehouse to set up a Genie space. When you save your Genie space, you are prompted to select a default SQL warehouse that will be used to generate responses to user questions.

Access to data for Genie space authors and end-users is governed by Unity Catalog permissions. See Manage privileges in Unity Catalog.

How do I add data?

Genie spaces work exclusively with data objects registered to Unity Catalog. They use the metadata attached to Unity Catalog objects to generate responses. Well-annotated datasets, paired with specific instructions that you provide, are key to creating a positive experience for end users.

Databricks recommends the following:

  • Curate data for analytical consumption: Layer views to reduce the number of columns and add use-case-specific information to increase response quality.

  • Minimize the number of columns in a Genie space: Use up to five closely related tables. Each table should hold fewer than 25 columns.

Genie Spaces fully respects and enforces UC permissions, including row-level security and column-based masking. Users must have SELECT privileges on the data and CAN USE privileges on the catalog and schema.

You can create new Genie spaces based on one or more Unity Catalog managed tables. Closely related, well-annotated datasets, paired with specific instructions that you provide, are critical to creating a positive experience for end users.

Create a new Genie space

When you create a new Genie space, a New Genie space dialog shows the following options.

  • Title: The title appears in the workspace browser with other workspace objects. Choose a title that will help end users discover your Genie space.

  • Description: Users see the description when they open the Genie space. Use this text area to describe the room’s purpose.

  • Default warehouse: This compute resource powers the SQL statements generated in the Genie spaces. A Genie space can use a pro or serverless SQL warehouse. Serverless SQL warehouses offer optimal performance.

  • Tables: Genie spaces can be based on one or more tables. The dialog prompts you to add a table by choosing from each drop-down selector: Catalog, Schema, and Table.

When you have selected a table, it is automatically added to the room. To add another table, use the drop-down selectors to choose another table.

Chat in the Genie space

After it is created, most Genie space interactions take place in the chat window.

A new chat window includes a set of Quick actions tiles that can help users get started with the Genie space. The text field, where users input questions, is near the bottom of the screen.

New chat window with help tiles at the top of the screen and a text input field at the bottom.

Responses appear above the text field. After a user enters a question, it is saved to a chat history thread in the left pane.

Chat history

Chat history threads are saved for each user so that they can refer to past questions and answers. Users can also resubmit or revise questions from a chat thread. The New chat button in the left pane starts a new thread.

Each chat thread maintains its context, so the Assistant considers previous questions it has been asked. This allows users to ask follow-up questions to further explore or refocus a result set.

Response structure

The precise response structure varies based on the question. Often, it includes a natural language explanation and a table that shows the relevant result set. All responses include the SQL query that was generated to answer the question. Click Show generated code to view the generated query.

The bottom-right side of the response includes optional actions. You can copy the response CSV to your clipboard, download it as a CSV file, add it as an instruction for the Genie space, and upvote or downvote the answer.

A set of Quick actions tiles follow responses that include tabular data. You can use them to generate visualizations.

Quick action tiles that suggest different visualization options.

You can also generate a visualization by describing it in words.

Provide instructions

Instructions help to guide the Assistant’s responses so that it can process the unique jargon, logic, and concepts in a given domain. You can write instructions as example queries or snippets of plain text that help the Assistant answer questions that room users are likely to ask. Comprehensive instructions are critical to a seamless, intuitive Genie space experience.

The following examples illustrate various types of instructions:

  • Company-specific business information:

    • “Our fiscal year starts in February”

  • Values, aliases, or common filters:

    • “Always convert to lowercase and use a like operator when applying filters.”

    • “Use abbreviations for states in filter values.”

  • User-defined functions available through Unity Catalog:

    • “For quarters use the adventureworks.oneb.get_quarter(date) UDF. The output of get_quarter is the quarter and is either 1,2,3, or 4. Use this to filter the data as needed.

    For example, for quarter 3, use where adventureworks.oneb.get_quarter(posted_date)= 3”`”

  • Sample SQL instructions:

    • You can provide samples of queries that you expect the Assistant to generate.

    • Focus on providing samples that highlight logic that is unique to your organization and data, as in the following example:

    -- Return our current total open pipeline by region.
    -- Opportunities are only considered pipeline if they are tagged as such.
    SELECT
      a.region__c AS `Region`,
      sum(o.amount) AS `Open Pipeline`
    FROM
      sales.crm.opportunity o
      JOIN sales.crm.accounts a ON o.accountid = a.id
    WHERE
      o.forecastcategory = 'Pipeline' AND
      o.stagename NOT ILIKE '%closed%'
    GROUP BY ALL;
    

You can organize Genie space instructions as one long note or group them by related topics for better structure. You can also add certified answers to provide validated responses for users. See Use certified answers in Genie spaces.

Best practices for room preparation

  • Include a set of well-defined questions that you want room users to be able to answer.

  • Test your Genie space to check response quality. Try the following to see if the model provides the expected response:

    • Rephrase the provided questions.

    • Ask other questions related to the datasets.

  • Add and refine Genie space instructions until questions provide the expected response.

Share a Genie space

Important

Genie space users must interact with data using their own credentials. Questions about data they cannot access generate empty responses.

Genie space users must have CAN_USE permissions on the warehouse attached to a Genie space, and access permissions on the Unity Catalog objects surfaced in the space. See How do I add data?.

New Genie spaces are saved to your user folder by default. Like other workspace objects, they inherit permissions from their enclosing folder. You can use your workspace folder structure to share them with other users. See Organize workspace objects into folders.

You can also specify certain users or groups to share with at a given permission level: Can Manage, Can Edit, Can Run, and Can View.

To share with specific users or groups:

  1. Click Share.

  2. In the Share dialog, click Open in Workspace.

  3. In the Workspace browser window, enter users or groups that you want to share with, and then set permission levels as appropriate.