Get started with Genie ZeroOps

Beta

This feature is in Beta. To use it, a workspace admin must turn on Genie ZeroOps from the Previews page. See Manage Databricks previews.

To start using Genie ZeroOps, choose the jobs, pipelines, and tables it monitors from the Jobs & Pipelines page. Genie ZeroOps then backfills insights from the previous 24 hours and keeps reporting new issues for you to review and resolve. For an overview, see Genie ZeroOps.

Requirements

• A Databricks workspace with the Genie ZeroOps preview enabled. A workspace admin must turn it on from the Previews page. See Manage Databricks previews.
• OWNER, CAN MANAGE, or CAN VIEW permission on the jobs, pipelines, or tables that you want to monitor.
• Optional: For jobs that run code from a Git repository, a Git credential with read access to the repository. To create pull requests from fixes, the credential also needs write access. See Add Git credentials.

Set up Genie ZeroOps

1. In the sidebar, click Jobs & Pipelines to open the Jobs & Pipelines page.

2. Click the Open ZeroOps button.

3. If you have not configured Genie ZeroOps yet, you are prompted to do so. Click the Configure button to open the Genie Code chat.

4. When Genie Code prompts you for which resources to monitor, select a suggested scope, or describe your own in natural language:

   prompt

   Tell Genie Code what to monitor, for example:

   all owned jobs

After you set the scope, Genie ZeroOps analyzes your selected assets, backfilling insights from the previous 24 hours for jobs that have already failed and for tables that are stale or incomplete. It then displays any insights it finds for issues within that scope.

You can select the Configure button at any time to adjust what Genie ZeroOps monitors. See Update Genie ZeroOps configuration.

Configure notifications

Genie ZeroOps can send a notification when an insight first appears in your inbox. It supports Slack, Microsoft Teams, PagerDuty, and generic webhook destinations. Genie ZeroOps currently doesn't support email destinations.

Before you configure notifications in Genie ZeroOps, a workspace admin must create the destinations in your workspace. See Manage notification destinations.

1. In the top-right corner of the Genie ZeroOps inbox, click Configure.

2. Ask Genie Code to configure notifications. For example:

   prompt

   Text

   Send new ZeroOps insights to <destination-name>.

   If you don't know the destination name, ask What notification destinations are available for ZeroOps insights?, then choose a destination from the response.

3. In the Your ZeroOps configuration card, click View, then review Notifications to confirm the saved destinations.

Example: Create Jira work items with a webhook

Use a generic webhook destination and a Jira Cloud automation flow to create a work item for each new insight:

1. In Jira Cloud, create an automation flow with an Incoming webhook trigger. See Configure the incoming webhook trigger in the Atlassian documentation.

2. Add a Create work item action. Use Jira smart values to map the Genie ZeroOps webhook fields, for example:

   Text

   Summary: {{webhookData.title}}
   
   Description:
   {{webhookData.insight_summary}}
   
   Asset type: {{webhookData.assets.first.asset_type}}
   Asset: {{webhookData.assets.first.asset_name}}
   View insight: {{webhookData.insight_link}}

3. Ask a workspace admin to create a Webhook notification destination with the Jira incoming webhook URL. Jira requires a secret for incoming webhooks. Because notification destinations don't support custom headers, use the URL format that includes the secret, as described in the Atlassian documentation.

4. In the Genie ZeroOps configuration chat, select the new webhook destination.

5. When a new insight appears in your inbox, check Jira for the corresponding work item.

Review an insight and its root cause

Besides opening the Genie ZeroOps inbox from the Jobs & Pipelines page, you can also access it from full page Genie Code, where it appears in the thread list as ZeroOps Inbox.

1. In the inbox, click an open insight to open its thread.
2. Review the root cause analysis and the proposed fix.
3. In the same insight thread, ask Genie Code to apply the proposed fix. Alternatively, follow the link to the source and apply the fix yourself.
4. Genie Code applies the fix directly to code stored in the workspace. For assets stored in Git, it helps you open a pull request. See Create a pull request from a fix.
5. To mark the insight as resolved, click Resolve in the inbox. Otherwise, Genie ZeroOps automatically marks an insight as resolved after the next successful job run. The status typically updates within 15 minutes.

Create a pull request from a fix

If a job runs code from a Git repository and the fix is a code change, you can have Genie ZeroOps push the fix to a new branch and then open a pull request. Genie ZeroOps prepares the change during its investigation and shows a Create PR option in the insight. It doesn't push anything to your repository until you explicitly instruct it to.

1. In the inbox, click the insight to open its thread.
2. Review the proposed fix. To change it, describe the change you want in the thread.
3. Click Create PR, and then confirm the push when prompted. Genie ZeroOps commits the fix to a new branch and pushes it.
4. To open a pull request, click the link in the Genie ZeroOps reply, or create the pull request from the new branch if the reply has no link. Then get the pull request reviewed and merged through your regular process.

Update Genie ZeroOps configuration

To change what Genie ZeroOps monitors after setup, click Configure in the top-right corner of the inbox. A Genie Code chat opens where you describe the jobs, pipelines, and tables you want to include or exclude.

Share your Genie ZeroOps configuration

Share your configuration with other users in your workspace so that they can work on the insights it generates. Their inbox lists your configuration's insights with Shared with me in the Source Config column, next to their own insights.

note

Users you share with see the code excerpts, suggested fixes, and sample data rows included in insights, even if they don't have access to the underlying assets. Sharing doesn't give them access to the assets themselves.

1. In the top-right corner of the inbox, click Configure to open the Genie Code chat.
2. On the Your ZeroOps configuration card, click Share.
3. Click Select user..., choose a workspace user, and then click Add. Repeat for each user you want to share with.
4. Click Save.

To stop sharing with a user, open the Share dialog, click the remove icon next to the user, and then click Save.

What shared users can do

Users you share with have the CAN MANAGE INSIGHTS permission on your configuration. They can do the following:

• Open your configuration's insights and review the root cause and the proposed fix.
• Follow up on an insight in their own thread, where they can ask about the root cause or refine the proposed fix.
• Mark an insight as Not an issue. The status changes for everyone who can see the insight.

Users you share with can't change your configuration, including what it monitors, who it's shared with, and its notification settings.

Customize investigation instructions

Custom instructions tell Genie ZeroOps what it should and shouldn't do when it investigates your issues. For example, you can tell it to check upstream data freshness before it recommends a retry, or to follow a specific step when it sees a particular error.

Your custom instructions apply only to investigations for your Genie ZeroOps configuration, and are stored in a Markdown file in your home folder:

Text

/Workspace/Users/<your-username>/.assistant/zeroops/instructions.md

Access to this file follows the same workspace permissions as any other workspace file.

Edit instructions

Using the chat interface

1. In the top-right corner of the inbox, click Configure to open the Genie Code chat.

2. Describe your guidance in natural language:

   prompt

   Tell Genie Code what Genie ZeroOps should do, for example:

   Text

   When a job fails, check upstream data freshness before recommending a retry.

Genie Code adds your guidance to the instructions, and future investigations account for it.

Direct edits

1. In the Genie Code configuration chat, find the Your ZeroOps configuration card and click View.
2. Under Custom instructions, click this file.
3. Edit the file. You can also describe your changes in the chat that opens next to it.

When your instructions file has content, the Custom instructions section shows Configured.

Limitations

• New investigations for issues in your configuration load your instructions. Saving instructions does not change an investigation that is already in progress.
• Genie ZeroOps reads only the first 20,000 characters of the file. Keep your instructions concise and specific, because irrelevant or conflicting guidance can reduce investigation quality.
• Each investigation report states whether it used your instructions.
• Instructions cannot override Genie ZeroOps safety constraints or tool permissions. If Genie ZeroOps does not apply an instruction, it briefly explains why in the investigation report.

Filter insights

1. Click Filter.
2. Select a status, such as Resolved.
3. Apply the filter to update the inbox.

Troubleshooting

• No insights appear after you set up Genie ZeroOps: Verify that you have OWNER, CAN MANAGE, or CAN VIEW permission on the jobs, pipelines, or tables you expect to see. Genie ZeroOps monitors only the assets you have permission on.
• Genie ZeroOps can't access a repository: Your Git credential is missing, expired, or can't access the repository. Click your username, select Settings > Linked accounts, and add or update the credential. See Add Git credentials.
• An insight doesn't show Create PR: Check that the job runs code from a Git repository and that your Git credential can access the repository. Genie ZeroOps offers Create PR only when the fix is a code change.
• Genie ZeroOps can't push the fix: Your Git credential needs write access to the repository. Click your username, select Settings > Linked accounts, and update the credential so that it has write access. Then ask Genie ZeroOps to push the fix again.