Lakeflow Designer
Lakeflow Designer is a visual, no-code, AI-native experience for data preparation and analytics. Built directly in Databricks, it lets users prepare and explore data through a drag-and-drop canvas and natural language. The workflows you build, called visual data preps, are backed by production-ready code and fully governed by Unity Catalog, so your work moves smoothly from prototype to production, with no need to reverse-engineer or rebuild.
-
- What is Lakeflow Designer?
- Understand the interface and concepts of Lakeflow Designer.
-
- Tutorial: Build a visual data prep
- Build your first visual data prep, from source to output, with sample data.
-
- The Lakeflow Designer interface
- Tour the header, left pane, canvas, configuration pane, and output pane, and add and connect operators.
-
- Use Genie Code in Lakeflow Designer
- Build and edit visual data preps in natural language with Genie Code.
-
- Preview and refine results in Lakeflow Designer
- Preview results, run the full visual data prep, and edit data in the results table.
-
- Use the table of contents in Lakeflow Designer
- Find, filter, and sort operators, and view generated code.
-
- Ingest data into Lakeflow Designer
- Ingest data into a visual data prep.
-
- Use parameters in Lakeflow Designer
- Use parameters to make a visual data prep reusable across environments.
-
- Built-in operators in Lakeflow Designer
- Explore the built-in operators available in a visual data prep.
-
- Manage visual data prep files in Lakeflow Designer
- Create, export, import, rename, delete, and review the history of visual data prep files.
-
- Move a visual data prep file to production
- Bring a visual data prep to production with Git, jobs, and Declarative Automation Bundles.
User-defined operators
You can create your own user-defined operators for use in Lakeflow Designer.
Task | Description |
|---|---|
Learn how user-defined operators work, and register, update, and troubleshoot them. | |
Write the Python or Unity Catalog function logic that runs when a user-defined operator executes. | |
A complete reference to the YAML schema for user-defined operators. | |
Build a user-defined operator that calculates compound interest, using Python in a Unity Catalog user-defined function (UDF). | |
Build a user-defined operator that uses K-means clustering, using Python in a Unity Catalog user-defined table function (UDTF). | |
Build a user-defined operator that sends a Slack message, using SQL in a Unity Catalog user-defined function (UDF). | |
Build a user-defined operator that uses all available UI widgets, using Python in a Unity Catalog user-defined function (UDF). | |
Build a user-defined operator that sends Gmail, using Python in a standalone |