What is Lakeflow Designer?
Lakeflow Designer provides a visual canvas for analysts to perform data analytics, preparation, and basic automation. In Designer, you create visual data prep files, each made up of a sequence of operators (such as filter, join, and transform) arranged as a graph to produce a result. All transformations are backed by code, which allows you to version files in Git and schedule them as jobs to move to production.
With Lakeflow Designer, you can:
- Design workflows using a drag-and-drop canvas.
- Transform data without writing code using built-in operators for filtering, aggregating, joining, and reshaping data.
- Use Genie Code to generate or refine transformations in your visual data prep using natural language.
- Preview each interim step without running the entire visual data prep.
To build your first visual data prep, see Tutorial: Build a visual data prep.
Tour the interface
The Lakeflow Designer editor has the following areas:
- Left pane: Browse operators, find operators in the table of contents, browse workspace files and catalog data, and define parameters.
- Header and toolbar: Pick compute, schedule the visual data prep, set preview limits, and zoom and auto-layout the canvas.
- Canvas: Add operators and connect them to build a visual workflow.
- Configuration pane: Fill in the details of the selected operator. It opens on the right.
- Output pane: View and compare the selected operator's input and output data. It opens at the bottom.
- Genie Code prompt: Describe changes in natural language.
For a guided tour of each area, see The Lakeflow Designer interface.
Key concepts
Visual data prep
A visual data prep is a Lakeflow Designer file that holds one data workflow, from sources through transformations to outputs. See Manage visual data prep files in Lakeflow Designer.
Operators
Operators are the building blocks of a visual data prep. Each operator performs one action, such as a join, filter, or write, and you configure it for your data. Lakeflow Designer includes built-in operators for common tasks, and you can add your own user-defined operators.
Connections
Connections link operators and define how data flows between them, from left to right. See Connect operators.
Previews
Lakeflow Designer previews each operator on a sample of rows, so you can check every step without running the whole visual data prep. See Preview and refine results in Lakeflow Designer.
Generated code
Every visual data prep is backed by generated code, so you can version it in Git and schedule it as a job. See Move a visual data prep file to production.
Genie Code
Genie Code is the AI assistant in Lakeflow Designer. Describe a transformation in natural language, and Genie Code adds or changes operators for you. See Use Genie Code in Lakeflow Designer.