Skip to main content
Last updated on

Create a Kafka connection

Beta

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.

Create a Kafka connection in Catalog Explorer to store authentication credentials for Lakeflow Connect ingestion. Any user with the USE CONNECTION privilege on the connection can then create Kafka ingestion pipelines without needing direct access to the credentials.

For privilege requirements, see Connect to managed ingestion sources.

Create a connection​

  1. In the Databricks workspace, click Data icon. Catalog > Create > Create a connection.

  2. On the Connection basics page of the Set up connection wizard, specify a unique Connection name.

  3. In the Connection type drop-down menu, select Kafka.

  4. In the Auth type drop-down menu, select either Username and Password or Service Credential. For instructions, see

    Authentication for Kafka.

  5. (Optional) Add a comment.

  6. (Username and Password) Enter the user identity and password of the Kafka cluster.

  7. (Service Credential) Select your service credential from the drop-down or click Create new service credential. Enter the bootstrap servers of the Kafka cluster.

  8. (Optional) Enter the URL, API key, and API secret for your schema registry.

  9. Click Create connection.

Next steps​

Create an ingestion pipeline.

On this page