Skip to main content
Last updated on

Connect to Apache Kafka for managed ingestion

Beta

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.

This page shows how to create a Unity Catalog connection to your Apache Kafka cluster. The managed Kafka connector uses this connection to authenticate to your cluster without requiring credentials in your pipeline configuration.

Authentication methods​

The Kafka connector supports two authentication methods. Choose the method that matches your Kafka cluster configuration:

  • Username and password: Authenticate using a Kafka username and password (SASL/PLAIN or SASL/SCRAM).
  • Service Credential: Authenticate using a Unity Catalog service credential. Select an existing service credential or create a new one, then provide the bootstrap servers of the Kafka cluster.

For details on setting up authentication on the Kafka side, see Authentication.

Next steps​

Use Catalog Explorer to create a Kafka connection so that any user with USE CONNECTION or ALL PRIVILEGES can create pipelines. See Create a Kafka connection.

On this page