Gmail connector

Beta

This feature is in Beta.

The managed Gmail connector in Lakeflow Connect ingests messages, labels, drafts, and filters from a Gmail mailbox into Databricks.

The connector reads from a single mailbox per connection using a Google service account, and it is read-only. It never modifies the source mailbox. The messages and message_labels tables sync incrementally using the Gmail History API. The other tables are re-ingested in full on every pipeline run.

Feature availability

Gmail is cloud-agnostic. The following feature availability applies on AWS and GCP.

Feature	Availability
UI-based pipeline authoring	Supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Supported
Incremental ingestion	Supported Only for the messages and message_labels tables. The other tables are full-refresh only.
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported Configuring SCD Type 2 for the messages or message_labels table causes pipeline validation to fail.

Feature	Availability
UI-based pipeline authoring	Supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Supported
Incremental ingestion	Supported Only for the messages and message_labels tables. The other tables are full-refresh only.
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported Configuring SCD Type 2 for the messages or message_labels table causes pipeline validation to fail.

Authentication methods

Authentication method	Availability
Google service account (OAuth service account with domain-wide delegation)	Supported
OAuth U2M	Not supported
Basic authentication (username/password)	Not supported

Authentication method	Availability
Google service account (OAuth service account with domain-wide delegation)	Supported
OAuth U2M	Not supported
Basic authentication (username/password)	Not supported

The connector authenticates to Gmail with a Google service account. You grant the service account domain-wide delegation for the https://www.googleapis.com/auth/gmail.readonly scope and choose the mailbox that the service account impersonates. For setup steps, see Configure Gmail for Databricks ingestion.

What to know before you start

Topic	Why it matters
Databricks user persona	The workflow depends on your Databricks user persona: Single-user: An admin user creates a Unity Catalog connection and an ingestion pipeline. Multi-user: An admin user creates a connection for non-admin users to create pipelines with.
Authentication method	The connector authenticates with a Google service account that impersonates a single mailbox.
Single mailbox per connection	Each connection ingests one mailbox. To ingest more than one mailbox, create a separate connection and pipeline per mailbox. The mailbox value is stamped as a mailbox column on every row.
Interface	The steps to create a pipeline depend on the interface.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.

Topic	Why it matters
Databricks user persona	The workflow depends on your Databricks user persona: Single-user: An admin user creates a Unity Catalog connection and an ingestion pipeline. Multi-user: An admin user creates a connection for non-admin users to create pipelines with.
Authentication method	The connector authenticates with a Google service account that impersonates a single mailbox.
Single mailbox per connection	Each connection ingests one mailbox. To ingest more than one mailbox, create a separate connection and pipeline per mailbox. The mailbox value is stamped as a mailbox column on every row.
Interface	The steps to create a pipeline depend on the interface.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.

Start ingesting from Gmail

The following table provides an overview of the end-to-end Gmail ingestion flow, based on user type:

User	Steps
Admin	Configure a Google Cloud service account to enable authentication from Databricks. See Configure Gmail for Databricks ingestion. Either: Use Catalog Explorer to create a connection to Gmail so that non-admins can create pipelines. See Create a Gmail connection. Use the data ingestion UI to create a connection and a pipeline at the same time. See Ingest data from Gmail.
Non-admin	Use any supported interface to create a pipeline from an existing connection. See Ingest data from Gmail.

User	Steps
Admin	Configure a Google Cloud service account to enable authentication from Databricks. See Configure Gmail for Databricks ingestion. Either: Use Catalog Explorer to create a connection to Gmail so that non-admins can create pipelines. See Create a Gmail connection. Use the data ingestion UI to create a connection and a pipeline at the same time. See Ingest data from Gmail.
Non-admin	Use any supported interface to create a pipeline from an existing connection. See Ingest data from Gmail.

Additional resources

• Gmail connector FAQs
• Gmail connector limitations
• Gmail connector reference
• Troubleshoot Gmail ingestion