Google Drive connector

The managed Google Drive connector in Lakeflow Connect allows you to ingest files from Google Drive into Databricks. Ingest unstructured files as binary data, parse structured formats (CSV, JSON, XML, EXCEL, and more) into Delta tables, or capture file metadata without loading file contents.

For the standard Google Drive connector that uses Spark reader APIs (read_files, spark.read, Auto Loader), see Ingest files from Google Drive.

What to know before you start

Topic	Why it matters
Connection privileges	To create a connection, you need the CREATE CONNECTION privilege on the metastore. To create a pipeline from an existing connection, you need the USE CONNECTION privilege or ALL PRIVILEGES on the connection.
Authentication method	The steps to create a connection depend on the authentication method you select.
Interface	The steps to create a pipeline depend on the interface.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.
Common patterns	Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability.

Topic	Why it matters
Connection privileges	To create a connection, you need the CREATE CONNECTION privilege on the metastore. To create a pipeline from an existing connection, you need the USE CONNECTION privilege or ALL PRIVILEGES on the connection.
Authentication method	The steps to create a connection depend on the authentication method you select.
Interface	The steps to create a pipeline depend on the interface.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.
Common patterns	Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability.

Start ingesting from Google Drive

The following table has an overview of the end-to-end Google Drive ingestion flow, based on connection privileges:

Required privilege	Steps
CREATE CONNECTION on the metastore	Configure Google Drive authentication. See Overview of Google Drive ingestion setup. Create a connection to Google Drive. See Create a Google Drive connection. Create an ingestion pipeline from the connection. See Ingest data from Google Drive.
USE CONNECTION or ALL PRIVILEGES on an existing connection	Use any supported interface to create a pipeline from an existing connection. See Ingest data from Google Drive.

Required privilege	Steps
CREATE CONNECTION on the metastore	Configure Google Drive authentication. See Overview of Google Drive ingestion setup. Create a connection to Google Drive. See Create a Google Drive connection. Create an ingestion pipeline from the connection. See Ingest data from Google Drive.
USE CONNECTION or ALL PRIVILEGES on an existing connection	Use any supported interface to create a pipeline from an existing connection. See Ingest data from Google Drive.

Feature availability

Feature	Availability
UI-based pipeline authoring	Supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Supported
Incremental ingestion	Supported
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported
Schema evolution	Supported Configurable via schema_evolution_mode. See Google Drive connector reference.
API-based row filtering	Supported

Feature	Availability
UI-based pipeline authoring	Supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Supported
Incremental ingestion	Supported
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported
Schema evolution	Supported Configurable via schema_evolution_mode. See Google Drive connector reference.
API-based row filtering	Supported

Authentication methods

Authentication method	Availability
OAuth U2M: Databricks-managed	Supported when enabled in the workspace No Google Cloud project or app registration required. If this option isn't listed, use OAuth U2M: Custom-managed. See Configure OAuth U2M: Databricks-managed for Google Drive ingestion.
OAuth U2M: Custom-managed	Supported Uses your own Google Cloud app. See Configure OAuth U2M: Custom-managed for Google Drive ingestion.
OAuth service account	Supported Uses a Google service account key. See Configure an OAuth service account for Google Drive ingestion.

Authentication method	Availability
OAuth U2M: Databricks-managed	Supported when enabled in the workspace No Google Cloud project or app registration required. If this option isn't listed, use OAuth U2M: Custom-managed. See Configure OAuth U2M: Databricks-managed for Google Drive ingestion.
OAuth U2M: Custom-managed	Supported Uses your own Google Cloud app. See Configure OAuth U2M: Custom-managed for Google Drive ingestion.
OAuth service account	Supported Uses a Google service account key. See Configure an OAuth service account for Google Drive ingestion.