OneDrive connector

Beta

This feature is in Beta. Workspace admins can control access to this feature from the Previews page by opting into the Lakeflow Connect for OneDrive For Business preview. See Manage Databricks previews.

The managed OneDrive connector in Lakeflow Connect allows you to ingest files from a user's OneDrive for Business My files drive into Databricks. Ingest unstructured files as binary data or managed FILE values, parse structured formats (CSV, JSON, XML, EXCEL, and more) into Delta tables, or capture file metadata without loading file contents.

To ingest files from SharePoint sites, team sites, or shared document libraries, use the managed SharePoint connector.

To read from OneDrive using SQL and Spark APIs (read_files, spark.read, Auto Loader), see Ingest files from OneDrive.

What to know before you start

Topic	Why it matters
Connection privileges	To create a connection, you need the CREATE CONNECTION privilege on the metastore. To create a pipeline from an existing connection, you need the USE CONNECTION privilege or ALL PRIVILEGES on the connection.
Authentication method	The steps to create a connection depend on the authentication method you select.
Interface	The steps to create a pipeline depend on the interface. UI-based pipeline authoring is not supported.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.
Common patterns	Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability.

Topic	Why it matters
Connection privileges	To create a connection, you need the CREATE CONNECTION privilege on the metastore. To create a pipeline from an existing connection, you need the USE CONNECTION privilege or ALL PRIVILEGES on the connection.
Authentication method	The steps to create a connection depend on the authentication method you select.
Interface	The steps to create a pipeline depend on the interface. UI-based pipeline authoring is not supported.
Ingestion frequency	The pipeline schedule depends on your latency and cost requirements.
Common patterns	Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability.

Start ingesting from OneDrive

The following table has an overview of the end-to-end OneDrive ingestion flow, based on connection privileges:

Required privilege	Steps
CREATE CONNECTION on the metastore	(OAuth U2M: Custom-managed only) Configure OneDrive authentication with your own Microsoft Entra ID app. See Configure OAuth U2M: Custom-managed for Microsoft OneDrive ingestion. Create a connection to OneDrive. See Create a OneDrive connection. Create an ingestion pipeline from the connection. See Ingest data from OneDrive.
USE CONNECTION or ALL PRIVILEGES on an existing connection	Use any supported interface to create a pipeline from an existing connection. See Ingest data from OneDrive.

Required privilege	Steps
CREATE CONNECTION on the metastore	(OAuth U2M: Custom-managed only) Configure OneDrive authentication with your own Microsoft Entra ID app. See Configure OAuth U2M: Custom-managed for Microsoft OneDrive ingestion. Create a connection to OneDrive. See Create a OneDrive connection. Create an ingestion pipeline from the connection. See Ingest data from OneDrive.
USE CONNECTION or ALL PRIVILEGES on an existing connection	Use any supported interface to create a pipeline from an existing connection. See Ingest data from OneDrive.

Feature availability

Feature	Availability
UI-based pipeline authoring	Not supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Not supported
Incremental ingestion	Supported
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported
Schema evolution	Supported Configurable via schema_evolution_mode. See Microsoft OneDrive connector reference.
API-based row filtering	Supported

Feature	Availability
UI-based pipeline authoring	Not supported
API-based pipeline authoring	Supported
Declarative Automation Bundles	Not supported
Incremental ingestion	Supported
Unity Catalog governance	Supported
Orchestration using Databricks Workflows	Supported
SCD type 2	Not supported
Schema evolution	Supported Configurable via schema_evolution_mode. See Microsoft OneDrive connector reference.
API-based row filtering	Supported

Supported file formats

The connector supports both unstructured and structured file ingestion:

• Unstructured: managed file references (FILE, Beta) and BINARYFILE.
• Structured and semi-structured: CSV, JSON, XML, EXCEL, PARQUET, AVRO, and ORC.

For destination table schemas, see Ingested data format. For connector-specific limitations, see Microsoft OneDrive connector limitations.

Authentication methods

Authentication method	Availability
OAuth U2M: Databricks-managed	Supported when enabled in the workspace No Microsoft Entra ID app registration required. If this option isn't listed, use OAuth U2M: Custom-managed. See Configure OAuth U2M: Databricks-managed for Microsoft OneDrive ingestion.
OAuth U2M: Custom-managed	Supported Uses your own Microsoft Entra ID app. See Configure OAuth U2M: Custom-managed for Microsoft OneDrive ingestion.
OAuth M2M	Not supported
OAuth (manual refresh token)	Not supported

Authentication method	Availability
OAuth U2M: Databricks-managed	Supported when enabled in the workspace No Microsoft Entra ID app registration required. If this option isn't listed, use OAuth U2M: Custom-managed. See Configure OAuth U2M: Databricks-managed for Microsoft OneDrive ingestion.
OAuth U2M: Custom-managed	Supported Uses your own Microsoft Entra ID app. See Configure OAuth U2M: Custom-managed for Microsoft OneDrive ingestion.
OAuth M2M	Not supported
OAuth (manual refresh token)	Not supported

With both U2M methods, the connection authenticates as the signed-in Microsoft account, so a pipeline can ingest only the files that account can access.