Managed file source connectors
Databricks Lakeflow Connect provides fully-managed connectors for ingesting unstructured files from enterprise file storage services. Each connector handles source-specific authentication, incremental reads, and automated retries.
Supported connectors
-
- Google Drive (managed)
- Ingest files from Google Drive into the Lakehouse using managed file source ingestion.
-
- Microsoft SharePoint (managed)
- Ingest files from Microsoft SharePoint sites using managed file source ingestion.
For standard connectors using Spark and SQL APIs, see Ingest files from Google Drive and Ingest files from SharePoint.
Connector components
A file source connector has the following components:
Component | Description |
|---|---|
Connection | A Unity Catalog securable object that stores authentication details for the file storage service. |
Ingestion pipeline | A pipeline that copies files from the source into the destination tables. The ingestion pipeline runs on serverless compute. |
Destination tables | The tables where the ingestion pipeline writes the data. These are streaming tables, which are Delta tables with extra support for incremental data processing. |