Skip to main content

Migrate to Databricks from other data platforms

You can migrate data warehouses, ETL jobs, and ML and analytics workloads from other data platforms to Databricks. To assess your source system and convert its code automatically, use Genie Migrate. See What is Genie Migrate?.

Migrate your enterprise data warehouse to the Databricks lakehouse​

To move from an enterprise data warehouse to the lakehouse and simplify your data architecture, see Migrate your data warehouse to the Databricks lakehouse. To convert legacy SQL from dialects such as T-SQL, Snowflake, and Oracle as part of this migration, see Convert SQL scripts and ETL pipelines with Genie Migrate.

Migrate ETL jobs to Databricks​

To scope the work of migrating ETL pipelines from specific source systems, see Migrate ETL pipelines to Databricks. To adapt Apache Spark jobs from on-premises or cloud-native implementations, see Adapt your exisiting Apache Spark code for Databricks.

Migrate ML, data science, and analytics workloads​

Because the lakehouse provides optimized access to cloud-based data files through table queries or file paths, you can do ML, data science, and analytics on a single copy of your data. Databricks supports workloads moved from both open source and proprietary tools, and maintains updated versions of many open source libraries used by analysts and data scientists.

Pandas workloads in Jupyter notebooks can be synced and run using Databricks Git folders. Databricks provides native support for pandas in all Databricks Runtime versions, and configures many popular ML and deep learning libraries in Databricks Runtime for Machine Learning. If you sync your local workloads using Git and workspace files in Git folders, you can use the same relative paths for data and custom libraries present in your local environment.

note

By default, Databricks maintains .ipynb extensions for Jupyter notebooks synced with Databricks Git folders, but automatically converts Jupyter notebooks to Databricks notebooks when imported with the UI. Databricks notebooks save with a .py extension, and so can live side by side with Jupyter notebooks in a Git repository.