Skip to main content

Migrate Community Edition workspaces to Free Edition

Preview

This feature is in Public Preview.

With the release of Databricks Free Edition, Community Edition (CE) will soon be retired. Community Edition workspace owners should use the workspace migration tool to migrate to Free Edition as soon as possible.

Migrate your workspace​

When you use the migration tool, Databricks creates a new Free Edition workspace linked to your existing login. Then, your notebooks and data is migrated over to your new Free Edition workspace.

You can only run the migration once, so be sure to clean up unused notebooks and tables before starting. For a list of limitations on data migration, see Limitations.

Step 1: Review your current workspace​

Before you use the migration tool, review your current workspace and identify key notebooks and data that you want to migrate. Move any tables and notebooks you want to migrate out of the tmp and Filestore directories.

Step 2: Start the migration process​

note

You must be the workspace owner to use the migration tool.

In your CE workspace:

  1. Click Move to Free Edition in the banner at the top of your Community Edition workspace.
  2. A dialog box explains the migration process.
  3. Click Migrate. Your CE workspace locks, and you are logged out. Avoid logging back in until the migration finishes.
  4. When the migration finishes (typically under 2 hours), you receive an email with a login link.

Step 3: Test your Free Edition workspace​

When you log in, you now see both your Community Edition and Free Edition workspaces. Open both workspaces to confirm that everything migrated to the Free Edition workspace.

  1. Open Workspace > Users > [your-email] and confirm that all your notebooks are present.
  2. Open Catalog > Default and confirm that all your tables are present.
  3. Run key notebooks, cell by cell. If you see errors, see Common issues.
note

You might notice that some tables were split into smaller parts. This is expected. See My table was split into smaller pieces for how to re-merge them.

Step 4: Download any missing assets from your CE workspace​

If any assets did not transfer, you can download them manually from your Community Edition workspace and re-upload them to your Free Edition workspace.

After 7 days, your Community Edition workspace is permanently deleted. Verify that all content has transferred to your new workspace before that date.

Limitations​

The migration might not transfer all resources. Review the following limitations before starting migration:

  • Free Edition has a hard limit of 500 tables. If you have more than 500 tables in your Community Edition workspace, not all tables will be migrated.
  • For tables or parts of tables that have been archived, the migration attempts to migrate them but they might show up as errors in the query history. Auto archiving happens for data that has not been accessed for 3+ months.
  • Migrating CSV files is best effort. There might be cases where the delimiter is used incorrectly and columns become grouped together or the table fails to create.
  • Only supported files and data types migrate. If you need any of these assets, manually download them from your workspace before starting migration. The following file and data types aren't migrated:
    • .zip files, .mp4 videos, system logs
    • XML files
    • Hidden files or unsupported formats
    • MLflow experiments
    • Anything under tmp or Filestore
    • Files stored outside dbfs:/

Troubleshoot common issues​

I can't choose a cluster size or instance type​

Free Edition uses serverless compute, so you can't customize the cluster size or instance type. Serverless compute is automatically scaled based on your workload requirements.

Solution

Run a cell to automatically start serverless compute, or select a compute resource from the drop-down menu. If you see delays or errors, keep workloads lightweight and retry after a few minutes.

One of my files didn't transfer​

This happens if the file was either a non-standard format (for example, .mp4, .zip), hidden, or unsupported.

Solution

Download the file from your Community Edition workspace within 7 days and manually upload it to your Free Edition workspace.

My table was split into smaller pieces​

Some CE tables were backed by large files that were stored in pieces. During migration, Databricks copies each part as its own table.

Solution

Recombine using UNION ALL:

SQL
CREATE OR REPLACE TABLE my_full_table AS
SELECT * FROM my_table_part1
UNION ALL
SELECT * FROM my_table_part2
UNION ALL
SELECT * FROM my_table_part3;

My table transferred, but my notebook can't find it​

Your notebook is likely referencing a table by name, but the table's name changed during migration.

In Free Edition, all tables are created in workspace.default.<table_name>. The table name is either:

  • The directory containing the file. So, /my_table/my_table_data_file.parquet is called my_table.
  • If it's at the base of DBFS root, the file name is used. So, /my_table_data_file.parquet is called my_table_data_file.
Solution

  1. Find the underlying table in Catalog.

  2. Copy the table's file name.

  3. Return to the notebook.

  4. Instruct the Databricks Assistant to replace all instances of the old table location with the new table location:

    Text
    Replace all references to 'old_table_name' with 'workspace.default.new_table_name' in this notebook

My notebook code doesn't work​

This usually falls into two categories:

Case 1: You're using RDDs

RDDs are a legacy Spark abstraction and are not supported in Free Edition. Replace them with DataFrames.

Solution

Ask the Databricks Assistant to help convert your RDD code:

Text
Convert all RDD operations in this notebook to DataFrame operations

Case 2: You're using Scala or R

Serverless compute supports Python and SQL only. If your notebook uses Scala or R, you need to translate it to Python.

Solution

Ask the Databricks Assistant to translate your code:

Text
Convert this Scala/R code to Python using PySpark DataFrames

dbutils.fs.mount commands fail​

New Databricks workspaces do not support legacy DBFS mounts.

Solution

Use Unity Catalog external locations and volumes instead. For storing datasets or shared files, create a volume:

SQL
CREATE VOLUME IF NOT EXISTS workspace.default.my_volume;

Then access files using:

Python
# Write data
df.write.mode("overwrite").option("path", "/Volumes/workspace/default/my_volume/my_data").saveAsTable("my_table")

# Read data
df = spark.read.table("my_table")

I can't read or write files in /dbfs/​

Free Edition restricts direct access to the DBFS root for security.

Solution

Use a Unity Catalog volume for storing datasets or shared files:

Python
# Create a volume (run once)
spark.sql("CREATE VOLUME IF NOT EXISTS workspace.default.my_data_volume")

# Write files
dbutils.fs.cp("file:/local/path/data.csv", "/Volumes/workspace/default/my_data_volume/")

# Read files
df = spark.read.csv("/Volumes/workspace/default/my_data_volume/data.csv", header=True, inferSchema=True)
Last updated on