Skip to main content

Manage dependencies for a Databricks app

Each Databricks app can include dependencies for Python, Node.js, or both. You define these dependencies in language-specific files:

  • Use a requirements.txt file to specify additional Python packages.
  • Use a package.json file to specify Node.js packages.

Each app also comes with a pre-installed set of Python libraries. See Pre-installed Python libraries.

Define Python dependencies​

To define additional Python libraries, use a requirements.txt file. If any listed packages match pre-installed ones, the versions in your file override the defaults.

For example:

# Override default version of dash
dash==2.10.0

# Add additional libraries not pre-installed
requests==2.31.0
numpy==1.24.3

# Specify a compatible version range
scikit-learn>=1.2.0,<1.3.0

Avoid version conflicts​

Keep the following in mind when you define dependencies:

  • Overriding pre-installed packages may cause compatibility issues if your specified version differs significantly from the pre-installed one.
  • Always test your app to ensure that package version changes don't introduce errors.
  • Pinning explicit versions in requirements.txt helps maintain consistent app behavior across deployments.

Define Node.js dependencies​

To define Node.js libraries, include a package.json file in the root of your app. During deployment, Databricks detects this file and runs npm install to install all dependencies listed in it.

For example, a package.json file for a React app using Vite might look like this:

JSON
{
  "name": "react-fastapi-app",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "npm run build:frontend",
    "build:frontend": "vite build frontend"
  },
  "dependencies": {
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "typescript": "^5.0.0",
    "vite": "^5.0.0",
    "@vitejs/plugin-react": "^4.2.0",
    "@types/react": "^18.2.0",
    "@types/react-dom": "^18.2.0"
  }
}
note

List all packages required for npm run build under dependencies, not devDependencies. If you set NODE_ENV=production in your environment variables, the deployment process skips installing devDependencies.

Dependency installation and management​

Libraries defined in requirements.txt and package.json are installed directly on the container running on your dedicated compute. You're responsible for managing and patching these dependencies.

You can specify libraries from multiple sources in your dependency files:

  • Libraries downloaded from public repositories like PyPI and npm
  • Private repositories that authenticate using credentials stored in Databricks secrets
  • Libraries stored in your /Volumes/ directory (for example, /Volumes/<catalog>/<schema>/<volume>/<path>)

To enhance security when accessing external package repositories, use serverless egress controls to restrict access to public repositories and configure private networking. See Configure networking for Databricks Apps.

Last updated on