Welcome to Databricks¶
Here you’ll find a variety of documentation on Databricks and Apache Spark.
Databricks Product Guide¶
This documentation specifically refers to how one should use Databricks as an end user.
Databricks Administration Guide¶
This documentation specifically refers to how one should use Databricks as an administrator.
Databricks API¶
Databricks Release Notes¶
Apache Spark Reference Guide¶
- Spark SQL Reference
- Spark SQL Language Manual
- Alter Database
- Alter Table or View
- Alter Table Partitions
- Analyze Table
- Cache Table
- Clear Cache
- Create Database
- Create Function
- Create Table
- Create View
- Describe Database
- Describe Function
- Describe Table
- Drop Database
- Drop Function
- Drop Table
- Explain
- Insert
- Load Data
- Refresh Table
- Reset
- Select
- Set
- Show Columns
- Show Create Table
- Show Functions
- Show Partitions
- Show Table Properties
- Truncate Table
- Uncache Table
- Use Database
- Vacuum
- Spark SQL Examples
- Transforming Complex Data Types
- Data Skipping Index
- Transactional Writes to Cloud Storage with DBIO
- Handling Bad Files/Records
- Higher Order Functions
- Task Preemption for High Concurrency
- Query Watchdog
- Skew Join Optimization
- Structured Data Access Controls
- User Defined Aggregate Functions - Scala
- User Defined Functions - Python
- User Defined Functions - Scala
- Compatibility with other systems
- Spark SQL Language Manual
- DataFrames and Datasets
- Spark Data Sources
- Cassandra
- Couchbase
- ElasticSearch
- Importing HIVE Tables
- MongoDB
- Neo4j
- Oracle
- Reading Avro Files
- Reading CSV Files
- Reading JSON Files
- Reading LZO Compressed Files
- Reading Parquet Files
- Redis
- Riak Time Series
- Connecting to SQL Databases using JDBC
- Zip Files
- Amazon Redshift
- Amazon S3 with Apache Spark
- Azure storage services
- Structured Streaming
- Spark Streaming (Legacy)
- MLlib and Machine Learning
- SparkR
- SparkR Overview
- SparkR Tutorials
- Latest SparkR Function Reference
- agg
- alias
- arrange
- cache
- cacheTable
- cast
- clearCache
- collect
- columns
- count
- countDistinct
- createDataFrame
- crosstab
- describe
- dim
- distinct
- dropTempTable
- dtypes
- except
- filter
- first
- groupBy
- head
- intersect
- join
- last
- limit
- matching operations
- max
- mean
- merge
- min
- ncol
- nrow
- persist
- printSchema
- rbind
- read.df
- registerTempTable
- sample
- saveAsParquetFile
- select
- selectExpr
- sql
- subset
- sum
- sumDistinct
- tableToDF
- tables
- take
- unionAll
- withColumn
- withColumnRenamed
- 1.6 SparkR Function Reference
- GraphX and GraphFrames
- Apache Spark FAQ
- Educational Materials and Tutorials
- Deep Learning