Databricks Runtime 6.5 (unsupported)

Databricks released this image in April 2020.

The following release notes provide information about Databricks Runtime 6.5, powered by Apache Spark.

New features

Operation metrics for all writes, updates, and deletes on a Delta table now shown in table history

You can now see operation metrics (for example, number of files and rows changed) for all writes, updates, and deletes on a Delta table in the table’s history. The generation of these metrics is enabled by default. You can see the history by running:

DESCRIBE HISTORY deltaTableName

Rate-limit the data processed in Delta Lake streaming micro-batches

When you process data from Delta Lake using Structured Streaming, you can now set the option maxBytesPerTrigger to rate-limit how much data gets processed in each micro-batch. This sets a “soft max,” meaning that a batch processes approximately this amount of data and may process more than the limit.

If you use Trigger.Once for your streaming, this option is ignored. In addition, if you use this option in conjunction with maxFilesPerTrigger, the micro-batch processes data until either the maxFilesPerTrigger or maxBytesPerTrigger limit is reached.

Improvements

Snowflake connector updated to 2.5.9

The updated connector promotes the Snowflake JDBC driver to 3.12.0.

Java upgraded to 1.8.0_242

This version introduces new checks on trust anchor certificates to ensure that trust anchors are CA certificates and contain proper extensions. Trust anchor certificates must include a Basic Constraints extension with the ca field set to true. In addition, if they include a Key Usage extension, the keyCertSign bit must be set.

If you use self-signed trust anchor certificates, you may hit the following exceptions introduced by the new checks:

TrustAnchor with subject "..." is not a CA certificate

TrustAnchor with subject "..." does not have keyCertSign bit set in KeyUsage extension

To keep using self-signed trust anchor certificates, you have two options to address the above exceptions:

Set Java system property jdk.security.allowNonCaAnchor introduced with Java 1.9.0_242 to true to restore the previous behavior. You can use spark.driver.extraJavaOptions and spark.executor.extraJavaOptions to set this system property to Spark driver and executors, respectively.
Include a Basic Constraints extension with the ca field set to true to your trust anchor certificates. If your trust anchor certificates include a Key Usage extension, set keyCertSign bit.

If you use Java Keytool to generate public/private key pairs, set -ext BasicConstraints=ca:true when generating trust anchor certificates. If your trust anchor certificates include a Key Usage extension, set keyCertSign bit -ext KeyUsage=...,keyCertSign when generating trust anchor certificates.

S3 Select connector: syntax fix for DECIMAL and CASE WHEN

DECIMAL type conversion ignores precision and scale when converting to the DECIMAL type in Amazon S3 Select because of removed support for DECIMAL(precision, scale) in Amazon S3 Select. All internal CASE WHEN references have been replaced with COALESCE and NULLIF functions that are officially supported by Amazon S3 Select.

Kinesis streaming source launches more efficiently sized tasks

The Kinesis streaming source now automatically coalesces prefetched data blocks to reduce the number of tasks generated to process the blocks.

Upgraded R libraries

Upgraded several installed R libraries. See Installed R Libraries.

Deprecations

The boto package is deprecated and will be removed in an upcoming major release of Databricks Runtime. Databricks recommends that you use boto3 instead.
The following R packages will be removed in an upcoming major release of Databricks Runtime:
- abind
- bitops
- car
- carData
- doMC
- gbm
- h2o
- littler
- lme4
- mapproj
- maps
- maptools
- MatrixModels
- minqa
- mvtnorm
- nloptr
- openxlsx
- pbkrtest
- pkgKitten
- quantreg
- R.methodsS3
- R.oo
- R.utils
- RcppEigen
- RCurl
- rio
- sp
- SparseM
- statmod
- zip

Databricks recommends using the following code to install and load R packages that are not installed by Databricks Runtime:

if (!require("package_name")) {
  install.packages("package_name")
  library("package_name")
}

Apache Spark

Databricks Runtime 6.5 includes Apache Spark 2.4.5. This release includes all Spark fixes and improvements included in Databricks Runtime 6.4 (unsupported), as well as the following additional bug fixes and improvements made to Spark:

[SPARK-27099] [SQL] Add ‘xxhash64’ for hashing arbitrary columns to Long
[SPARK-26151] [SQL] Return partial results for bad CSV records
[SPARK-30927] [SS] StreamingQueryManager should avoid keeping reference to terminated StreamingQuery
[SPARK-22590] Copy sparkContext.localproperties to child thread in BroadcastExchangeExec.executionContext
[SPARK-30671] emptyDataFrame should use a LocalRelation
[SPARK-30556] [SQL] Reset the status changed in SQLExecution withThreadLocalCaptured
[SPARK-30763] [SQL] Fix java.lang.IndexOutOfBoundsException No group 1 for regexp_extract
[SPARK-30806] [SQL] Evaluate once per group in UnboundedWindowFunctionFrame
[SPARK-30731] Update deprecated Mkdocs option
[SPARK-30857] [SQL] Fix truncations of timestamps before the epoch to hours and days
[SPARK-30798] Scope Session.active in QueryExecution
[SPARK-30793] [SQL] Fix truncations of timestamps before the epoch to minutes and seconds
[SPARK-30826] [SQL] Respect reference case in StringStartsWith pushed down to parquet
[SPARK-30797] [SQL] Set tradition user/group/other permission to ACL entries when setting up ACLs in truncate table
[SPARK-30065] [SQL] DataFrameNaFunctions.drop should handle duplicate columns
[SPARK-29890] [SQL] DataFrameNaFunctions.fill should handle duplicate columns
[SPARK-30310] [CORE] Resolve missing match case in SparkUncaughtExceptionHandler and added tests
[SPARK-30512] Added a dedicated boss event loop group

Maintenance updates

See Databricks Runtime 6.5 maintenance updates.

System environment

Operating System: Ubuntu 16.04.6 LTS
Java: 1.8.0_252
Scala: 2.11.12
Python: 3.7.3
R: R version 3.6.3 (2020-02-29)
Delta Lake: 0.5.0

Note

Although Scala 2.12 is available as an experimental feature in Apache Spark 2.4, it is not supported in Databricks Runtime 6.5.

In this section:

Installed Python libraries
Installed R libraries
Installed Java and Scala libraries (Scala 2.11 cluster version)

Installed Python libraries

Library	Version	Library	Version	Library	Version
asn1crypto	0.24.0	backcall	0.1.0	boto	2.49.0
boto3	1.9.162	botocore	1.12.163	certifi	2019.3.9
cffi	1.12.2	chardet	3.0.4	cryptography	2.6.1
cycler	0.10.0	Cython	0.29.6	decorator	4.4.0
docutils	0.14	idna	2.8	ipykernel	5.1.0
ipython	7.4.0	ipython-genutils	0.2.0	jedi	0.13.3
jmespath	0.9.4	jupyter-client	5.2.4	jupyter-core	4.4.0
kiwisolver	1.1.0	matplotlib	3.0.3	numpy	1.16.2
pandas	0.24.2	parso	0.3.4	patsy	0.5.1
pexpect	4.6.0	pickleshare	0.7.5	pip	19.0.3
prompt-toolkit	2.0.9	psycopg2	2.7.6.1	ptyprocess	0.6.0
pyarrow	0.13.0	pycparser	2.19	pycurl	7.43.0
Pygments	2.3.1	PyGObject	3.20.0	pyOpenSSL	19.0.0
pyparsing	2.4.6	PySocks	1.6.8	python-apt	1.1.0.b1+ubuntu0.16.04.8
python-dateutil	2.8.0	pytz	2018.9	pyzmq	18.0.0
requests	2.21.0	s3transfer	0.2.1	scikit-learn	0.20.3
scipy	1.2.1	seaborn	0.9.0	setuptools	40.8.0
six	1.12.0	ssh-import-id	5.5	statsmodels	0.9.0
tornado	6.0.2	traitlets	4.3.2	unattended-upgrades	0.1
urllib3	1.24.1	virtualenv	16.4.1	wcwidth	0.1.7
wheel	0.33.1

Installed R libraries

R libraries are installed from Microsoft CRAN snapshot on 2020-02-26.

Library	Version	Library	Version	Library	Version
abind	1.4-5	askpass	1.1	assertthat	0.2.1
backports	1.1.5	base	3.6.3	base64enc	0.1-3
BH	1.72.0-3	bit	1.1-15.2	bit64	0.9-7
bitops	1.0-6	blob	1.2.1	boot	1.3-23
brew	1.0-6	callr	3.4.2	car	3.0-6
carData	3.0-3	caret	6.0-85	cellranger	1.1.0
chron	2.3-55	class	7.3-15	cli	2.0.1
clipr	0.7.0	clisymbols	1.2.0	cluster	2.1.0
codetools	0.2-16	colorspace	1.4-1	commonmark	1.7
compiler	3.6.3	config	0.3	covr	3.4.0
crayon	1.3.4	crosstalk	1.0.0	curl	4.3
data.table	1.12.8	datasets	3.6.3	DBI	1.1.0
dbplyr	1.4.2	desc	1.2.0	devtools	2.2.2
digest	0.6.25	doMC	1.3.6	dplyr	0.8.4
DT	0.12	ellipsis	0.3.0	evaluate	0.14
fansi	0.4.1	farver	2.0.3	fastmap	1.0.1
forcats	0.4.0	foreach	1.4.8	foreign	0.8-75
forge	0.2.0	fs	1.3.1	gbm	2.1.5
generics	0.0.2	ggplot2	3.2.1	gh	1.1.0
git2r	0.26.1	glmnet	3.0-2	glue	1.3.1
gower	0.2.1	graphics	3.6.3	grDevices	3.6.3
grid	3.6.3	gridExtra	2.3	gsubfn	0.7
gtable	0.3.0	h2o	3.28.0.2	haven	2.2.0
hms	0.5.3	htmltools	0.4.0	htmlwidgets	1.5.1
httpuv	1.5.2	httr	1.4.1	hwriter	1.3.2
hwriterPlus	1.0-3	ini	0.3.1	ipred	0.9-9
iterators	1.0.12	jsonlite	1.6.1	KernSmooth	2.23-16
labeling	0.3	later	1.0.0	lattice	0.20-40
lava	1.6.6	lazyeval	0.2.2	lifecycle	0.1.0
littler	0.3.9	lme4	1.1-21	lubridate	1.7.4
magrittr	1.5	mapproj	1.2.7	maps	3.3.0
maptools	0.9-9	MASS	7.3-51.5	Matrix	1.2-18
MatrixModels	0.4-1	memoise	1.1.0	methods	3.6.3
mgcv	1.8-31	mime	0.9	minqa	1.2.4
ModelMetrics	1.2.2.1	munsell	0.5.0	mvtnorm	1.1-0
nlme	3.1-144	nloptr	1.2.1	nnet	7.3-13
numDeriv	2016.8-1.1	openssl	1.4.1	openxlsx	4.1.4
parallel	3.6.3	pbkrtest	0.4-8.6	pillar	1.4.3
pkgbuild	1.0.6	pkgconfig	2.0.3	pkgKitten	0.1.5
pkgload	1.0.2	plogr	0.2.0	plyr	1.8.5
praise	1.0.0	prettyunits	1.1.1	pROC	1.16.1
processx	3.4.2	prodlim	2019.11.13	progress	1.2.2
promises	1.1.0	proto	1.0.0	ps	1.3.2
purrr	0.3.3	quantreg	5.54	R.methodsS3	1.8.0
R.oo	1.23.0	R.utils	2.9.2	r2d3	0.2.3
R6	2.4.1	randomForest	4.6-14	rappdirs	0.3.1
rcmdcheck	1.3.3	RColorBrewer	1.1-2	Rcpp	1.0.3
RcppEigen	0.3.3.7.0	RCurl	1.98-1.1	readr	1.3.1
readxl	1.3.1	recipes	0.1.9	rematch	1.0.1
remotes	2.1.1	reshape2	1.4.3	rex	1.1.2
rio	0.5.16	rlang	0.4.4	RODBC	1.3-16
roxygen2	7.0.2	rpart	4.1-15	rprojroot	1.3-2
Rserve	1.8-6	RSQLite	2.2.0	rstudioapi	0.11
rversions	2.0.1	scales	1.1.0	sessioninfo	1.1.1
shape	1.4.4	shiny	1.4.0	sourcetools	0.1.7
sp	1.4-0	sparklyr	1.1.0	SparkR	2.4.5
SparseM	1.78	spatial	7.3-11	splines	3.6.3
sqldf	0.4-11	SQUAREM	2020.2	statmod	1.4.34
stats	3.6.3	stats4	3.6.3	stringi	1.4.6
stringr	1.4.0	survival	2.44-1.1	sys	3.3
tcltk	3.6.3	TeachingDemos	2.10	testthat	2.3.1
tibble	2.1.3	tidyr	1.0.2	tidyselect	1.0.0
timeDate	3043.102	tools	3.6.3	usethis	1.5.1
utf8	1.1.4	utils	3.6.3	vctrs	0.2.3
viridisLite	0.3.0	whisker	0.4	withr	2.1.2
xml2	1.2.2	xopen	1.0.0	xtable	1.8-4
yaml	2.2.1	zip	2.0.4

Installed Java and Scala libraries (Scala 2.11 cluster version)

Group ID	Artifact ID	Version
antlr	antlr	2.7.7
com.amazonaws	amazon-kinesis-client	1.8.10
com.amazonaws	aws-java-sdk-autoscaling	1.11.595
com.amazonaws	aws-java-sdk-cloudformation	1.11.595
com.amazonaws	aws-java-sdk-cloudfront	1.11.595
com.amazonaws	aws-java-sdk-cloudhsm	1.11.595
com.amazonaws	aws-java-sdk-cloudsearch	1.11.595
com.amazonaws	aws-java-sdk-cloudtrail	1.11.595
com.amazonaws	aws-java-sdk-cloudwatch	1.11.595
com.amazonaws	aws-java-sdk-cloudwatchmetrics	1.11.595
com.amazonaws	aws-java-sdk-codedeploy	1.11.595
com.amazonaws	aws-java-sdk-cognitoidentity	1.11.595
com.amazonaws	aws-java-sdk-cognitosync	1.11.595
com.amazonaws	aws-java-sdk-config	1.11.595
com.amazonaws	aws-java-sdk-core	1.11.595
com.amazonaws	aws-java-sdk-datapipeline	1.11.595
com.amazonaws	aws-java-sdk-directconnect	1.11.595
com.amazonaws	aws-java-sdk-directory	1.11.595
com.amazonaws	aws-java-sdk-dynamodb	1.11.595
com.amazonaws	aws-java-sdk-ec2	1.11.595
com.amazonaws	aws-java-sdk-ecs	1.11.595
com.amazonaws	aws-java-sdk-efs	1.11.595
com.amazonaws	aws-java-sdk-elasticache	1.11.595
com.amazonaws	aws-java-sdk-elasticbeanstalk	1.11.595
com.amazonaws	aws-java-sdk-elasticloadbalancing	1.11.595
com.amazonaws	aws-java-sdk-elastictranscoder	1.11.595
com.amazonaws	aws-java-sdk-emr	1.11.595
com.amazonaws	aws-java-sdk-glacier	1.11.595
com.amazonaws	aws-java-sdk-glue	1.11.595
com.amazonaws	aws-java-sdk-iam	1.11.595
com.amazonaws	aws-java-sdk-importexport	1.11.595
com.amazonaws	aws-java-sdk-kinesis	1.11.595
com.amazonaws	aws-java-sdk-kms	1.11.595
com.amazonaws	aws-java-sdk-lambda	1.11.595
com.amazonaws	aws-java-sdk-logs	1.11.595
com.amazonaws	aws-java-sdk-machinelearning	1.11.595
com.amazonaws	aws-java-sdk-opsworks	1.11.595
com.amazonaws	aws-java-sdk-rds	1.11.595
com.amazonaws	aws-java-sdk-redshift	1.11.595
com.amazonaws	aws-java-sdk-route53	1.11.595
com.amazonaws	aws-java-sdk-s3	1.11.595
com.amazonaws	aws-java-sdk-ses	1.11.595
com.amazonaws	aws-java-sdk-simpledb	1.11.595
com.amazonaws	aws-java-sdk-simpleworkflow	1.11.595
com.amazonaws	aws-java-sdk-sns	1.11.595
com.amazonaws	aws-java-sdk-sqs	1.11.595
com.amazonaws	aws-java-sdk-ssm	1.11.595
com.amazonaws	aws-java-sdk-storagegateway	1.11.595
com.amazonaws	aws-java-sdk-sts	1.11.595
com.amazonaws	aws-java-sdk-support	1.11.595
com.amazonaws	aws-java-sdk-swf-libraries	1.11.22
com.amazonaws	aws-java-sdk-workspaces	1.11.595
com.amazonaws	jmespath-java	1.11.595
com.carrotsearch	hppc	0.7.2
com.chuusai	shapeless_2.11	2.3.2
com.clearspring.analytics	stream	2.7.0
com.databricks	Rserve	1.8-3
com.databricks	dbml-local_2.11	0.5.0-db8-spark2.4
com.databricks	dbml-local_2.11-tests	0.5.0-db8-spark2.4
com.databricks	jets3t	0.7.1-0
com.databricks.scalapb	compilerplugin_2.11	0.4.15-9
com.databricks.scalapb	scalapb-runtime_2.11	0.4.15-9
com.esotericsoftware	kryo-shaded	4.0.2
com.esotericsoftware	minlog	1.3.0
com.fasterxml	classmate	1.0.0
com.fasterxml.jackson.core	jackson-annotations	2.6.7
com.fasterxml.jackson.core	jackson-core	2.6.7
com.fasterxml.jackson.core	jackson-databind	2.6.7.3
com.fasterxml.jackson.dataformat	jackson-dataformat-cbor	2.6.7
com.fasterxml.jackson.datatype	jackson-datatype-joda	2.6.7
com.fasterxml.jackson.module	jackson-module-paranamer	2.6.7
com.fasterxml.jackson.module	jackson-module-scala_2.11	2.6.7.1
com.github.fommil	jniloader	1.1
com.github.fommil.netlib	core	1.1.2
com.github.fommil.netlib	native_ref-java	1.1
com.github.fommil.netlib	native_ref-java-natives	1.1
com.github.fommil.netlib	native_system-java	1.1
com.github.fommil.netlib	native_system-java-natives	1.1
com.github.fommil.netlib	netlib-native_ref-linux-x86_64-natives	1.1
com.github.fommil.netlib	netlib-native_system-linux-x86_64-natives	1.1
com.github.luben	zstd-jni	1.3.2-2
com.github.rwl	jtransforms	2.4.0
com.google.code.findbugs	jsr305	2.0.1
com.google.code.gson	gson	2.2.4
com.google.guava	guava	15.0
com.google.protobuf	protobuf-java	2.6.1
com.googlecode.javaewah	JavaEWAH	0.3.2
com.h2database	h2	1.3.174
com.jcraft	jsch	0.1.50
com.jolbox	bonecp	0.8.0.RELEASE
com.microsoft.azure	azure-data-lake-store-sdk	2.2.8
com.microsoft.azure	azure-storage	7.0.0
com.microsoft.sqlserver	mssql-jdbc	6.2.2.jre8
com.ning	compress-lzf	1.0.3
com.sun.mail	javax.mail	1.5.2
com.thoughtworks.paranamer	paranamer	2.8
com.trueaccord.lenses	lenses_2.11	0.3
com.twitter	chill-java	0.9.3
com.twitter	chill_2.11	0.9.3
com.twitter	parquet-hadoop-bundle	1.6.0
com.twitter	util-app_2.11	6.23.0
com.twitter	util-core_2.11	6.23.0
com.twitter	util-jvm_2.11	6.23.0
com.typesafe	config	1.2.1
com.typesafe.scala-logging	scala-logging-api_2.11	2.1.2
com.typesafe.scala-logging	scala-logging-slf4j_2.11	2.1.2
com.univocity	univocity-parsers	2.7.3
com.vlkan	flatbuffers	1.2.0-3f79e055
com.zaxxer	HikariCP	3.1.0
commons-beanutils	commons-beanutils	1.9.4
commons-cli	commons-cli	1.2
commons-codec	commons-codec	1.10
commons-collections	commons-collections	3.2.2
commons-configuration	commons-configuration	1.6
commons-dbcp	commons-dbcp	1.4
commons-digester	commons-digester	1.8
commons-fileupload	commons-fileupload	1.3.3
commons-httpclient	commons-httpclient	3.1
commons-io	commons-io	2.4
commons-lang	commons-lang	2.6
commons-logging	commons-logging	1.1.3
commons-net	commons-net	3.1
commons-pool	commons-pool	1.5.4
info.ganglia.gmetric4j	gmetric4j	1.0.7
io.airlift	aircompressor	0.10
io.dropwizard.metrics	metrics-core	3.1.5
io.dropwizard.metrics	metrics-ganglia	3.1.5
io.dropwizard.metrics	metrics-graphite	3.1.5
io.dropwizard.metrics	metrics-healthchecks	3.1.5
io.dropwizard.metrics	metrics-jetty9	3.1.5
io.dropwizard.metrics	metrics-json	3.1.5
io.dropwizard.metrics	metrics-jvm	3.1.5
io.dropwizard.metrics	metrics-log4j	3.1.5
io.dropwizard.metrics	metrics-servlets	3.1.5
io.netty	netty	3.9.9.Final
io.netty	netty-all	4.1.42.Final
javax.activation	activation	1.1.1
javax.annotation	javax.annotation-api	1.2
javax.el	javax.el-api	2.2.4
javax.jdo	jdo-api	3.0.1
javax.servlet	javax.servlet-api	3.1.0
javax.servlet.jsp	jsp-api	2.1
javax.transaction	jta	1.1
javax.validation	validation-api	1.1.0.Final
javax.ws.rs	javax.ws.rs-api	2.0.1
javax.xml.bind	jaxb-api	2.2.2
javax.xml.stream	stax-api	1.0-2
javolution	javolution	5.5.1
jline	jline	2.14.6
joda-time	joda-time	2.9.3
junit	junit	4.12
log4j	apache-log4j-extras	1.2.17
log4j	log4j	1.2.17
net.hydromatic	eigenbase-properties	1.1.5
net.razorvine	pyrolite	4.13
net.sf.jpam	jpam	1.1
net.sf.opencsv	opencsv	2.3
net.sf.supercsv	super-csv	2.2.0
net.snowflake	snowflake-ingest-sdk	0.9.6
net.snowflake	snowflake-jdbc	3.12.0
net.snowflake	spark-snowflake_2.11	2.5.9-spark_2.4
net.sourceforge.f2j	arpack_combined_all	0.1
org.acplt	oncrpc	1.0.7
org.antlr	ST4	4.0.4
org.antlr	antlr-runtime	3.4
org.antlr	antlr4-runtime	4.7
org.antlr	stringtemplate	3.2.1
org.apache.ant	ant	1.9.2
org.apache.ant	ant-jsch	1.9.2
org.apache.ant	ant-launcher	1.9.2
org.apache.arrow	arrow-format	0.10.0
org.apache.arrow	arrow-memory	0.10.0
org.apache.arrow	arrow-vector	0.10.0
org.apache.avro	avro	1.8.2
org.apache.avro	avro-ipc	1.8.2
org.apache.avro	avro-mapred-hadoop2	1.8.2
org.apache.calcite	calcite-avatica	1.2.0-incubating
org.apache.calcite	calcite-core	1.2.0-incubating
org.apache.calcite	calcite-linq4j	1.2.0-incubating
org.apache.commons	commons-compress	1.8.1
org.apache.commons	commons-crypto	1.0.0
org.apache.commons	commons-lang3	3.5
org.apache.commons	commons-math3	3.4.1
org.apache.curator	curator-client	2.7.1
org.apache.curator	curator-framework	2.7.1
org.apache.curator	curator-recipes	2.7.1
org.apache.derby	derby	10.12.1.1
org.apache.directory.api	api-asn1-api	1.0.0-M20
org.apache.directory.api	api-util	1.0.0-M20
org.apache.directory.server	apacheds-i18n	2.0.0-M15
org.apache.directory.server	apacheds-kerberos-codec	2.0.0-M15
org.apache.hadoop	hadoop-annotations	2.7.3
org.apache.hadoop	hadoop-auth	2.7.3
org.apache.hadoop	hadoop-client	2.7.3
org.apache.hadoop	hadoop-common	2.7.3
org.apache.hadoop	hadoop-hdfs	2.7.3
org.apache.hadoop	hadoop-mapreduce-client-app	2.7.3
org.apache.hadoop	hadoop-mapreduce-client-common	2.7.3
org.apache.hadoop	hadoop-mapreduce-client-core	2.7.3
org.apache.hadoop	hadoop-mapreduce-client-jobclient	2.7.3
org.apache.hadoop	hadoop-mapreduce-client-shuffle	2.7.3
org.apache.hadoop	hadoop-yarn-api	2.7.3
org.apache.hadoop	hadoop-yarn-client	2.7.3
org.apache.hadoop	hadoop-yarn-common	2.7.3
org.apache.hadoop	hadoop-yarn-server-common	2.7.3
org.apache.htrace	htrace-core	3.1.0-incubating
org.apache.httpcomponents	httpclient	4.5.6
org.apache.httpcomponents	httpcore	4.4.10
org.apache.ivy	ivy	2.4.0
org.apache.orc	orc-core-nohive	1.5.5
org.apache.orc	orc-mapreduce-nohive	1.5.5
org.apache.orc	orc-shims	1.5.5
org.apache.parquet	parquet-column	1.10.1.2-databricks4
org.apache.parquet	parquet-common	1.10.1.2-databricks4
org.apache.parquet	parquet-encoding	1.10.1.2-databricks4
org.apache.parquet	parquet-format	2.4.0
org.apache.parquet	parquet-hadoop	1.10.1.2-databricks4
org.apache.parquet	parquet-jackson	1.10.1.2-databricks4
org.apache.thrift	libfb303	0.9.3
org.apache.thrift	libthrift	0.9.3
org.apache.xbean	xbean-asm6-shaded	4.8
org.apache.zookeeper	zookeeper	3.4.6
org.codehaus.jackson	jackson-core-asl	1.9.13
org.codehaus.jackson	jackson-jaxrs	1.9.13
org.codehaus.jackson	jackson-mapper-asl	1.9.13
org.codehaus.jackson	jackson-xc	1.9.13
org.codehaus.janino	commons-compiler	3.0.10
org.codehaus.janino	janino	3.0.10
org.datanucleus	datanucleus-api-jdo	3.2.6
org.datanucleus	datanucleus-core	3.2.10
org.datanucleus	datanucleus-rdbms	3.2.9
org.eclipse.jetty	jetty-client	9.3.27.v20190418
org.eclipse.jetty	jetty-continuation	9.3.27.v20190418
org.eclipse.jetty	jetty-http	9.3.27.v20190418
org.eclipse.jetty	jetty-io	9.3.27.v20190418
org.eclipse.jetty	jetty-jndi	9.3.27.v20190418
org.eclipse.jetty	jetty-plus	9.3.27.v20190418
org.eclipse.jetty	jetty-proxy	9.3.27.v20190418
org.eclipse.jetty	jetty-security	9.3.27.v20190418
org.eclipse.jetty	jetty-server	9.3.27.v20190418
org.eclipse.jetty	jetty-servlet	9.3.27.v20190418
org.eclipse.jetty	jetty-servlets	9.3.27.v20190418
org.eclipse.jetty	jetty-util	9.3.27.v20190418
org.eclipse.jetty	jetty-webapp	9.3.27.v20190418
org.eclipse.jetty	jetty-xml	9.3.27.v20190418
org.fusesource.leveldbjni	leveldbjni-all	1.8
org.glassfish.hk2	hk2-api	2.4.0-b34
org.glassfish.hk2	hk2-locator	2.4.0-b34
org.glassfish.hk2	hk2-utils	2.4.0-b34
org.glassfish.hk2	osgi-resource-locator	1.0.1
org.glassfish.hk2.external	aopalliance-repackaged	2.4.0-b34
org.glassfish.hk2.external	javax.inject	2.4.0-b34
org.glassfish.jersey.bundles.repackaged	jersey-guava	2.22.2
org.glassfish.jersey.containers	jersey-container-servlet	2.22.2
org.glassfish.jersey.containers	jersey-container-servlet-core	2.22.2
org.glassfish.jersey.core	jersey-client	2.22.2
org.glassfish.jersey.core	jersey-common	2.22.2
org.glassfish.jersey.core	jersey-server	2.22.2
org.glassfish.jersey.media	jersey-media-jaxb	2.22.2
org.hamcrest	hamcrest-core	1.3
org.hamcrest	hamcrest-library	1.3
org.hibernate	hibernate-validator	5.1.1.Final
org.iq80.snappy	snappy	0.2
org.javassist	javassist	3.18.1-GA
org.jboss.logging	jboss-logging	3.1.3.GA
org.jdbi	jdbi	2.63.1
org.joda	joda-convert	1.7
org.jodd	jodd-core	3.5.2
org.json4s	json4s-ast_2.11	3.5.3
org.json4s	json4s-core_2.11	3.5.3
org.json4s	json4s-jackson_2.11	3.5.3
org.json4s	json4s-scalap_2.11	3.5.3
org.lz4	lz4-java	1.4.0
org.mariadb.jdbc	mariadb-java-client	2.1.2
org.mockito	mockito-core	1.10.19
org.objenesis	objenesis	2.5.1
org.postgresql	postgresql	42.1.4
org.roaringbitmap	RoaringBitmap	0.7.45
org.roaringbitmap	shims	0.7.45
org.rocksdb	rocksdbjni	6.2.2
org.rosuda.REngine	REngine	2.1.0
org.scala-lang	scala-compiler_2.11	2.11.12
org.scala-lang	scala-library_2.11	2.11.12
org.scala-lang	scala-reflect_2.11	2.11.12
org.scala-lang.modules	scala-parser-combinators_2.11	1.1.0
org.scala-lang.modules	scala-xml_2.11	1.0.5
org.scala-sbt	test-interface	1.0
org.scalacheck	scalacheck_2.11	1.12.5
org.scalactic	scalactic_2.11	3.0.3
org.scalanlp	breeze-macros_2.11	0.13.2
org.scalanlp	breeze_2.11	0.13.2
org.scalatest	scalatest_2.11	3.0.3
org.slf4j	jcl-over-slf4j	1.7.16
org.slf4j	jul-to-slf4j	1.7.16
org.slf4j	slf4j-api	1.7.16
org.slf4j	slf4j-log4j12	1.7.16
org.spark-project.hive	hive-beeline	1.2.1.spark2
org.spark-project.hive	hive-cli	1.2.1.spark2
org.spark-project.hive	hive-jdbc	1.2.1.spark2
org.spark-project.hive	hive-metastore	1.2.1.spark2
org.spark-project.spark	unused	1.0.0
org.spire-math	spire-macros_2.11	0.13.0
org.spire-math	spire_2.11	0.13.0
org.springframework	spring-core	4.1.4.RELEASE
org.springframework	spring-test	4.1.4.RELEASE
org.tukaani	xz	1.5
org.typelevel	machinist_2.11	0.6.1
org.typelevel	macro-compat_2.11	1.1.1
org.xerial	sqlite-jdbc	3.8.11.2
org.xerial.snappy	snappy-java	1.1.7.3
org.yaml	snakeyaml	1.16
oro	oro	2.0.8
software.amazon.ion	ion-java	1.0.2
stax	stax-api	1.0.1
xmlenc	xmlenc	0.52