Amazon
Web Services
Microsoft Azure
Google Cloud PlatformDatabricks Runtime 6.2 (unsupported)
Databricks released this image in December 2019.
The following release notes provide information about Databricks Runtime 6.2, powered by Apache Spark.
New features
Delta Lake insert-only merge is optimized
The performance of Delta Lake merge operations that have no update clauses has been optimized. Writes made by such insert-only merge operations now only append data and do not modify existing data, thus allowing downstream streaming queries to continuously read the appended data. This better supports merge-based deduplication: batch or streaming ETL jobs can write deduplicated data into tables using insert-only merge, and downstream streaming queries can continuously read that deduplicated data.
Redshift connector adds multi-region support for reads
The Redshift connector now supports using an S3 bucket located in a different region than the Redshift cluster. You specify the following DataFrameReader and Writer option: .option("awsregion", <region>). If specified, the <region> value is passed to the Redshift UNLOAD and COPY commands as REGION '<region>'.
Note
This functionality could be achieved in previous Databricks Runtime versions for Redshift writes by setting .option(“extracopyoptions”, “REGION '<region>'“), but not for reads. The former option is still supported.
Improvements
display()ofBinaryTypecolumns are automatically truncated to the base64 encoding of first 128 bytes, instead of all bytes. If the field is truncated, you will see"(truncated)"at the end.Upgraded several installed R libraries. See Installed R libraries.
Deprecations
display() of spark.mllib models is deprecated and it will be removed in the next major release. Databricks recommends using spark.ml models.
Apache Spark
Databricks Runtime 6.2 includes Apache Spark 2.4.4. This release includes all Spark fixes and improvements included in Databricks Runtime 6.1 (unsupported), as well as the following additional bug fixes and improvements made to Spark:
[SPARK-29875][PYTHON][SQL] Avoid to use deprecated pyarrow.open_stream API in Spark 2.4.x
[SPARK-29001][CORE] Print events that take too long time to process
[SPARK-29850][SQL] sort-merge-join an empty table should not memory leak
[SPARK-29790][DOC] Note required port for Kube API
[SPARK-29743][SQL] sample should set needCopyResult to true if its child is
[SPARK-29651][SQL] Fix parsing of interval seconds fraction
[SPARK-29604][SQL] Force initialize SessionState before initializing HiveClient in SparkSQLEnv
[SPARK-29637][CORE] Add description to Job SHS web API
[SPARK-26985][CORE] Fix “access only some column of the all of columns ” for big endian architecture
[SPARK-29498][SQL] CatalogTable to HiveTable should not change the table’s ownership
[SPARK-29530][SQL] Make SQLConf in SQL parse process thread safe
[SPARK-21287][SQL] Remove requirement of fetch_size>=0 from JDBCOptions
[SPARK-29081]Replace calls to SerializationUtils.clone on properties with a faster implementation
[SPARK-21492][SQL][FOLLOW UP] Reimplement UnsafeExternalRowSorter in database style iterator
[SPARK-21492][SQL] Fix memory leak in SortMergeJoin
[SPARK-29556][CORE] Avoid putting request path in error response in ErrorServlet
[SPARK-29494][SQL] Fix for ArrayOutofBoundsException while converting string to timestamp
[SPARK-27259][CORE] Allow setting -1 as length for FileBlock
[SPARK-29445][CORE] Bump netty-all from 4.1.39.Final to 4.1.42.Final
[SPARK-29410] Update commons-beanutils to 1.9.4
[SPARK-28917][CORE] Synchronize access to RDD mutable state
[SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION
[SPARK-29048] Improve performance on Column.isInCollection() with a large size collection
[SPARK-28962][SQL] Provide index argument to filter lambda functions
[SPARK-25753][CORE] Fix reading small files via BinaryFileRDD
[SPARK-28140][MLLIB][PYTHON] Accept DataFrames in RowMatrix and IndexedRowMatrix constructors
[SPARK-29244][CORE] Prevent freed page in BytesToBytesMap free again
[SPARK-29055][CORE] Update driver/executors’ storage memory when block is removed from BlockManager
[SPARK-29186][SQL] AliasIdentifier should be converted to Json in prettyJson
[SPARK-29186][SQL] AliasIdentifier should be converted to Json in prettyJson
[SPARK-29247][SQL] Redact sensitive information in when construct HiveClientHive.state
[SPARK-29263][SCHEDULER] Update availableSlots in resourceOffers() before checking available slots for barrier taskSet
[SPARK-29240][PYTHON] Pass Py4J column instance to support PySpark column in element_at function
[SPARK-29213][SQL] Generate extra IsNotNull predicate in FilterExec
[SPARK-29229][SQL] Change the additional remote repository in IsolatedClientLoader to google minor
[SPARK-28599][SQL] Fix Duration column sorting for ThriftServerSessionPage
[SPARK-25903][CORE] TimerTask should be synchronized on ContextBarrierState
[SPARK-29053][WEBUI] Sort does not work on some columns
[SPARK-29177][CORE] fix zombie tasks after stage abort
System environment
Operating System: Ubuntu 16.04.6 LTS
Java: 1.8.0_232
Scala: 2.11.12
Python: 3.7.3
R: R version 3.6.1 (2019-07-05)
Delta Lake: 0.4.0
Note
Although Scala 2.12 is available as an experimental feature in Apache Spark 2.4, it is not supported in Databricks Runtime 6.2.
In this section:
Installed Python libraries
Library |
Version |
Library |
Version |
Library |
Version |
|---|---|---|---|---|---|
asn1crypto |
0.24.0 |
backcall |
0.1.0 |
boto |
2.49.0 |
boto3 |
1.9.162 |
botocore |
1.12.163 |
certifi |
2019.3.9 |
cffi |
1.12.2 |
chardet |
3.0.4 |
cryptography |
2.6.1 |
cycler |
0.10.0 |
Cython |
0.29.6 |
decorator |
4.4.0 |
docutils |
0.14 |
idna |
2.8 |
ipython |
7.4.0 |
ipython-genutils |
0.2.0 |
jedi |
0.13.3 |
jmespath |
0.9.4 |
kiwisolver |
1.1.0 |
matplotlib |
3.0.3 |
numpy |
1.16.2 |
pandas |
0.24.2 |
parso |
0.3.4 |
patsy |
0.5.1 |
pexpect |
4.6.0 |
pickleshare |
0.7.5 |
pip |
19.0.3 |
prompt-toolkit |
2.0.9 |
psycopg2 |
2.7.6.1 |
ptyprocess |
0.6.0 |
pyarrow |
0.13.0 |
pycparser |
2.19 |
pycurl |
7.43.0 |
Pygments |
2.3.1 |
PyGObject |
3.20.0 |
pyOpenSSL |
19.0.0 |
pyparsing |
2.4.2 |
PySocks |
1.6.8 |
python-apt |
1.1.0.b1+ubuntu0.16.04.5 |
python-dateutil |
2.8.0 |
pytz |
2018.9 |
requests |
2.21.0 |
s3transfer |
0.2.1 |
scikit-learn |
0.20.3 |
scipy |
1.2.1 |
seaborn |
0.9.0 |
setuptools |
40.8.0 |
six |
1.12.0 |
ssh-import-id |
5.5 |
statsmodels |
0.9.0 |
traitlets |
4.3.2 |
unattended-upgrades |
0.1 |
urllib3 |
1.24.1 |
virtualenv |
16.4.1 |
wcwidth |
0.1.7 |
wheel |
0.33.1 |
Installed R libraries
Library |
Version |
Library |
Version |
Library |
Version |
|---|---|---|---|---|---|
abind |
1.4-5 |
askpass |
1.1 |
assertthat |
0.2.1 |
backports |
1.1.5 |
base |
3.6.1 |
base64enc |
0.1-3 |
BH |
1.69.0-1 |
bit |
1.1-14 |
bit64 |
0.9-7 |
bitops |
1.0-6 |
blob |
1.2.0 |
boot |
1.3-23 |
brew |
1.0-6 |
callr |
3.3.2 |
car |
3.0-4 |
carData |
3.0-2 |
caret |
6.0-84 |
cellranger |
1.1.0 |
chron |
2.3-54 |
class |
7.3-15 |
cli |
1.1.0 |
clipr |
0.7.0 |
clisymbols |
1.2.0 |
cluster |
2.1.0 |
codetools |
0.2-16 |
colorspace |
1.4-1 |
commonmark |
1.7 |
compiler |
3.6.1 |
config |
0.3 |
covr |
3.3.2 |
crayon |
1.3.4 |
crosstalk |
1.0.0 |
curl |
4.2 |
data.table |
1.12.6 |
datasets |
3.6.1 |
DBI |
1.0.0 |
dbplyr |
1.4.2 |
desc |
1.2.0 |
devtools |
2.2.1 |
digest |
0.6.22 |
doMC |
1.3.6 |
dplyr |
0.8.3 |
DT |
0.9 |
ellipsis |
0.3.0 |
evaluate |
0.14 |
fansi |
0.4.0 |
fastmap |
1.0.1 |
forcats |
0.4.0 |
foreach |
1.4.7 |
foreign |
0.8-72 |
forge |
0.2.0 |
fs |
1.3.1 |
gbm |
2.1.5 |
generics |
0.0.2 |
ggplot2 |
3.2.1 |
gh |
1.0.1 |
git2r |
0.26.1 |
glmnet |
2.0-18 |
glue |
1.3.1 |
gower |
0.2.1 |
graphics |
3.6.1 |
grDevices |
3.6.1 |
grid |
3.6.1 |
gridExtra |
2.3 |
gsubfn |
0.7 |
gtable |
0.3.0 |
h2o |
3.26.0.2 |
haven |
2.1.1 |
hms |
0.5.1 |
htmltools |
0.4.0 |
htmlwidgets |
1.5.1 |
httpuv |
1.5.2 |
httr |
1.4.1 |
hwriter |
1.3.2 |
hwriterPlus |
1.0-3 |
ini |
0.3.1 |
ipred |
0.9-9 |
iterators |
1.0.12 |
jsonlite |
1.6 |
KernSmooth |
2.23-16 |
labeling |
0.3 |
later |
1.0.0 |
lattice |
0.20-38 |
lava |
1.6.6 |
lazyeval |
0.2.2 |
lifecycle |
0.1.0 |
littler |
0.3.9 |
lme4 |
1.1-21 |
lubridate |
1.7.4 |
magrittr |
1.5 |
mapproj |
1.2.6 |
maps |
3.3.0 |
maptools |
0.9-8 |
MASS |
7.3-51.4 |
Matrix |
1.2-17 |
MatrixModels |
0.4-1 |
memoise |
1.1.0 |
methods |
3.6.1 |
mgcv |
1.8-30 |
mime |
0.7 |
minqa |
1.2.4 |
ModelMetrics |
1.2.2 |
munsell |
0.5.0 |
mvtnorm |
1.0-11 |
nlme |
3.1-141 |
nloptr |
1.2.1 |
nnet |
7.3-12 |
numDeriv |
2016.8-1.1 |
openssl |
1.4.1 |
openxlsx |
4.1.2 |
parallel |
3.6.1 |
pbkrtest |
0.4-7 |
pillar |
1.4.2 |
pkgbuild |
1.0.6 |
pkgconfig |
2.0.3 |
pkgKitten |
0.1.5 |
pkgload |
1.0.2 |
plogr |
0.2.0 |
plyr |
1.8.4 |
praise |
1.0.0 |
prettyunits |
1.0.2 |
pROC |
1.15.3 |
processx |
3.4.1 |
prodlim |
2018.04.18 |
progress |
1.2.2 |
promises |
1.1.0 |
proto |
1.0.0 |
ps |
1.3.0 |
purrr |
0.3.3 |
quantreg |
5.51 |
R.methodsS3 |
1.7.1 |
R.oo |
1.22.0 |
R.utils |
2.9.0 |
r2d3 |
0.2.3 |
R6 |
2.4.0 |
randomForest |
4.6-14 |
rappdirs |
0.3.1 |
rcmdcheck |
1.3.3 |
RColorBrewer |
1.1-2 |
Rcpp |
1.0.2 |
RcppEigen |
0.3.3.5.0 |
RCurl |
1.95-4.12 |
readr |
1.3.1 |
readxl |
1.3.1 |
recipes |
0.1.7 |
rematch |
1.0.1 |
remotes |
2.1.0 |
reshape2 |
1.4.3 |
rex |
1.1.2 |
rio |
0.5.16 |
rlang |
0.4.1 |
RODBC |
1.3-16 |
roxygen2 |
6.1.1 |
rpart |
4.1-15 |
rprojroot |
1.3-2 |
Rserve |
1.8-6 |
RSQLite |
2.1.2 |
rstudioapi |
0.10 |
rversions |
2.0.0 |
scales |
1.0.0 |
sessioninfo |
1.1.1 |
shiny |
1.4.0 |
sourcetools |
0.1.7 |
sp |
1.3-1 |
sparklyr |
1.0.4 |
SparkR |
2.4.5 |
SparseM |
1.77 |
spatial |
7.3-11 |
splines |
3.6.1 |
sqldf |
0.4-11 |
SQUAREM |
2017.10-1 |
statmod |
1.4.32 |
stats |
3.6.1 |
stats4 |
3.6.1 |
stringi |
1.4.3 |
stringr |
1.4.0 |
survival |
2.44-1.1 |
sys |
3.3 |
tcltk |
3.6.1 |
TeachingDemos |
2.10 |
testthat |
2.2.1 |
tibble |
2.1.3 |
tidyr |
1.0.0 |
tidyselect |
0.2.5 |
timeDate |
3043.102 |
tools |
3.6.1 |
usethis |
1.5.1 |
utf8 |
1.1.4 |
utils |
3.6.1 |
vctrs |
0.2.0 |
viridisLite |
0.3.0 |
whisker |
0.4 |
withr |
2.1.2 |
xml2 |
1.2.2 |
xopen |
1.0.0 |
xtable |
1.8-4 |
yaml |
2.2.0 |
zeallot |
0.1.0 |
zip |
2.0.4 |
Installed Java and Scala libraries (Scala 2.11 cluster version)
Group ID |
Artifact ID |
Version |
|---|---|---|
antlr |
antlr |
2.7.7 |
com.amazonaws |
amazon-kinesis-client |
1.8.10 |
com.amazonaws |
aws-java-sdk-autoscaling |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudformation |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudfront |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudhsm |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudsearch |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudtrail |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudwatch |
1.11.595 |
com.amazonaws |
aws-java-sdk-cloudwatchmetrics |
1.11.595 |
com.amazonaws |
aws-java-sdk-codedeploy |
1.11.595 |
com.amazonaws |
aws-java-sdk-cognitoidentity |
1.11.595 |
com.amazonaws |
aws-java-sdk-cognitosync |
1.11.595 |
com.amazonaws |
aws-java-sdk-config |
1.11.595 |
com.amazonaws |
aws-java-sdk-core |
1.11.595 |
com.amazonaws |
aws-java-sdk-datapipeline |
1.11.595 |
com.amazonaws |
aws-java-sdk-directconnect |
1.11.595 |
com.amazonaws |
aws-java-sdk-directory |
1.11.595 |
com.amazonaws |
aws-java-sdk-dynamodb |
1.11.595 |
com.amazonaws |
aws-java-sdk-ec2 |
1.11.595 |
com.amazonaws |
aws-java-sdk-ecs |
1.11.595 |
com.amazonaws |
aws-java-sdk-efs |
1.11.595 |
com.amazonaws |
aws-java-sdk-elasticache |
1.11.595 |
com.amazonaws |
aws-java-sdk-elasticbeanstalk |
1.11.595 |
com.amazonaws |
aws-java-sdk-elasticloadbalancing |
1.11.595 |
com.amazonaws |
aws-java-sdk-elastictranscoder |
1.11.595 |
com.amazonaws |
aws-java-sdk-emr |
1.11.595 |
com.amazonaws |
aws-java-sdk-glacier |
1.11.595 |
com.amazonaws |
aws-java-sdk-glue |
1.11.595 |
com.amazonaws |
aws-java-sdk-iam |
1.11.595 |
com.amazonaws |
aws-java-sdk-importexport |
1.11.595 |
com.amazonaws |
aws-java-sdk-kinesis |
1.11.595 |
com.amazonaws |
aws-java-sdk-kms |
1.11.595 |
com.amazonaws |
aws-java-sdk-lambda |
1.11.595 |
com.amazonaws |
aws-java-sdk-logs |
1.11.595 |
com.amazonaws |
aws-java-sdk-machinelearning |
1.11.595 |
com.amazonaws |
aws-java-sdk-opsworks |
1.11.595 |
com.amazonaws |
aws-java-sdk-rds |
1.11.595 |
com.amazonaws |
aws-java-sdk-redshift |
1.11.595 |
com.amazonaws |
aws-java-sdk-route53 |
1.11.595 |
com.amazonaws |
aws-java-sdk-s3 |
1.11.595 |
com.amazonaws |
aws-java-sdk-ses |
1.11.595 |
com.amazonaws |
aws-java-sdk-simpledb |
1.11.595 |
com.amazonaws |
aws-java-sdk-simpleworkflow |
1.11.595 |
com.amazonaws |
aws-java-sdk-sns |
1.11.595 |
com.amazonaws |
aws-java-sdk-sqs |
1.11.595 |
com.amazonaws |
aws-java-sdk-ssm |
1.11.595 |
com.amazonaws |
aws-java-sdk-storagegateway |
1.11.595 |
com.amazonaws |
aws-java-sdk-sts |
1.11.595 |
com.amazonaws |
aws-java-sdk-support |
1.11.595 |
com.amazonaws |
aws-java-sdk-swf-libraries |
1.11.22 |
com.amazonaws |
aws-java-sdk-workspaces |
1.11.595 |
com.amazonaws |
jmespath-java |
1.11.595 |
com.carrotsearch |
hppc |
0.7.2 |
com.chuusai |
shapeless_2.11 |
2.3.2 |
com.clearspring.analytics |
stream |
2.7.0 |
com.databricks |
Rserve |
1.8-3 |
com.databricks |
dbml-local_2.11 |
0.5.0-db8-spark2.4 |
com.databricks |
dbml-local_2.11-tests |
0.5.0-db8-spark2.4 |
com.databricks |
jets3t |
0.7.1-0 |
com.databricks.scalapb |
compilerplugin_2.11 |
0.4.15-9 |
com.databricks.scalapb |
scalapb-runtime_2.11 |
0.4.15-9 |
com.esotericsoftware |
kryo-shaded |
4.0.2 |
com.esotericsoftware |
minlog |
1.3.0 |
com.fasterxml |
classmate |
1.0.0 |
com.fasterxml.jackson.core |
jackson-annotations |
2.6.7 |
com.fasterxml.jackson.core |
jackson-core |
2.6.7 |
com.fasterxml.jackson.core |
jackson-databind |
2.6.7.1 |
com.fasterxml.jackson.dataformat |
jackson-dataformat-cbor |
2.6.7 |
com.fasterxml.jackson.datatype |
jackson-datatype-joda |
2.6.7 |
com.fasterxml.jackson.module |
jackson-module-paranamer |
2.6.7 |
com.fasterxml.jackson.module |
jackson-module-scala_2.11 |
2.6.7.1 |
com.github.fommil |
jniloader |
1.1 |
com.github.fommil.netlib |
core |
1.1.2 |
com.github.fommil.netlib |
native_ref-java |
1.1 |
com.github.fommil.netlib |
native_ref-java-natives |
1.1 |
com.github.fommil.netlib |
native_system-java |
1.1 |
com.github.fommil.netlib |
native_system-java-natives |
1.1 |
com.github.fommil.netlib |
netlib-native_ref-linux-x86_64-natives |
1.1 |
com.github.fommil.netlib |
netlib-native_system-linux-x86_64-natives |
1.1 |
com.github.luben |
zstd-jni |
1.3.2-2 |
com.github.rwl |
jtransforms |
2.4.0 |
com.google.code.findbugs |
jsr305 |
2.0.1 |
com.google.code.gson |
gson |
2.2.4 |
com.google.guava |
guava |
15.0 |
com.google.protobuf |
protobuf-java |
2.6.1 |
com.googlecode.javaewah |
JavaEWAH |
0.3.2 |
com.h2database |
h2 |
1.3.174 |
com.jcraft |
jsch |
0.1.50 |
com.jolbox |
bonecp |
0.8.0.RELEASE |
com.microsoft.azure |
azure-data-lake-store-sdk |
2.2.8 |
com.microsoft.azure |
azure-storage |
7.0.0 |
com.microsoft.sqlserver |
mssql-jdbc |
6.2.2.jre8 |
com.ning |
compress-lzf |
1.0.3 |
com.sun.mail |
javax.mail |
1.5.2 |
com.thoughtworks.paranamer |
paranamer |
2.8 |
com.trueaccord.lenses |
lenses_2.11 |
0.3 |
com.twitter |
chill-java |
0.9.3 |
com.twitter |
chill_2.11 |
0.9.3 |
com.twitter |
parquet-hadoop-bundle |
1.6.0 |
com.twitter |
util-app_2.11 |
6.23.0 |
com.twitter |
util-core_2.11 |
6.23.0 |
com.twitter |
util-jvm_2.11 |
6.23.0 |
com.typesafe |
config |
1.2.1 |
com.typesafe.scala-logging |
scala-logging-api_2.11 |
2.1.2 |
com.typesafe.scala-logging |
scala-logging-slf4j_2.11 |
2.1.2 |
com.univocity |
univocity-parsers |
2.7.3 |
com.vlkan |
flatbuffers |
1.2.0-3f79e055 |
com.zaxxer |
HikariCP |
3.1.0 |
commons-beanutils |
commons-beanutils |
1.9.4 |
commons-cli |
commons-cli |
1.2 |
commons-codec |
commons-codec |
1.10 |
commons-collections |
commons-collections |
3.2.2 |
commons-configuration |
commons-configuration |
1.6 |
commons-dbcp |
commons-dbcp |
1.4 |
commons-digester |
commons-digester |
1.8 |
commons-httpclient |
commons-httpclient |
3.1 |
commons-io |
commons-io |
2.4 |
commons-lang |
commons-lang |
2.6 |
commons-logging |
commons-logging |
1.1.3 |
commons-net |
commons-net |
3.1 |
commons-pool |
commons-pool |
1.5.4 |
info.ganglia.gmetric4j |
gmetric4j |
1.0.7 |
io.airlift |
aircompressor |
0.10 |
io.dropwizard.metrics |
metrics-core |
3.1.5 |
io.dropwizard.metrics |
metrics-ganglia |
3.1.5 |
io.dropwizard.metrics |
metrics-graphite |
3.1.5 |
io.dropwizard.metrics |
metrics-healthchecks |
3.1.5 |
io.dropwizard.metrics |
metrics-jetty9 |
3.1.5 |
io.dropwizard.metrics |
metrics-json |
3.1.5 |
io.dropwizard.metrics |
metrics-jvm |
3.1.5 |
io.dropwizard.metrics |
metrics-log4j |
3.1.5 |
io.dropwizard.metrics |
metrics-servlets |
3.1.5 |
io.netty |
netty |
3.9.9.Final |
io.netty |
netty-all |
4.1.42.Final |
javax.activation |
activation |
1.1.1 |
javax.annotation |
javax.annotation-api |
1.2 |
javax.el |
javax.el-api |
2.2.4 |
javax.jdo |
jdo-api |
3.0.1 |
javax.servlet |
javax.servlet-api |
3.1.0 |
javax.servlet.jsp |
jsp-api |
2.1 |
javax.transaction |
jta |
1.1 |
javax.validation |
validation-api |
1.1.0.Final |
javax.ws.rs |
javax.ws.rs-api |
2.0.1 |
javax.xml.bind |
jaxb-api |
2.2.2 |
javax.xml.stream |
stax-api |
1.0-2 |
javolution |
javolution |
5.5.1 |
jline |
jline |
2.14.6 |
joda-time |
joda-time |
2.9.3 |
junit |
junit |
4.12 |
log4j |
apache-log4j-extras |
1.2.17 |
log4j |
log4j |
1.2.17 |
net.hydromatic |
eigenbase-properties |
1.1.5 |
net.razorvine |
pyrolite |
4.13 |
net.sf.jpam |
jpam |
1.1 |
net.sf.opencsv |
opencsv |
2.3 |
net.sf.supercsv |
super-csv |
2.2.0 |
net.snowflake |
snowflake-ingest-sdk |
0.9.6 |
net.snowflake |
snowflake-jdbc |
3.12.0 |
net.snowflake |
spark-snowflake_2.11 |
2.5.9-spark_2.4 |
net.sourceforge.f2j |
arpack_combined_all |
0.1 |
org.acplt |
oncrpc |
1.0.7 |
org.antlr |
ST4 |
4.0.4 |
org.antlr |
antlr-runtime |
3.4 |
org.antlr |
antlr4-runtime |
4.7 |
org.antlr |
stringtemplate |
3.2.1 |
org.apache.ant |
ant |
1.9.2 |
org.apache.ant |
ant-jsch |
1.9.2 |
org.apache.ant |
ant-launcher |
1.9.2 |
org.apache.arrow |
arrow-format |
0.10.0 |
org.apache.arrow |
arrow-memory |
0.10.0 |
org.apache.arrow |
arrow-vector |
0.10.0 |
org.apache.avro |
avro |
1.8.2 |
org.apache.avro |
avro-ipc |
1.8.2 |
org.apache.avro |
avro-mapred-hadoop2 |
1.8.2 |
org.apache.calcite |
calcite-avatica |
1.2.0-incubating |
org.apache.calcite |
calcite-core |
1.2.0-incubating |
org.apache.calcite |
calcite-linq4j |
1.2.0-incubating |
org.apache.commons |
commons-compress |
1.8.1 |
org.apache.commons |
commons-crypto |
1.0.0 |
org.apache.commons |
commons-lang3 |
3.5 |
org.apache.commons |
commons-math3 |
3.4.1 |
org.apache.curator |
curator-client |
2.7.1 |
org.apache.curator |
curator-framework |
2.7.1 |
org.apache.curator |
curator-recipes |
2.7.1 |
org.apache.derby |
derby |
10.12.1.1 |
org.apache.directory.api |
api-asn1-api |
1.0.0-M20 |
org.apache.directory.api |
api-util |
1.0.0-M20 |
org.apache.directory.server |
apacheds-i18n |
2.0.0-M15 |
org.apache.directory.server |
apacheds-kerberos-codec |
2.0.0-M15 |
org.apache.hadoop |
hadoop-annotations |
2.7.3 |
org.apache.hadoop |
hadoop-auth |
2.7.3 |
org.apache.hadoop |
hadoop-client |
2.7.3 |
org.apache.hadoop |
hadoop-common |
2.7.3 |
org.apache.hadoop |
hadoop-hdfs |
2.7.3 |
org.apache.hadoop |
hadoop-mapreduce-client-app |
2.7.3 |
org.apache.hadoop |
hadoop-mapreduce-client-common |
2.7.3 |
org.apache.hadoop |
hadoop-mapreduce-client-core |
2.7.3 |
org.apache.hadoop |
hadoop-mapreduce-client-jobclient |
2.7.3 |
org.apache.hadoop |
hadoop-mapreduce-client-shuffle |
2.7.3 |
org.apache.hadoop |
hadoop-yarn-api |
2.7.3 |
org.apache.hadoop |
hadoop-yarn-client |
2.7.3 |
org.apache.hadoop |
hadoop-yarn-common |
2.7.3 |
org.apache.hadoop |
hadoop-yarn-server-common |
2.7.3 |
org.apache.htrace |
htrace-core |
3.1.0-incubating |
org.apache.httpcomponents |
httpclient |
4.5.6 |
org.apache.httpcomponents |
httpcore |
4.4.10 |
org.apache.ivy |
ivy |
2.4.0 |
org.apache.orc |
orc-core-nohive |
1.5.5 |
org.apache.orc |
orc-mapreduce-nohive |
1.5.5 |
org.apache.orc |
orc-shims |
1.5.5 |
org.apache.parquet |
parquet-column |
1.10.1.2-databricks4 |
org.apache.parquet |
parquet-common |
1.10.1.2-databricks4 |
org.apache.parquet |
parquet-encoding |
1.10.1.2-databricks4 |
org.apache.parquet |
parquet-format |
2.4.0 |
org.apache.parquet |
parquet-hadoop |
1.10.1.2-databricks4 |
org.apache.parquet |
parquet-jackson |
1.10.1.2-databricks4 |
org.apache.thrift |
libfb303 |
0.9.3 |
org.apache.thrift |
libthrift |
0.9.3 |
org.apache.xbean |
xbean-asm6-shaded |
4.8 |
org.apache.zookeeper |
zookeeper |
3.4.6 |
org.codehaus.jackson |
jackson-core-asl |
1.9.13 |
org.codehaus.jackson |
jackson-jaxrs |
1.9.13 |
org.codehaus.jackson |
jackson-mapper-asl |
1.9.13 |
org.codehaus.jackson |
jackson-xc |
1.9.13 |
org.codehaus.janino |
commons-compiler |
3.0.10 |
org.codehaus.janino |
janino |
3.0.10 |
org.datanucleus |
datanucleus-api-jdo |
3.2.6 |
org.datanucleus |
datanucleus-core |
3.2.10 |
org.datanucleus |
datanucleus-rdbms |
3.2.9 |
org.eclipse.jetty |
jetty-client |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-continuation |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-http |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-io |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-jndi |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-plus |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-proxy |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-security |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-server |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-servlet |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-servlets |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-util |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-webapp |
9.3.27.v20190418 |
org.eclipse.jetty |
jetty-xml |
9.3.27.v20190418 |
org.fusesource.leveldbjni |
leveldbjni-all |
1.8 |
org.glassfish.hk2 |
hk2-api |
2.4.0-b34 |
org.glassfish.hk2 |
hk2-locator |
2.4.0-b34 |
org.glassfish.hk2 |
hk2-utils |
2.4.0-b34 |
org.glassfish.hk2 |
osgi-resource-locator |
1.0.1 |
org.glassfish.hk2.external |
aopalliance-repackaged |
2.4.0-b34 |
org.glassfish.hk2.external |
javax.inject |
2.4.0-b34 |
org.glassfish.jersey.bundles.repackaged |
jersey-guava |
2.22.2 |
org.glassfish.jersey.containers |
jersey-container-servlet |
2.22.2 |
org.glassfish.jersey.containers |
jersey-container-servlet-core |
2.22.2 |
org.glassfish.jersey.core |
jersey-client |
2.22.2 |
org.glassfish.jersey.core |
jersey-common |
2.22.2 |
org.glassfish.jersey.core |
jersey-server |
2.22.2 |
org.glassfish.jersey.media |
jersey-media-jaxb |
2.22.2 |
org.hamcrest |
hamcrest-core |
1.3 |
org.hamcrest |
hamcrest-library |
1.3 |
org.hibernate |
hibernate-validator |
5.1.1.Final |
org.iq80.snappy |
snappy |
0.2 |
org.javassist |
javassist |
3.18.1-GA |
org.jboss.logging |
jboss-logging |
3.1.3.GA |
org.jdbi |
jdbi |
2.63.1 |
org.joda |
joda-convert |
1.7 |
org.jodd |
jodd-core |
3.5.2 |
org.json4s |
json4s-ast_2.11 |
3.5.3 |
org.json4s |
json4s-core_2.11 |
3.5.3 |
org.json4s |
json4s-jackson_2.11 |
3.5.3 |
org.json4s |
json4s-scalap_2.11 |
3.5.3 |
org.lz4 |
lz4-java |
1.4.0 |
org.mariadb.jdbc |
mariadb-java-client |
2.1.2 |
org.mockito |
mockito-core |
1.10.19 |
org.objenesis |
objenesis |
2.5.1 |
org.postgresql |
postgresql |
42.1.4 |
org.roaringbitmap |
RoaringBitmap |
0.7.45 |
org.roaringbitmap |
shims |
0.7.45 |
org.rocksdb |
rocksdbjni |
6.2.2 |
org.rosuda.REngine |
REngine |
2.1.0 |
org.scala-lang |
scala-compiler_2.11 |
2.11.12 |
org.scala-lang |
scala-library_2.11 |
2.11.12 |
org.scala-lang |
scala-reflect_2.11 |
2.11.12 |
org.scala-lang.modules |
scala-parser-combinators_2.11 |
1.1.0 |
org.scala-lang.modules |
scala-xml_2.11 |
1.0.5 |
org.scala-sbt |
test-interface |
1.0 |
org.scalacheck |
scalacheck_2.11 |
1.12.5 |
org.scalactic |
scalactic_2.11 |
3.0.3 |
org.scalanlp |
breeze-macros_2.11 |
0.13.2 |
org.scalanlp |
breeze_2.11 |
0.13.2 |
org.scalatest |
scalatest_2.11 |
3.0.3 |
org.slf4j |
jcl-over-slf4j |
1.7.16 |
org.slf4j |
jul-to-slf4j |
1.7.16 |
org.slf4j |
slf4j-api |
1.7.16 |
org.slf4j |
slf4j-log4j12 |
1.7.16 |
org.spark-project.hive |
hive-beeline |
1.2.1.spark2 |
org.spark-project.hive |
hive-cli |
1.2.1.spark2 |
org.spark-project.hive |
hive-jdbc |
1.2.1.spark2 |
org.spark-project.hive |
hive-metastore |
1.2.1.spark2 |
org.spark-project.spark |
unused |
1.0.0 |
org.spire-math |
spire-macros_2.11 |
0.13.0 |
org.spire-math |
spire_2.11 |
0.13.0 |
org.springframework |
spring-core |
4.1.4.RELEASE |
org.springframework |
spring-test |
4.1.4.RELEASE |
org.tukaani |
xz |
1.5 |
org.typelevel |
machinist_2.11 |
0.6.1 |
org.typelevel |
macro-compat_2.11 |
1.1.1 |
org.xerial |
sqlite-jdbc |
3.8.11.2 |
org.xerial.snappy |
snappy-java |
1.1.7.3 |
org.yaml |
snakeyaml |
1.16 |
oro |
oro |
2.0.8 |
software.amazon.ion |
ion-java |
1.0.2 |
stax |
stax-api |
1.0.1 |
xmlenc |
xmlenc |
0.52 |
