Databricks Runtime 6.0 (Unsupported)

Databricks released this image in October 2019.

The following release notes provide information about Databricks Runtime 6.0, powered by Apache Spark.

New features

Python environment

Databricks Runtime 6.0 includes major changes to Python and the way Python environments are configured, including upgrading Python to 3.7.3, refining the list of installed Python packages, and upgrading those installed packages to newer versions. For details, see Installed Python libraries.

In addition, as was previously announced, Databricks Runtime 6.0 does not support Python 2.

Major changes include:

  • Upgraded Python from 3.5.2 to 3.7.3. Some old versions of Python packages may not be forward compatible with Python 3.7 because they depend on old versions of Cython that are not compatible with Python 3.7. Installing such a package can trigger errors similar to ‘PyThreadState’ {‘struct _ts’} has no member named ‘exc_type’ (see GitHub issue 1978 for details). Install Python 3.7 compatible versions of Python packages instead.

  • Major package upgrades:

    • boto3 to 1.9.162

    • ipython to 7.4.0

    • matplotlib to 3.0.3

    • numpy to 1.16.2

    • pandas to 0.24.2

    • pyarrow to 0.13.0

  • Compared with Databricks Runtime 5.5 LTS (Unsupported), the following Python packages are newly included: asn1crypto, backcall, jedi, kiwisolver, parso, and PySocks.

  • Compared with Databricks Runtime 5.5 LTS (Unsupported), the following Python packages are not installed: ansi2html, brewer2mpl, colorama, configobj, enum34, et-xmlfile, freetype-py, funcsigs, fusepy, ggplot, html5lib, ipaddress, jdcal, Jinja2, llvmlite, lxml, MarkupSafe, mpld3, msgpack-python, ndg-httpsclient, numba, openpyxl, pathlib2, Pillow, ply, pyasn1, pypng, python-geohash, scour, simplejson, and singledispatch.

  • The display function on Python ggplot objects is no longer supported because the ggplot package is not compatible with newer version of pandas.

  • Setting PYSPARK_PYTHON to /databricks/python2/bin/python is not supported because Databricks Runtime 6.0 does not support Python 2. A cluster with such setting can still start. However, Python notebooks and Python commands will not work, that is, Python command cells will fail with a “Canceled” error and a Python shell failed to start error will appear in the driver logs.

  • If PYSPARK_PYTHON points to a Python executable that is in an environment managed by Virtualenv , this environment will be activated for init scripts, SSH sessions, and notebooks. You can use python and pip commands that are defined in the activated environment directly without having to specify the absolute locations of these commands. By default, PYSPARK_PYTHON is set to /databricks/python3/bin/python. Thus, by default, python points to /databricks/python3/bin/python and pip points to /databricks/python3/bin/pip for init scripts, SSH sessions, and notebooks. If your PYSPARK_PYTHON points to a Python executable that is not in an environment managed by Virtualenv or if you are writing an init script to create the Python specified by PYSPARK_PYTHON, you will need to use absolute paths to access the correct python and pip. When Python library isolation is enabled (it is enabled by default), the activated environment is still the environment that PYSPARK_PYTHON is associated with. We recommend that you use Library utility (dbutils.library) to modify the isolated environment associated with a Python notebook.

Scala and Java APIs for Delta Lake DML commands

You can now modify data in Delta tables using programmatic APIs for delete, update, and merge. These APIs mirror the syntax and semantics of their corresponding SQL commands and are great for many workloads, for example, slowly changing dimension (SCD) operations, merging change data for replication, and upserts from streaming queries.

For details, see Table deletes, updates, and merges.

Scala and Java APIs for Delta Lake utility commands

Databricks Runtime now has programmatic APIs for the vacuum and history utility commands. These APIs mirror the syntax and semantics of their corresponding SQL commands available in earlier versions of Databricks Runtime.

You can clean up files that are no longer referenced by a Delta table and are older than the retention threshold by running vacuum on the table. Running the vacuum command on the table recursively vacuums the directories associated with the Delta table. The default retention threshold for the files is 7 days. The ability to time travel back to a version older than the retention period is lost after running vacuum. vacuum is not triggered automatically.

You can retrieve information on the operations, user, timestamp, and so on for each write to a Delta table by running the history command. The operations are returned in reverse chronological order. By default, table history is retained for 30 days.

For details, see Table utility commands.

Optimized storage using local file APIs

Local file APIs are useful as they allow you to access files from underlying distributed object storage as local files. In Databricks Runtime 6.0, we enhanced the FUSE mount that enables local file APIs to address key limitations. Databricks Runtime 6.0 significantly improves read and write speed and supports files that are larger than 2 GB. If you need faster and more reliable reads and writes such as for distributed model training, you would find this enhancement particularly useful. In addition, you would not need to load data to a local storage for your workloads, both saving cost and improving productivity.

For details, see Local file APIs.

Multiple matplotlib plots per notebook cell

You can now display multiple matplotlib plots per notebook cell:

Multiple matplotlib plots in cell

Service credentials for multiple Azure Data Lake Storage Gen1 accounts

You can now set up service credentials for multiple Azure Data Lake Storage Gen1 accounts for use within in a single Apache Spark session. To do this, add account.<account-name> to the configuration keys. For example, if you want to set up credentials for accounts to access both adl://example1.azuredatalakestore.net and adl://example2.azuredatalakestore.net, you can do this as follows:

spark.conf.set("fs.adl.oauth2.access.token.provider.type", "ClientCredential")

spark.conf.set("fs.adl.account.example1.oauth2.client.id", "<application-id-example1>")
spark.conf.set("fs.adl.account.example1.oauth2.credential", dbutils.secrets.get(scope = "<scope-name>", key = "<key-name-for-service-credential-example1>"))
spark.conf.set("fs.adl.account.example1.oauth2.refresh.url", "https://login.microsoftonline.com/<directory-id-example1>/oauth2/token")

spark.conf.set("fs.adl.account.example2.oauth2.client.id", "<application-id-example2>")
spark.conf.set("fs.adl.account.example2.oauth2.credential", dbutils.secrets.get(scope = "<scope-name>", key = "<key-name-for-service-credential-example2>"))
spark.conf.set("fs.adl.account.example2.oauth2.refresh.url", "https://login.microsoftonline.com/<directory-id-example2>/oauth2/token")

Improvements

  • Upgraded AWS SDK to 1.11.596.

  • Upgraded Azure Storage SDK in the WASB driver to 7.0.

  • OPTIMIZE now provides a summary of metrics such as number of files added, number of files removed, and max and min file size. See Compaction (bin-packing).

Removal

Databricks ML Model Export is removed. Use MLeap for importing and exporting models instead.

Apache Spark

Note

This article contains references to the term slave, a term that Databricks no longer uses. When the term is removed from the software, we’ll remove it from this article.

Databricks Runtime 6.0 includes Apache Spark 2.4.3. This release includes all Spark fixes and improvements included in Databricks Runtime 5.5 LTS (Unsupported), as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-27992][SPARK-28881][PYTHON] Allow Python to join with connection thread to propagate errors

  • [SPARK-27330][SS]support task abort in foreach writer (6.0, 5.x)

  • [SPARK-28642][SQL] Hide credentials in SHOW CREATE TABLE

  • [SPARK-28699][CORE] Fix a corner case for aborting indeterminate stage

  • [SPARK-28647][WEBUI] Recover additional metric feature

  • [SPARK-28766][R][DOC] Fix CRAN incoming feasibility warning on invalid URL

  • [SPARK-28486][CORE][PYTHON] Map PythonBroadcast’s data file to a BroadcastBlock to avoid delete by GC

  • [SPARK-25035][CORE] Avoiding memory mapping at disk-stored blocks replication

  • [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs)

  • [SPARK-28638][WEBUI] Task summary should only contain successful tasks’ metrics

  • [SPARK-28153][PYTHON] Use AtomicReference at InputFileBlockHolder (to support input_file_name with Python UDF)

  • [SPARK-28564][CORE] Access history application defaults to the last attempt id

  • [SPARK-28260] Cluster can auto-terminate while thriftserver query is still fetching results

  • [SPARK-26152][CORE] Synchronize Worker Cleanup with Worker Shutdown

  • [SPARK-28545][SQL] Add the hash map size to the directional log of ObjectAggregationIterator

  • [SPARK-28489][SS] Fix a bug that KafkaOffsetRangeCalculator.getRanges may drop offsets

  • [SPARK-28421][ML] SparseVector.apply performance optimization

  • [SPARK-28156][SQL] Self-join should not miss cached view

  • [SPARK-28152][SQL] Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect

  • [SPARK-28054][SQL] Fix error when insert Hive partitioned table dynamically where partition name is upper case

  • [SPARK-27159][SQL] update mssql server dialect to support binary type

  • [SPARK-28355][CORE][PYTHON] Use Spark conf for threshold at which com…

  • [SPARK-27989][CORE] Added retries on the connection to the driver for k8s

  • [SPARK-27416][SQL] UnsafeMapData & UnsafeArrayData Kryo serialization …

  • [SPARK-28430][UI] Fix stage table rendering when some tasks’ metrics are missing

  • [SPARK-27485] EnsureRequirements.reorder should handle duplicate expressions gracefully

  • [SPARK-28404][SS] Fix negative timeout value in RateStreamContinuousPartitionReader

  • [SPARK-28378][PYTHON] Remove usage of cgi.escape

  • [SPARK-28371][SQL] Make Parquet “StartsWith” filter null-safe

  • [SPARK-28015][SQL] Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

  • [SPARK-28302][CORE] Make sure to generate unique output file for SparkLauncher on Windows

  • [SPARK-28308][CORE] CalendarInterval sub-second part should be padded before parsing

  • [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation

  • [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed

  • [SPARK-27839][SQL] Change UTF8String.replace() to operate on UTF8 bytes

  • [SPARK-28157][CORE] Make SHS clear KVStore LogInfo for the blacklisted entries

  • [SPARK-28128][PYTHON][SQL] pandas Grouped UDFs skip empty partitions

  • [SPARK-28012][SQL] Hive UDF supports struct type foldable expression

  • [SPARK-28164] Fix usage description of start-slave.sh

  • [SPARK-27100][SQL] Use Array instead of Seq in FilePartition to prevent StackOverflowError

  • [SPARK-28154][ML] GMM fix double caching

System environment

  • Operating System: Ubuntu 16.04.6 LTS

  • Java: 1.8.0_232

  • Scala: 2.11.12

  • Python: 3.7.3

  • R: R version 3.6.1 (2019-07-05)

  • Delta Lake: 0.3.0

Note

Although Scala 2.12 is available as an experimental feature in Apache Spark 2.4, it is not supported in Databricks Runtime 6.0.

Installed Python libraries

Library

Version

Library

Version

Library

Version

asn1crypto

0.24.0

backcall

0.1.0

boto

2.49.0

boto3

1.9.162

botocore

1.12.163

certifi

2019.3.9

cffi

1.12.2

chardet

3.0.4

cryptography

2.6.1

cycler

0.10.0

Cython

0.29.6

decorator

4.4.0

docutils

0.14

idna

2.8

ipython

7.4.0

ipython-genutils

0.2.0

jedi

0.13.3

jmespath

0.9.4

kiwisolver

1.1.0

matplotlib

3.0.3

numpy

1.16.2

pandas

0.24.2

parso

0.3.4

patsy

0.5.1

pexpect

4.6.0

pickleshare

0.7.5

pip

19.0.3

prompt-toolkit

2.0.9

psycopg2

2.7.6.1

ptyprocess

0.6.0

pyarrow

0.13.0

pycparser

2.19

pycurl

7.43.0

Pygments

2.3.1

PyGObject

3.20.0

pyOpenSSL

19.0.0

pyparsing

2.4.2

PySocks

1.6.8

python-apt

1.1.0.b1+ubuntu0.16.04.5

python-dateutil

2.8.0

pytz

2018.9

requests

2.21.0

s3transfer

0.2.1

scikit-learn

0.20.3

scipy

1.2.1

seaborn

0.9.0

setuptools

40.8.0

six

1.12.0

ssh-import-id

5.5

statsmodels

0.9.0

traitlets

4.3.2

unattended-upgrades

0.1

urllib3

1.24.1

virtualenv

16.4.1

wcwidth

0.1.7

wheel

0.33.1

Installed R libraries

Library

Version

Library

Version

Library

Version

abind

1.4-5

askpass

1.1

assertthat

0.2.1

backports

1.1.3

base

3.6.1

base64enc

0.1-3

BH

1.69.0-1

bit

1.1-14

bit64

0.9-7

bitops

1.0-6

blob

1.1.1

boot

1.3-23

brew

1.0-6

callr

3.2.0

car

3.0-2

carData

3.0-2

caret

6.0-82

cellranger

1.1.0

chron

2.3-53

class

7.3-15

cli

1.1.0

clipr

0.5.0

clisymbols

1.2.0

cluster

2.1.0

codetools

0.2-16

colorspace

1.4-1

commonmark

1.7

compiler

3.6.1

config

0.3

crayon

1.3.4

curl

3.3

data.table

1.12.0

datasets

3.6.1

DBI

1.0.0

dbplyr

1.3.0

desc

1.2.0

devtools

2.0.1

digest

0.6.18

doMC

1.3.5

dplyr

0.8.0.1

ellipsis

0.1.0

fansi

0.4.0

forcats

0.4.0

foreach

1.4.4

foreign

0.8-72

forge

0.2.0

fs

1.2.7

gbm

2.1.5

generics

0.0.2

ggplot2

3.1.0

gh

1.0.1

git2r

0.25.2

glmnet

2.0-16

glue

1.3.1

gower

0.2.0

graphics

3.6.1

grDevices

3.6.1

grid

3.6.1

gridExtra

2.3

gsubfn

0.7

gtable

0.3.0

h2o

3.22.1.1

haven

2.1.0

hms

0.4.2

htmltools

0.3.6

htmlwidgets

1.3

httr

1.4.0

hwriter

1.3.2

hwriterPlus

1.0-3

ini

0.3.1

ipred

0.9-8

iterators

1.0.10

jsonlite

1.6

KernSmooth

2.23-15

labeling

0.3

lattice

0.20-38

lava

1.6.5

lazyeval

0.2.2

littler

0.3.7

lme4

1.1-21

lubridate

1.7.4

magrittr

1.5

mapproj

1.2.6

maps

3.3.0

maptools

0.9-5

MASS

7.3-51.4

Matrix

1.2-17

MatrixModels

0.4-1

memoise

1.1.0

methods

3.6.1

mgcv

1.8-28

mime

0.6

minqa

1.2.4

ModelMetrics

1.2.2

munsell

0.5.0

mvtnorm

1.0-10

nlme

3.1-141

nloptr

1.2.1

nnet

7.3-12

numDeriv

2016.8-1

openssl

1.3

openxlsx

4.1.0

parallel

3.6.1

pbkrtest

0.4-7

pillar

1.3.1

pkgbuild

1.0.3

pkgconfig

2.0.2

pkgKitten

0.1.4

pkgload

1.0.2

plogr

0.2.0

plyr

1.8.4

praise

1.0.0

prettyunits

1.0.2

pROC

1.14.0

processx

3.3.0

prodlim

2018.04.18

progress

1.2.0

proto

1.0.0

ps

1.3.0

purrr

0.3.2

quantreg

5.38

R.methodsS3

1.7.1

R.oo

1.22.0

R.utils

2.8.0

r2d3

0.2.3

R6

2.4.0

randomForest

4.6-14

rappdirs

0.3.1

rcmdcheck

1.3.2

RColorBrewer

1.1-2

Rcpp

1.0.1

RcppEigen

0.3.3.5.0

RcppRoll

0.3.0

RCurl

1.95-4.12

readr

1.3.1

readxl

1.3.1

recipes

0.1.5

rematch

1.0.1

remotes

2.0.2

reshape2

1.4.3

rio

0.5.16

rlang

0.3.3

RODBC

1.3-15

roxygen2

6.1.1

rpart

4.1-15

rprojroot

1.3-2

Rserve

1.8-6

RSQLite

2.1.1

rstudioapi

0.10

scales

1.0.0

sessioninfo

1.1.1

sp

1.3-1

sparklyr

1.0.0

SparkR

2.4.4

SparseM

1.77

spatial

7.3-11

splines

3.6.1

sqldf

0.4-11

SQUAREM

2017.10-1

statmod

1.4.30

stats

3.6.1

stats4

3.6.1

stringi

1.4.3

stringr

1.4.0

survival

2.44-1.1

sys

3.1

tcltk

3.6.1

TeachingDemos

2.10

testthat

2.0.1

tibble

2.1.1

tidyr

0.8.3

tidyselect

0.2.5

timeDate

3043.102

tools

3.6.1

usethis

1.4.0

utf8

1.1.4

utils

3.6.1

viridisLite

0.3.0

whisker

0.3-2

withr

2.1.2

xml2

1.2.0

xopen

1.0.0

yaml

2.2.0

zip

2.0.1

Installed Java and Scala libraries (Scala 2.11 cluster version)

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.8.10

com.amazonaws

aws-java-sdk-autoscaling

1.11.595

com.amazonaws

aws-java-sdk-cloudformation

1.11.595

com.amazonaws

aws-java-sdk-cloudfront

1.11.595

com.amazonaws

aws-java-sdk-cloudhsm

1.11.595

com.amazonaws

aws-java-sdk-cloudsearch

1.11.595

com.amazonaws

aws-java-sdk-cloudtrail

1.11.595

com.amazonaws

aws-java-sdk-cloudwatch

1.11.595

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.11.595

com.amazonaws

aws-java-sdk-codedeploy

1.11.595

com.amazonaws

aws-java-sdk-cognitoidentity

1.11.595

com.amazonaws

aws-java-sdk-cognitosync

1.11.595

com.amazonaws

aws-java-sdk-config

1.11.595

com.amazonaws

aws-java-sdk-core

1.11.595

com.amazonaws

aws-java-sdk-datapipeline

1.11.595

com.amazonaws

aws-java-sdk-directconnect

1.11.595

com.amazonaws

aws-java-sdk-directory

1.11.595

com.amazonaws

aws-java-sdk-dynamodb

1.11.595

com.amazonaws

aws-java-sdk-ec2

1.11.595

com.amazonaws

aws-java-sdk-ecs

1.11.595

com.amazonaws

aws-java-sdk-efs

1.11.595

com.amazonaws

aws-java-sdk-elasticache

1.11.595

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.11.595

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.11.595

com.amazonaws

aws-java-sdk-elastictranscoder

1.11.595

com.amazonaws

aws-java-sdk-emr

1.11.595

com.amazonaws

aws-java-sdk-glacier

1.11.595

com.amazonaws

aws-java-sdk-glue

1.11.595

com.amazonaws

aws-java-sdk-iam

1.11.595

com.amazonaws

aws-java-sdk-importexport

1.11.595

com.amazonaws

aws-java-sdk-kinesis

1.11.595

com.amazonaws

aws-java-sdk-kms

1.11.595

com.amazonaws

aws-java-sdk-lambda

1.11.595

com.amazonaws

aws-java-sdk-logs

1.11.595

com.amazonaws

aws-java-sdk-machinelearning

1.11.595

com.amazonaws

aws-java-sdk-opsworks

1.11.595

com.amazonaws

aws-java-sdk-rds

1.11.595

com.amazonaws

aws-java-sdk-redshift

1.11.595

com.amazonaws

aws-java-sdk-route53

1.11.595

com.amazonaws

aws-java-sdk-s3

1.11.595

com.amazonaws

aws-java-sdk-ses

1.11.595

com.amazonaws

aws-java-sdk-simpledb

1.11.595

com.amazonaws

aws-java-sdk-simpleworkflow

1.11.595

com.amazonaws

aws-java-sdk-sns

1.11.595

com.amazonaws

aws-java-sdk-sqs

1.11.595

com.amazonaws

aws-java-sdk-ssm

1.11.595

com.amazonaws

aws-java-sdk-storagegateway

1.11.595

com.amazonaws

aws-java-sdk-sts

1.11.595

com.amazonaws

aws-java-sdk-support

1.11.595

com.amazonaws

aws-java-sdk-swf-libraries

1.11.22

com.amazonaws

aws-java-sdk-workspaces

1.11.595

com.amazonaws

jmespath-java

1.11.595

com.carrotsearch

hppc

0.7.2

com.chuusai

shapeless_2.11

2.3.2

com.clearspring.analytics

stream

2.7.0

com.databricks

Rserve

1.8-3

com.databricks

dbml-local_2.11

0.5.0-db8-spark2.4

com.databricks

dbml-local_2.11-tests

0.5.0-db8-spark2.4

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

compilerplugin_2.11

0.4.15-9

com.databricks.scalapb

scalapb-runtime_2.11

0.4.15-9

com.esotericsoftware

kryo-shaded

4.0.2

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.0.0

com.fasterxml.jackson.core

jackson-annotations

2.6.7

com.fasterxml.jackson.core

jackson-core

2.6.7

com.fasterxml.jackson.core

jackson-databind

2.6.7.1

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.6.7

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.6.7

com.fasterxml.jackson.module

jackson-module-paranamer

2.6.7

com.fasterxml.jackson.module

jackson-module-scala_2.11

2.6.7.1

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

core

1.1.2

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java-natives

1.1

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java-natives

1.1

com.github.fommil.netlib

netlib-native_ref-linux-x86_64-natives

1.1

com.github.fommil.netlib

netlib-native_system-linux-x86_64-natives

1.1

com.github.luben

zstd-jni

1.3.2-2

com.github.rwl

jtransforms

2.4.0

com.google.code.findbugs

jsr305

2.0.1

com.google.code.gson

gson

2.2.4

com.google.guava

guava

15.0

com.google.protobuf

protobuf-java

2.6.1

com.googlecode.javaewah

JavaEWAH

0.3.2

com.h2database

h2

1.3.174

com.jcraft

jsch

0.1.50

com.jolbox

bonecp

0.8.0.RELEASE

com.microsoft.azure

azure-data-lake-store-sdk

2.2.8

com.microsoft.azure

azure-storage

7.0.0

com.microsoft.sqlserver

mssql-jdbc

6.2.2.jre8

com.ning

compress-lzf

1.0.3

com.sun.mail

javax.mail

1.5.2

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.11

0.3

com.twitter

chill-java

0.9.3

com.twitter

chill_2.11

0.9.3

com.twitter

parquet-hadoop-bundle

1.6.0

com.twitter

util-app_2.11

6.23.0

com.twitter

util-core_2.11

6.23.0

com.twitter

util-jvm_2.11

6.23.0

com.typesafe

config

1.2.1

com.typesafe.scala-logging

scala-logging-api_2.11

2.1.2

com.typesafe.scala-logging

scala-logging-slf4j_2.11

2.1.2

com.univocity

univocity-parsers

2.7.3

com.vlkan

flatbuffers

1.2.0-3f79e055

com.zaxxer

HikariCP

3.1.0

commons-beanutils

commons-beanutils

1.9.3

commons-cli

commons-cli

1.2

commons-codec

commons-codec

1.10

commons-collections

commons-collections

3.2.2

commons-configuration

commons-configuration

1.6

commons-dbcp

commons-dbcp

1.4

commons-digester

commons-digester

1.8

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.4

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-net

commons-net

3.1

commons-pool

commons-pool

1.5.4

info.ganglia.gmetric4j

gmetric4j

1.0.7

io.airlift

aircompressor

0.10

io.dropwizard.metrics

metrics-core

3.1.5

io.dropwizard.metrics

metrics-ganglia

3.1.5

io.dropwizard.metrics

metrics-graphite

3.1.5

io.dropwizard.metrics

metrics-healthchecks

3.1.5

io.dropwizard.metrics

metrics-jetty9

3.1.5

io.dropwizard.metrics

metrics-json

3.1.5

io.dropwizard.metrics

metrics-jvm

3.1.5

io.dropwizard.metrics

metrics-log4j

3.1.5

io.dropwizard.metrics

metrics-servlets

3.1.5

io.netty

netty

3.9.9.Final

io.netty

netty-all

4.1.17.Final

javax.activation

activation

1.1.1

javax.annotation

javax.annotation-api

1.2

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.servlet

javax.servlet-api

3.1.0

javax.servlet.jsp

jsp-api

2.1

javax.transaction

jta

1.1

javax.validation

validation-api

1.1.0.Final

javax.ws.rs

javax.ws.rs-api

2.0.1

javax.xml.bind

jaxb-api

2.2.2

javax.xml.stream

stax-api

1.0-2

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.9.3

junit

junit

4.12

log4j

apache-log4j-extras

1.2.17

log4j

log4j

1.2.17

net.hydromatic

eigenbase-properties

1.1.5

net.razorvine

pyrolite

4.13

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.5

net.snowflake

snowflake-jdbc

3.6.15

net.snowflake

spark-snowflake_2.11

2.4.10-spark_2.4

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt

oncrpc

1.0.7

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.4

org.antlr

antlr4-runtime

4.7

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.9.2

org.apache.ant

ant-jsch

1.9.2

org.apache.ant

ant-launcher

1.9.2

org.apache.arrow

arrow-format

0.10.0

org.apache.arrow

arrow-memory

0.10.0

org.apache.arrow

arrow-vector

0.10.0

org.apache.avro

avro

1.8.2

org.apache.avro

avro-ipc

1.8.2

org.apache.avro

avro-mapred-hadoop2

1.8.2

org.apache.calcite

calcite-avatica

1.2.0-incubating

org.apache.calcite

calcite-core

1.2.0-incubating

org.apache.calcite

calcite-linq4j

1.2.0-incubating

org.apache.commons

commons-compress

1.8.1

org.apache.commons

commons-crypto

1.0.0

org.apache.commons

commons-lang3

3.5

org.apache.commons

commons-math3

3.4.1

org.apache.curator

curator-client

2.7.1

org.apache.curator

curator-framework

2.7.1

org.apache.curator

curator-recipes

2.7.1

org.apache.derby

derby

10.12.1.1

org.apache.directory.api

api-asn1-api

1.0.0-M20

org.apache.directory.api

api-util

1.0.0-M20

org.apache.directory.server

apacheds-i18n

2.0.0-M15

org.apache.directory.server

apacheds-kerberos-codec

2.0.0-M15

org.apache.hadoop

hadoop-annotations

2.7.3

org.apache.hadoop

hadoop-auth

2.7.3

org.apache.hadoop

hadoop-client

2.7.3

org.apache.hadoop

hadoop-common

2.7.3

org.apache.hadoop

hadoop-hdfs

2.7.3

org.apache.hadoop

hadoop-mapreduce-client-app

2.7.3

org.apache.hadoop

hadoop-mapreduce-client-common

2.7.3

org.apache.hadoop

hadoop-mapreduce-client-core

2.7.3

org.apache.hadoop

hadoop-mapreduce-client-jobclient

2.7.3

org.apache.hadoop

hadoop-mapreduce-client-shuffle

2.7.3

org.apache.hadoop

hadoop-yarn-api

2.7.3

org.apache.hadoop

hadoop-yarn-client

2.7.3

org.apache.hadoop

hadoop-yarn-common

2.7.3

org.apache.hadoop

hadoop-yarn-server-common

2.7.3

org.apache.htrace

htrace-core

3.1.0-incubating

org.apache.httpcomponents

httpclient

4.5.6

org.apache.httpcomponents

httpcore

4.4.10

org.apache.ivy

ivy

2.4.0

org.apache.orc

orc-core-nohive

1.5.5

org.apache.orc

orc-mapreduce-nohive

1.5.5

org.apache.orc

orc-shims

1.5.5

org.apache.parquet

parquet-column

1.10.1.2-databricks3

org.apache.parquet

parquet-common

1.10.1.2-databricks3

org.apache.parquet

parquet-encoding

1.10.1.2-databricks3

org.apache.parquet

parquet-format

2.4.0

org.apache.parquet

parquet-hadoop

1.10.1.2-databricks3

org.apache.parquet

parquet-jackson

1.10.1.2-databricks3

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.9.3

org.apache.xbean

xbean-asm6-shaded

4.8

org.apache.zookeeper

zookeeper

3.4.6

org.codehaus.jackson

jackson-core-asl

1.9.13

org.codehaus.jackson

jackson-jaxrs

1.9.13

org.codehaus.jackson

jackson-mapper-asl

1.9.13

org.codehaus.jackson

jackson-xc

1.9.13

org.codehaus.janino

commons-compiler

3.0.10

org.codehaus.janino

janino

3.0.10

org.datanucleus

datanucleus-api-jdo

3.2.6

org.datanucleus

datanucleus-core

3.2.10

org.datanucleus

datanucleus-rdbms

3.2.9

org.eclipse.jetty

jetty-client

9.3.27.v20190418

org.eclipse.jetty

jetty-continuation

9.3.27.v20190418

org.eclipse.jetty

jetty-http

9.3.27.v20190418

org.eclipse.jetty

jetty-io

9.3.27.v20190418

org.eclipse.jetty

jetty-jndi

9.3.27.v20190418

org.eclipse.jetty

jetty-plus

9.3.27.v20190418

org.eclipse.jetty

jetty-proxy

9.3.27.v20190418

org.eclipse.jetty

jetty-security

9.3.27.v20190418

org.eclipse.jetty

jetty-server

9.3.27.v20190418

org.eclipse.jetty

jetty-servlet

9.3.27.v20190418

org.eclipse.jetty

jetty-servlets

9.3.27.v20190418

org.eclipse.jetty

jetty-util

9.3.27.v20190418

org.eclipse.jetty

jetty-webapp

9.3.27.v20190418

org.eclipse.jetty

jetty-xml

9.3.27.v20190418

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.4.0-b34

org.glassfish.hk2

hk2-locator

2.4.0-b34

org.glassfish.hk2

hk2-utils

2.4.0-b34

org.glassfish.hk2

osgi-resource-locator

1.0.1

org.glassfish.hk2.external

aopalliance-repackaged

2.4.0-b34

org.glassfish.hk2.external

javax.inject

2.4.0-b34

org.glassfish.jersey.bundles.repackaged

jersey-guava

2.22.2

org.glassfish.jersey.containers

jersey-container-servlet

2.22.2

org.glassfish.jersey.containers

jersey-container-servlet-core

2.22.2

org.glassfish.jersey.core

jersey-client

2.22.2

org.glassfish.jersey.core

jersey-common

2.22.2

org.glassfish.jersey.core

jersey-server

2.22.2

org.glassfish.jersey.media

jersey-media-jaxb

2.22.2

org.hamcrest

hamcrest-core

1.3

org.hamcrest

hamcrest-library

1.3

org.hibernate

hibernate-validator

5.1.1.Final

org.iq80.snappy

snappy

0.2

org.javassist

javassist

3.18.1-GA

org.jboss.logging

jboss-logging

3.1.3.GA

org.jdbi

jdbi

2.63.1

org.joda

joda-convert

1.7

org.jodd

jodd-core

3.5.2

org.json4s

json4s-ast_2.11

3.5.3

org.json4s

json4s-core_2.11

3.5.3

org.json4s

json4s-jackson_2.11

3.5.3

org.json4s

json4s-scalap_2.11

3.5.3

org.lz4

lz4-java

1.4.0

org.mariadb.jdbc

mariadb-java-client

2.1.2

org.mockito

mockito-core

1.10.19

org.objenesis

objenesis

2.5.1

org.postgresql

postgresql

42.1.4

org.roaringbitmap

RoaringBitmap

0.7.45

org.roaringbitmap

shims

0.7.45

org.rocksdb

rocksdbjni

6.2.2

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.11

2.11.12

org.scala-lang

scala-library_2.11

2.11.12

org.scala-lang

scala-reflect_2.11

2.11.12

org.scala-lang.modules

scala-parser-combinators_2.11

1.1.0

org.scala-lang.modules

scala-xml_2.11

1.0.5

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.11

1.12.5

org.scalactic

scalactic_2.11

3.0.3

org.scalanlp

breeze-macros_2.11

0.13.2

org.scalanlp

breeze_2.11

0.13.2

org.scalatest

scalatest_2.11

3.0.3

org.slf4j

jcl-over-slf4j

1.7.16

org.slf4j

jul-to-slf4j

1.7.16

org.slf4j

slf4j-api

1.7.16

org.slf4j

slf4j-log4j12

1.7.16

org.spark-project.hive

hive-beeline

1.2.1.spark2

org.spark-project.hive

hive-cli

1.2.1.spark2

org.spark-project.hive

hive-jdbc

1.2.1.spark2

org.spark-project.hive

hive-metastore

1.2.1.spark2

org.spark-project.spark

unused

1.0.0

org.spire-math

spire-macros_2.11

0.13.0

org.spire-math

spire_2.11

0.13.0

org.springframework

spring-core

4.1.4.RELEASE

org.springframework

spring-test

4.1.4.RELEASE

org.tukaani

xz

1.5

org.typelevel

machinist_2.11

0.6.1

org.typelevel

macro-compat_2.11

1.1.1

org.xerial

sqlite-jdbc

3.8.11.2

org.xerial.snappy

snappy-java

1.1.7.3

org.yaml

snakeyaml

1.16

oro

oro

2.0.8

software.amazon.ion

ion-java

1.0.2

stax

stax-api

1.0.1

xmlenc

xmlenc

0.52