Databricks Runtime 7.6 (Unsupported)

Databricks released this image in February 2021.

The following release notes provide information about Databricks Runtime 7.6, powered by Apache Spark 3.0.

For help with migration from Databricks Runtime 6.x, see Databricks Runtime 7.x migration guide.

New features

Better JSON schema evolution handling in Auto Loader

When loading JSON data, the default behavior is that fields that are not part of the original schema are silently dropped. Now with Auto Loader, you can set the JSON option unparsedDataColumn to specify the name of a column, which will be used to store all the new fields as a JSON string. For example, by setting

.option("unparsedDataColumn", "_unparsed_data")

the result will have a column _unparsed_data, where new fields are stored as a JSON string. These new fields can be parsed later for further processing. If there are no new fields, the column value is null.

Support for reading overwritten files in Auto Loader

You can now configure streams to allow input directory file changes to overwrite existing data. Set .option("cloudFiles.allowOverwrites", "true"). For details, see What is Auto Loader?.

Improvements

Better multi-dimensional clustering and data skipping performance with OPTIMIZE ZORDER

The OPTIMIZE ZORDER operation now uses Hilbert space-filling curves by default. This approach provides better clustering characteristics than Z-order in higher dimensions. For Delta tables using OPTIMIZE ZORDER with many columns, Hilbert curves can speed up read queries by skipping more data than Z-order. For details, see Z-Ordering (multi-dimensional clustering).

Performance improvements for some MERGE operations with only MATCHED clauses

You may get better performance in MERGE operations that have only MATCHED clauses with clause conditions.

Auto-tuning file sizes for MERGE-heavy Delta tables (Public Preview)

Delta tables that undergo repeated MERGE operations (for example, continuous upserts of row-level change data) repeatedly rewrite files to update data. Such tables can benefit from having smaller files than those produced by default by data layout operations like OPTIMIZE, Optimized Write, and Auto Compaction. With Databricks Runtime 7.6, you can automatically detect MERGE-heavy tables and tune file sizes. Contact Databricks support or your Databricks representative for more information on how to enable this feature.

Multi-cluster writes in SparkR

Multi-cluster writes now support SparkR. You can write to a Delta table using SparkR from multiple clusters in the same workspace with full ACID guarantees.

Library upgrades

  • Upgraded Python libraries:

    • certifi upgraded from 2020.6.20 to 2020.12.5.

    • koalas upgraded from 1.4.0 to 1.5.0.

    • python-apt upgraded from 1.6.5+ubuntu0.3 to 1.6.5+ubuntu0.4

  • Upgraded R library:

    • nlme from 3.1-150 to 3.1-151.

  • Upgraded Java library:

    • azure-data-lake-store-sdk upgraded from 2.3.8 to 2.3.9.

  • OpenJDK 8 build upgraded to Zulu 8.50.0.51-CA-linux64 (build 1.8.0_275-b01).

Apache Spark

Databricks Runtime 7.6 includes Apache Spark 3.0.1. This release includes all Spark fixes and improvements included in Databricks Runtime 7.5 (Unsupported), as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-33740] [SQL]hadoop configs in hive-site.xml can overrides pre-existing hadoop ones

  • [SPARK-33822] [SQL] Use the CastSupport.cast method in HashJoin

  • [SPARK-33398] Fix loading tree models prior to Spark 3.0

  • [SPARK-33029] [CORE][WEBUI] Fix to UI executor page incorrectly marking the driver as blocklisted

  • [SPARK-34012] [SQL] Keep behavior consistent when conf spark.sqllegacy.parser.havingWithoutGroupByAsWhere is true with migration guide

  • [SPARK-33635] [SS] Adjust the order of check in KafkaTokenUtil.needTokenUpdate to remedy perf regression

  • [SPARK-33900] [WEBUI] Show shuffle read size / records correctly when only remotebytesread is available

  • [SPARK-33593] [SQL] Vector reader got incorrect data with binary partition value

  • [SPARK-31170] [SQL] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir

  • [SPARK-33853] [SQL] EXPLAIN CODEGEN and BenchmarkQueryTest don’t show subquery code

  • [SPARK-33786] [SQL] The storage level for a cache should be respected when a table name is altered.

  • [SPARK-33841] [CORE] Fix issue with jobs disappearing intermittently from the SHS under high load

  • [SPARK-33480] [SQL] do not expose user data in error message

  • [SPARK-33756] [SQL] Make BytesToBytesMap’s MapIterator idempotent

  • [SPARK-33774] [UI][CORE]”Back to Master” returns 500 error in Standalone cluster

  • [SPARK-33733] [SQL] PullOutNondeterministic should check and collect deterministic field

  • [SPARK-33670] [SQL] Verify the partition provider is Hive in v1 SHOW TABLE EXTENDED

  • [SPARK-33788] [SQL][3.1][2.4] Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

  • [SPARK-33819] [CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be package private

  • [SPARK-33742] [SQL] Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

  • [SPARK-32110] [SQL] normalize special floating numbers in HyperLogLog++

  • [SPARK-33071] [SPARK-33536] [SQL] Do not change Alias constructor to not break snowflake connector

  • [SPARK-32680] [SQL] Don’t Preprocess V2 CTAS with Unresolved Query

  • [SPARK-33520] [ML][PYSPARK] Cherry-pick: make CrossValidator/TrainValidateSplit/OneVsRest Reader/Writer support Python backend estimator/evaluator

  • [SPARK-33592] [ML][PYTHON]Pyspark ML Validator params in estimatorParamMaps may be lost after saving and reloading

  • [SPARK-33667] [SQL] Respect the spark.sql.caseSensitive config while resolving partition spec in v1 SHOW PARTITIONS

  • [SPARK-33141] [SQL] Capture SQL configs when creating permanent views

  • [SPARK-33636] [PYTHON][ML] Add labelsArray to PySpark StringIndexer

  • [SPARK-33629] [PYTHON]Make spark.buffer.size configuration visible on driver side

  • [SPARK-33389] [SQL] Make internal classes of SparkSession always using active SQLConf

  • [SPARK-33071] [SPARK-33536] [SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

  • [SPARK-33551] [SQL] Do not use custom shuffle reader for repartition

  • [SPARK-33432] [SQL] SQL parser should use active SQLConf

  • [SPARK-33588] [SQL] Respect the spark.sql.caseSensitive config while resolving partition spec in v1 SHOW TABLE EXTENDED

  • [SPARK-33440] [CORE] Use current timestamp with warning log in HadoopFSDelegationTokenProvider when the issue date for token is not set up properly

  • [SPARK-33494] [SQL][AQE] Do not use local shuffle reader for repartition

System environment

  • Operating System: Ubuntu 18.04.5 LTS

  • Java: Zulu 8.50.0.51-CA-linux64 (build 1.8.0_275-b01)

  • Scala: 2.12.10

  • Python: 3.7.5

  • R: R version 3.6.3 (2020-02-29)

  • Delta Lake 0.7.0

Installed Python libraries

Library

Version

Library

Version

Library

Version

asn1crypto

1.3.0

backcall

0.1.0

boto3

1.12.0

botocore

1.15.0

certifi

2020.12.5

cffi

1.14.0

chardet

3.0.4

cryptography

2.8

cycler

0.10.0

Cython

0.29.15

decorator

4.4.1

docutils

0.15.2

entrypoints

0.3

idna

2.8

ipykernel

5.1.4

ipython

7.12.0

ipython-genutils

0.2.0

jedi

0.17.2

jmespath

0.10.0

joblib

0.14.1

jupyter-client

5.3.4

jupyter-core

4.6.1

kiwisolver

1.1.0

koalas

1.5.0

matplotlib

3.1.3

numpy

1.18.1

pandas

1.0.1

parso

0.7.0

patsy

0.5.1

pexpect

4.8.0

pickleshare

0.7.5

pip

20.0.2

prompt-toolkit

3.0.3

psycopg2

2.8.4

ptyprocess

0.6.0

pyarrow

1.0.1

pycparser

2.19

Pygments

2.5.2

PyGObject

3.26.1

pyOpenSSL

19.1.0

pyparsing

2.4.6

PySocks

1.7.1

python-apt

1.6.5+ubuntu0.4

python-dateutil

2.8.1

pytz

2019.3

pyzmq

18.1.1

requests

2.22.0

s3transfer

0.3.3

scikit-learn

0.22.1

scipy

1.4.1

seaborn

0.10.0

setuptools

45.2.0

six

1.14.0

ssh-import-id

5.7

statsmodels

0.11.0

tornado

6.0.3

traitlets

4.3.3

unattended-upgrades

0.1

urllib3

1.25.8

virtualenv

16.7.10

wcwidth

0.1.8

wheel

0.34.2

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2020-11-02.

Library

Version

Library

Version

Library

Version

askpass

1.1

assertthat

0.2.1

backports

1.1.10

base

3.6.3

base64enc

0.1-3

BH

1.72.0-3

bit

4.0.4

bit64

4.0.5

blob

1.2.1

boot

1.3-25

brew

1.0-6

brio

1.1.0

broom

0.7.2

callr

3.5.1

caret

6.0-86

cellranger

1.1.0

chron

2.3-56

class

7.3-17

cli

2.1.0

clipr

0.7.1

cluster

2.1.0

codetools

0.2-18

colorspace

1.4-1

commonmark

1.7

compiler

3.6.3

config

0.3

covr

3.5.1

cpp11

0.2.3

crayon

1.3.4

crosstalk

1.1.0.1

curl

4.3

data.table

1.13.2

datasets

3.6.3

DBI

1.1.0

dbplyr

1.4.4

desc

1.2.0

devtools

2.3.2

diffobj

0.3.2

digest

0.6.27

dplyr

0.8.5

DT

0.16

ellipsis

0.3.1

evaluate

0.14

fansi

0.4.1

farver

2.0.3

fastmap

1.0.1

forcats

0.5.0

foreach

1.5.1

foreign

0.8-76

forge

0.2.0

fs

1.5.0

generics

0.1.0

ggplot2

3.3.2

gh

1.1.0

git2r

0.27.1

glmnet

3.0-2

globals

0.13.1

glue

1.4.2

gower

0.2.2

graphics

3.6.3

grDevices

3.6.3

grid

3.6.3

gridExtra

2.3

gsubfn

0.7

gtable

0.3.0

haven

2.3.1

highr

0.8

hms

0.5.3

htmltools

0.5.0

htmlwidgets

1.5.2

httpuv

1.5.4

httr

1.4.2

hwriter

1.3.2

hwriterPlus

1.0-3

ini

0.3.1

ipred

0.9-9

isoband

0.2.2

iterators

1.0.13

jsonlite

1.7.1

KernSmooth

2.23-18

knitr

1.30

labeling

0.4.2

later

1.1.0.1

lattice

0.20-41

lava

1.6.8

lazyeval

0.2.2

lifecycle

0.2.0

lubridate

1.7.9

magrittr

1.5

markdown

1.1

MASS

7.3-53

Matrix

1.2-18

memoise

1.1.0

methods

3.6.3

mgcv

1.8-33

mime

0.9

ModelMetrics

1.2.2.2

modelr

0.1.8

munsell

0.5.0

nlme

3.1-151

nnet

7.3-14

numDeriv

2016.8-1.1

openssl

1.4.3

parallel

3.6.3

pillar

1.4.6

pkgbuild

1.1.0

pkgconfig

2.0.3

pkgload

1.1.0

plogr

0.2.0

plyr

1.8.6

praise

1.0.0

prettyunits

1.1.1

pROC

1.16.2

processx

3.4.4

prodlim

2019.11.13

progress

1.2.2

promises

1.1.1

proto

1.0.0

ps

1.4.0

purrr

0.3.4

r2d3

0.2.3

R6

2.5.0

randomForest

4.6-14

rappdirs

0.3.1

rcmdcheck

1.3.3

RColorBrewer

1.1-2

Rcpp

1.0.5

readr

1.4.0

readxl

1.3.1

recipes

0.1.14

rematch

1.0.1

rematch2

2.1.2

remotes

2.2.0

reprex

0.3.0

reshape2

1.4.4

rex

1.2.0

rjson

0.2.20

rlang

0.4.8

rmarkdown

2.5

RODBC

1.3-16

roxygen2

7.1.1

rpart

4.1-15

rprojroot

1.3-2

Rserve

1.8-7

RSQLite

2.2.1

rstudioapi

0.11

rversions

2.0.2

rvest

0.3.6

scales

1.1.1

selectr

0.4-2

sessioninfo

1.1.1

shape

1.4.4

shiny

1.5.0

sourcetools

0.1.7

sparklyr

1.4.0

SparkR

3.0.0

spatial

7.3-11

splines

3.6.3

sqldf

0.4-11

SQUAREM

2020.5

stats

3.6.3

stats4

3.6.3

stringi

1.5.3

stringr

1.4.0

survival

3.2-7

sys

3.4

tcltk

3.6.3

TeachingDemos

2.10

testthat

3.0.0

tibble

3.0.4

tidyr

1.1.2

tidyselect

1.1.0

tidyverse

1.3.0

timeDate

3043.102

tinytex

0.27

tools

3.6.3

usethis

1.6.3

utf8

1.1.4

utils

3.6.3

uuid

0.1-4

vctrs

0.3.4

viridisLite

0.3.0

waldo

0.2.2

whisker

0.4

withr

2.3.0

xfun

0.19

xml2

1.3.2

xopen

1.0.0

xtable

1.8-4

yaml

2.2.1

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.12.0

com.amazonaws

aws-java-sdk-autoscaling

1.11.655

com.amazonaws

aws-java-sdk-cloudformation

1.11.655

com.amazonaws

aws-java-sdk-cloudfront

1.11.655

com.amazonaws

aws-java-sdk-cloudhsm

1.11.655

com.amazonaws

aws-java-sdk-cloudsearch

1.11.655

com.amazonaws

aws-java-sdk-cloudtrail

1.11.655

com.amazonaws

aws-java-sdk-cloudwatch

1.11.655

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.11.655

com.amazonaws

aws-java-sdk-codedeploy

1.11.655

com.amazonaws

aws-java-sdk-cognitoidentity

1.11.655

com.amazonaws

aws-java-sdk-cognitosync

1.11.655

com.amazonaws

aws-java-sdk-config

1.11.655

com.amazonaws

aws-java-sdk-core

1.11.655

com.amazonaws

aws-java-sdk-datapipeline

1.11.655

com.amazonaws

aws-java-sdk-directconnect

1.11.655

com.amazonaws

aws-java-sdk-directory

1.11.655

com.amazonaws

aws-java-sdk-dynamodb

1.11.655

com.amazonaws

aws-java-sdk-ec2

1.11.655

com.amazonaws

aws-java-sdk-ecs

1.11.655

com.amazonaws

aws-java-sdk-efs

1.11.655

com.amazonaws

aws-java-sdk-elasticache

1.11.655

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.11.655

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.11.655

com.amazonaws

aws-java-sdk-elastictranscoder

1.11.655

com.amazonaws

aws-java-sdk-emr

1.11.655

com.amazonaws

aws-java-sdk-glacier

1.11.655

com.amazonaws

aws-java-sdk-iam

1.11.655

com.amazonaws

aws-java-sdk-importexport

1.11.655

com.amazonaws

aws-java-sdk-kinesis

1.11.655

com.amazonaws

aws-java-sdk-kms

1.11.655

com.amazonaws

aws-java-sdk-lambda

1.11.655

com.amazonaws

aws-java-sdk-logs

1.11.655

com.amazonaws

aws-java-sdk-machinelearning

1.11.655

com.amazonaws

aws-java-sdk-opsworks

1.11.655

com.amazonaws

aws-java-sdk-rds

1.11.655

com.amazonaws

aws-java-sdk-redshift

1.11.655

com.amazonaws

aws-java-sdk-route53

1.11.655

com.amazonaws

aws-java-sdk-s3

1.11.655

com.amazonaws

aws-java-sdk-ses

1.11.655

com.amazonaws

aws-java-sdk-simpledb

1.11.655

com.amazonaws

aws-java-sdk-simpleworkflow

1.11.655

com.amazonaws

aws-java-sdk-sns

1.11.655

com.amazonaws

aws-java-sdk-sqs

1.11.655

com.amazonaws

aws-java-sdk-ssm

1.11.655

com.amazonaws

aws-java-sdk-storagegateway

1.11.655

com.amazonaws

aws-java-sdk-sts

1.11.655

com.amazonaws

aws-java-sdk-support

1.11.655

com.amazonaws

aws-java-sdk-swf-libraries

1.11.22

com.amazonaws

aws-java-sdk-workspaces

1.11.655

com.amazonaws

jmespath-java

1.11.655

com.chuusai

shapeless_2.12

2.3.3

com.clearspring.analytics

stream

2.9.6

com.databricks

Rserve

1.8-3

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

compilerplugin_2.12

0.4.15-10

com.databricks.scalapb

scalapb-runtime_2.12

0.4.15-10

com.esotericsoftware

kryo-shaded

4.0.2

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.3.4

com.fasterxml.jackson.core

jackson-annotations

2.10.0

com.fasterxml.jackson.core

jackson-core

2.10.0

com.fasterxml.jackson.core

jackson-databind

2.10.0

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.10.0

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.10.0

com.fasterxml.jackson.module

jackson-module-paranamer

2.10.0

com.fasterxml.jackson.module

jackson-module-scala_2.12

2.10.0

com.github.ben-manes.caffeine

caffeine

2.3.4

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

core

1.1.2

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java-natives

1.1

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java-natives

1.1

com.github.fommil.netlib

netlib-native_ref-linux-x86_64-natives

1.1

com.github.fommil.netlib

netlib-native_system-linux-x86_64-natives

1.1

com.github.joshelser

dropwizard-metrics-hadoop-metrics2-reporter

0.1.2

com.github.luben

zstd-jni

1.4.4-3

com.github.wendykierp

JTransforms

3.1

com.google.code.findbugs

jsr305

3.0.0

com.google.code.gson

gson

2.2.4

com.google.flatbuffers

flatbuffers-java

1.9.0

com.google.guava

guava

15.0

com.google.protobuf

protobuf-java

2.6.1

com.h2database

h2

1.4.195

com.helger

profiler

1.1.1

com.jcraft

jsch

0.1.50

com.jolbox

bonecp

0.8.0.RELEASE

com.lihaoyi

sourcecode_2.12

0.1.9

com.microsoft.sqlserver

mssql-jdbc

8.2.1.jre8

com.microsoft.azure

azure-data-lake-store-sdk

2.3.9

com.ning

compress-lzf

1.0.3

com.sun.mail

javax.mail

1.5.2

com.tdunning

json

1.8

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.12

0.4.12

com.twitter

chill-java

0.9.5

com.twitter

chill_2.12

0.9.5

com.twitter

util-app_2.12

7.1.0

com.twitter

util-core_2.12

7.1.0

com.twitter

util-function_2.12

7.1.0

com.twitter

util-jvm_2.12

7.1.0

com.twitter

util-lint_2.12

7.1.0

com.twitter

util-registry_2.12

7.1.0

com.twitter

util-stats_2.12

7.1.0

com.typesafe

config

1.2.1

com.typesafe.scala-logging

scala-logging_2.12

3.7.2

com.univocity

univocity-parsers

2.9.0

com.zaxxer

HikariCP

3.1.0

commons-beanutils

commons-beanutils

1.9.4

commons-cli

commons-cli

1.2

commons-codec

commons-codec

1.10

commons-collections

commons-collections

3.2.2

commons-configuration

commons-configuration

1.6

commons-dbcp

commons-dbcp

1.4

commons-digester

commons-digester

1.8

commons-fileupload

commons-fileupload

1.3.3

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.4

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-net

commons-net

3.1

commons-pool

commons-pool

1.5.4

info.ganglia.gmetric4j

gmetric4j

1.0.10

io.airlift

aircompressor

0.10

io.dropwizard.metrics

metrics-core

4.1.1

io.dropwizard.metrics

metrics-graphite

4.1.1

io.dropwizard.metrics

metrics-healthchecks

4.1.1

io.dropwizard.metrics

metrics-jetty9

4.1.1

io.dropwizard.metrics

metrics-jmx

4.1.1

io.dropwizard.metrics

metrics-json

4.1.1

io.dropwizard.metrics

metrics-jvm

4.1.1

io.dropwizard.metrics

metrics-servlets

4.1.1

io.netty

netty-all

4.1.47.Final

io.prometheus

simpleclient

0.7.0

io.prometheus

simpleclient_common

0.7.0

io.prometheus

simpleclient_dropwizard

0.7.0

io.prometheus

simpleclient_pushgateway

0.7.0

io.prometheus

simpleclient_servlet

0.7.0

io.prometheus.jmx

collector

0.12.0

jakarta.annotation

jakarta.annotation-api

1.3.5

jakarta.validation

jakarta.validation-api

2.0.2

jakarta.ws.rs

jakarta.ws.rs-api

2.1.6

javax.activation

activation

1.1.1

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.servlet

javax.servlet-api

3.1.0

javax.servlet.jsp

jsp-api

2.1

javax.transaction

jta

1.1

javax.transaction

transaction-api

1.1

javax.xml.bind

jaxb-api

2.2.2

javax.xml.stream

stax-api

1.0-2

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.10.5

log4j

apache-log4j-extras

1.2.17

log4j

log4j

1.2.17

net.razorvine

pyrolite

4.30

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.6

net.snowflake

snowflake-jdbc

3.12.8

net.snowflake

spark-snowflake_2.12

2.8.1-spark_3.0

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt.remotetea

remotetea-oncrpc

1.1.2

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.5.2

org.antlr

antlr4-runtime

4.7.1

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.9.2

org.apache.ant

ant-jsch

1.9.2

org.apache.ant

ant-launcher

1.9.2

org.apache.arrow

arrow-format

0.15.1

org.apache.arrow

arrow-memory

0.15.1

org.apache.arrow

arrow-vector

0.15.1

org.apache.avro

avro

1.8.2

org.apache.avro

avro-ipc

1.8.2

org.apache.avro

avro-mapred-hadoop2

1.8.2

org.apache.commons

commons-compress

1.8.1

org.apache.commons

commons-crypto

1.0.0

org.apache.commons

commons-lang3

3.9

org.apache.commons

commons-math3

3.4.1

org.apache.commons

commons-text

1.6

org.apache.curator

curator-client

2.7.1

org.apache.curator

curator-framework

2.7.1

org.apache.curator

curator-recipes

2.7.1

org.apache.derby

derby

10.12.1.1

org.apache.directory.api

api-asn1-api

1.0.0-M20

org.apache.directory.api

api-util

1.0.0-M20

org.apache.directory.server

apacheds-i18n

2.0.0-M15

org.apache.directory.server

apacheds-kerberos-codec

2.0.0-M15

org.apache.hadoop

hadoop-annotations

2.7.4

org.apache.hadoop

hadoop-auth

2.7.4

org.apache.hadoop

hadoop-client

2.7.4

org.apache.hadoop

hadoop-common

2.7.4

org.apache.hadoop

hadoop-hdfs

2.7.4

org.apache.hadoop

hadoop-mapreduce-client-app

2.7.4

org.apache.hadoop

hadoop-mapreduce-client-common

2.7.4

org.apache.hadoop

hadoop-mapreduce-client-core

2.7.4

org.apache.hadoop

hadoop-mapreduce-client-jobclient

2.7.4

org.apache.hadoop

hadoop-mapreduce-client-shuffle

2.7.4

org.apache.hadoop

hadoop-yarn-api

2.7.4

org.apache.hadoop

hadoop-yarn-client

2.7.4

org.apache.hadoop

hadoop-yarn-common

2.7.4

org.apache.hadoop

hadoop-yarn-server-common

2.7.4

org.apache.hive

hive-beeline

2.3.7

org.apache.hive

hive-cli

2.3.7

org.apache.hive

hive-common

2.3.7

org.apache.hive

hive-exec-core

2.3.7

org.apache.hive

hive-jdbc

2.3.7

org.apache.hive

hive-llap-client

2.3.7

org.apache.hive

hive-llap-common

2.3.7

org.apache.hive

hive-metastore

2.3.7

org.apache.hive

hive-serde

2.3.7

org.apache.hive

hive-shims

2.3.7

org.apache.hive

hive-storage-api

2.7.1

org.apache.hive

hive-vector-code-gen

2.3.7

org.apache.hive.shims

hive-shims-0.23

2.3.7

org.apache.hive.shims

hive-shims-common

2.3.7

org.apache.hive.shims

hive-shims-scheduler

2.3.7

org.apache.htrace

htrace-core

3.1.0-incubating

org.apache.httpcomponents

httpclient

4.5.6

org.apache.httpcomponents

httpcore

4.4.12

org.apache.ivy

ivy

2.4.0

org.apache.mesos

mesos-shaded-protobuf

1.4.0

org.apache.orc

orc-core

1.5.10

org.apache.orc

orc-mapreduce

1.5.10

org.apache.orc

orc-shims

1.5.10

org.apache.parquet

parquet-column

1.10.1-databricks6

org.apache.parquet

parquet-common

1.10.1-databricks6

org.apache.parquet

parquet-encoding

1.10.1-databricks6

org.apache.parquet

parquet-format

2.4.0

org.apache.parquet

parquet-hadoop

1.10.1-databricks6

org.apache.parquet

parquet-jackson

1.10.1-databricks6

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.12.0

org.apache.velocity

velocity

1.5

org.apache.xbean

xbean-asm7-shaded

4.15

org.apache.yetus

audience-annotations

0.5.0

org.apache.zookeeper

zookeeper

3.4.14

org.codehaus.jackson

jackson-core-asl

1.9.13

org.codehaus.jackson

jackson-jaxrs

1.9.13

org.codehaus.jackson

jackson-mapper-asl

1.9.13

org.codehaus.jackson

jackson-xc

1.9.13

org.codehaus.janino

commons-compiler

3.0.16

org.codehaus.janino

janino

3.0.16

org.datanucleus

datanucleus-api-jdo

4.2.4

org.datanucleus

datanucleus-core

4.1.17

org.datanucleus

datanucleus-rdbms

4.1.19

org.datanucleus

javax.jdo

3.2.0-m3

org.eclipse.jetty

jetty-client

9.4.18.v20190429

org.eclipse.jetty

jetty-continuation

9.4.18.v20190429

org.eclipse.jetty

jetty-http

9.4.18.v20190429

org.eclipse.jetty

jetty-io

9.4.18.v20190429

org.eclipse.jetty

jetty-jndi

9.4.18.v20190429

org.eclipse.jetty

jetty-plus

9.4.18.v20190429

org.eclipse.jetty

jetty-proxy

9.4.18.v20190429

org.eclipse.jetty

jetty-security

9.4.18.v20190429

org.eclipse.jetty

jetty-server

9.4.18.v20190429

org.eclipse.jetty

jetty-servlet

9.4.18.v20190429

org.eclipse.jetty

jetty-servlets

9.4.18.v20190429

org.eclipse.jetty

jetty-util

9.4.18.v20190429

org.eclipse.jetty

jetty-webapp

9.4.18.v20190429

org.eclipse.jetty

jetty-xml

9.4.18.v20190429

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.6.1

org.glassfish.hk2

hk2-locator

2.6.1

org.glassfish.hk2

hk2-utils

2.6.1

org.glassfish.hk2

osgi-resource-locator

1.0.3

org.glassfish.hk2.external

aopalliance-repackaged

2.6.1

org.glassfish.hk2.external

jakarta.inject

2.6.1

org.glassfish.jersey.containers

jersey-container-servlet

2.30

org.glassfish.jersey.containers

jersey-container-servlet-core

2.30

org.glassfish.jersey.core

jersey-client

2.30

org.glassfish.jersey.core

jersey-common

2.30

org.glassfish.jersey.core

jersey-server

2.30

org.glassfish.jersey.inject

jersey-hk2

2.30

org.glassfish.jersey.media

jersey-media-jaxb

2.30

org.hibernate.validator

hibernate-validator

6.1.0.Final

org.javassist

javassist

3.25.0-GA

org.jboss.logging

jboss-logging

3.3.2.Final

org.jdbi

jdbi

2.63.1

org.joda

joda-convert

1.7

org.jodd

jodd-core

3.5.2

org.json4s

json4s-ast_2.12

3.6.6

org.json4s

json4s-core_2.12

3.6.6

org.json4s

json4s-jackson_2.12

3.6.6

org.json4s

json4s-scalap_2.12

3.6.6

org.lz4

lz4-java

1.7.1

org.mariadb.jdbc

mariadb-java-client

2.1.2

org.objenesis

objenesis

2.5.1

org.postgresql

postgresql

42.1.4

org.roaringbitmap

RoaringBitmap

0.7.45

org.roaringbitmap

shims

0.7.45

org.rocksdb

rocksdbjni

6.2.2

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.12

2.12.10

org.scala-lang

scala-library_2.12

2.12.10

org.scala-lang

scala-reflect_2.12

2.12.10

org.scala-lang.modules

scala-collection-compat_2.12

2.1.1

org.scala-lang.modules

scala-parser-combinators_2.12

1.1.2

org.scala-lang.modules

scala-xml_2.12

1.2.0

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.12

1.14.2

org.scalactic

scalactic_2.12

3.0.8

org.scalanlp

breeze-macros_2.12

1.0

org.scalanlp

breeze_2.12

1.0

org.scalatest

scalatest_2.12

3.0.8

org.slf4j

jcl-over-slf4j

1.7.30

org.slf4j

jul-to-slf4j

1.7.30

org.slf4j

slf4j-api

1.7.30

org.slf4j

slf4j-log4j12

1.7.30

org.spark-project.spark

unused

1.0.0

org.springframework

spring-core

4.1.4.RELEASE

org.springframework

spring-test

4.1.4.RELEASE

org.threeten

threeten-extra

1.5.0

org.tukaani

xz

1.5

org.typelevel

algebra_2.12

2.0.0-M2

org.typelevel

cats-kernel_2.12

2.0.0-M4

org.typelevel

machinist_2.12

0.6.8

org.typelevel

macro-compat_2.12

1.1.1

org.typelevel

spire-macros_2.12

0.17.0-M1

org.typelevel

spire-platform_2.12

0.17.0-M1

org.typelevel

spire-util_2.12

0.17.0-M1

org.typelevel

spire_2.12

0.17.0-M1

org.wildfly.openssl

wildfly-openssl

1.0.7.Final

org.xerial

sqlite-jdbc

3.8.11.2

org.xerial.snappy

snappy-java

1.1.7.5

org.yaml

snakeyaml

1.24

oro

oro

2.0.8

pl.edu.icm

JLargeArrays

1.5

software.amazon.ion

ion-java

1.0.2

stax

stax-api

1.0.1

xmlenc

xmlenc

0.52