1.6.3-db2 cluster image (EoS)
Databricks released this update in March, 2017.
This release has been deprecated. For more information about the Databricks Runtime deprecation policy and schedule, see Databricks support lifecycles.
The following release notes provide information about the 1.6.3-db2 cluster image powered by Apache Spark.
Changes and Improvements
- Python 3 support (beta): users can now configure clusters to use Python 3 in notebooks and Spark tasks.
- Results in notebook cells will appear in a streaming fashion instead of all at once the command finishes running, which help users with more feedback on what the state of a command is.
- Fixed bugs impacting Spark's monitoring REST API
- Disallow dbutils.fs.mv to move a directory to the sub-directory of this directory.
- Fix regression when calling .toDF() and .toDS() on dbutils.fs responses.
pipupgraded from 8.1.2 to 9.0.1 (this refers to/databricks/bin/python/pipand thepiplibrary, not the systempipexecutable).setuptoolsupgraded from 28.6.0 to 32.3.1.docutilsupgraded from 0.12 to 0.13.1.
Apache Spark
The 1.6.3-db2 cluster image includes Apache Spark 1.6.3. In addition to 1.6.3-db1, the 1.6.3-db2 cluster image also includes the following extra bug fixes and improvements made to Spark:
- [SPARK-19529]TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()
- [SPARK-18372][SQL] Staging directory fail to be removed
- [SPARK-12446][SQL] Add unit tests for JDBCRDD internal functions
- [SPARK-18553][CORE] Fix leak of TaskSetManager following executor loss
System Environment
- Operating System: Ubuntu 16.04.1 LTS
- Java: 1.8.0_91
- Scala: 2.10.6
- Python: 2.7.12
- R: R version 3.2.3 (2015-12-10)
Pre-installed Python Libraries
Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
ansi2html | 1.1.1 | argparse | 1.2.1 | boto | 2.42.0 |
boto3 | 1.4.1 | botocore | 1.4.70 | brewer2mpl | 1.4.1 |
certifi | 2016.2.28 | cffi | 1.7.0 | chardet | 2.3.0 |
colorama | 0.3.7 | configobj | 5.0.6 | cryptography | 1.5 |
cycler | 0.10.0 | Cython | 0.24.1 | decorator | 4.0.10 |
docutils | 0.13.1 | enum34 | 1.1.6 | et-xmlfile | 1.0.1 |
freetype-py | 1.0.2 | funcsigs | 1.0.2 | fusepy | 2.0.4 |
futures | 3.0.5 | ggplot | 0.6.8 | html5lib | 0.999 |
idna | 2.1 | ipaddress | 1.0.16 | ipython | 2.2.0 |
ipython-genutils | 0.1.0 | jdcal | 1.2 | Jinja2 | 2.8 |
jmespath | 0.9.0 | llvmlite | 0.13.0 | lxml | 3.6.4 |
MarkupSafe | 0.23 | matplotlib | 1.5.3 | mpld3 | 0.2 |
msgpack-python | 0.4.7 | ndg-httpsclient | 0.3.3 | numba | 0.28.1 |
numpy | 1.11.1 | openpyxl | 2.3.2 | pandas | 0.18.1 |
pathlib2 | 2.1.0 | patsy | 0.4.1 | pexpect | 4.0.1 |
pickleshare | 0.7.4 | Pillow | 3.3.1 | pip | 9.0.1 |
pkg_resources | 0.0.0 | ply | 3.9 | prompt-toolkit | 1.0.7 |
psycopg2 | 2.6.2 | ptyprocess | 0.5.1 | py4j | 0.10.3 |
pyasn1 | 0.1.9 | pycparser | 2.14 | Pygments | 2.1.3 |
PyGObject | 3.20.0 | pyOpenSSL | 16.0.0 | pyparsing | 2.1.4 |
pypng | 0.0.18 | Python | 2.7.12 | python-dateutil | 2.5.3 |
python-geohash | 0.8.5 | pytz | 2016.6.1 | requests | 2.11.1 |
s3transfer | 0.1.9 | scikit-learn | 0.17.1 | scipy | 0.18.1 |
scour | 0.32 | seaborn | 0.7.1 | setuptools | 32.3.1 |
simplejson | 3.8.2 | simples3 | 1.0 | singledispatch | 3.4.0.3 |
six | 1.10.0 | statsmodels | 0.6.1 | traitlets | 4.3.0 |
urllib3 | 1.19.1 | virtualenv | 15.0.1 | wcwidth | 0.1.7 |
wheel | 0.30.0a0 | wsgiref | 0.1.2 |
Pre-installed R Libraries
Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
abind | 1.4-3 | assertthat | 0.1 | base | 3.2.3 |
BH | 1.60.0-2 | bitops | 1.0-6 | boot | 1.3-17 |
brew | 1.0-6 | car | 2.1-3 | caret | 6.0-71 |
chron | 2.3-47 | class | 7.3-14 | cluster | 2.0.5 |
codetools | 0.2-14 | colorspace | 1.2-4 | compiler | 3.2.3 |
crayon | 1.3.1 | curl | 2.2 | data.table | 1.9.6 |
datasets | 3.2.3 | DBI | 0.5-1 | devtools | 1.12.0 |
dichromat | 2.0-0 | digest | 0.6.9 | doMC | 1.3.4 |
dplyr | 0.5.0 | foreach | 1.4.3 | foreign | 0.8-66 |
gbm | 2.1.1 | ggplot2 | 2.1.0 | git2r | 0.15.0 |
glmnet | 2.0-5 | graphics | 3.2.3 | grDevices | 3.2.3 |
grid | 3.2.3 | gsubfn | 0.6-6 | gtable | 0.1.2 |
h2o | 3.10.0.8 | httr | 1.2.1 | hwriter | 1.3.2 |
hwriterPlus | 1.0-3 | iterators | 1.0.8 | jsonlite | 1.1 |
KernSmooth | 2.23-15 | labeling | 0.3 | lattice | 0.20-34 |
lazyeval | 0.2.0 | littler | 0.3.0 | lme4 | 1.1-12 |
lubridate | 1.6.0 | magrittr | 1.5 | mapproj | 1.2-4 |
maps | 3.0.2 | MASS | 7.3-45 | Matrix | 1.2-7.1 |
MatrixModels | 0.4-1 | memoise | 1.0.0 | methods | 3.2.3 |
mgcv | 1.8-11 | mime | 0.5 | minqa | 1.2.4 |
multicore | 0.2 | munsell | 0.4.2 | mvtnorm | 1.0-5 |
nlme | 3.1-124 | nloptr | 1.0.4 | nnet | 7.3-12 |
openssl | 0.9.4 | parallel | 3.2.3 | pbkrtest | 0.4-6 |
pkgKitten | 0.1.3 | plyr | 1.8.4 | praise | 1.0.0 |
pROC | 1.8 | proto | 0.3-10 | quantreg | 5.29 |
R.methodsS3 | 1.7.1 | R.oo | 1.20.0 | R.utils | 2.4.0 |
R6 | 2.2.0 | randomForest | 4.6-12 | RColorBrewer | 1.1-2 |
Rcpp | 0.12.7 | RcppEigen | 0.3.2.9.0 | RCurl | 1.95-4.8 |
reshape2 | 1.4.2 | RODBC | 1.3-12 | roxygen2 | 5.0.1 |
rpart | 4.1-10 | Rserve | 1.7-3 | RSQLite | 1.0.0 |
rstudioapi | 0.6 | scales | 0.3.0 | sp | 1.0-15 |
SparkR | 1.6.3 | SparseM | 1.72 | spatial | 7.3-11 |
splines | 3.2.3 | sqldf | 0.4-10 | statmod | 1.4.26 |
stats | 3.2.3 | stats4 | 3.2.3 | stringi | 1.0-1 |
stringr | 1.0.0 | survival | 2.38-3 | tcltk | 3.2.3 |
TeachingDemos | 2.10 | testthat | 1.0.2 | tibble | 1.2 |
tools | 3.2.3 | utils | 3.2.3 | whisker | 0.3-2 |
withr | 1.0.2 |
Pre-installed Java and Scala libraries (Hadoop 1 cluster version)
Group ID | Artifact ID | Version |
|---|---|---|
antlr | antlr | 2.7.7 |
asm | asm | 3.1 |
com.amazonaws | amazon-sqs-java-messaging-lib | 1.0.0 |
com.amazonaws | aws-java-sdk | 1.9.40 |
com.amazonaws | aws-java-sdk-autoscaling | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudformation | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudfront | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudhsm | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudsearch | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudtrail | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudwatch | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.9.40 |
com.amazonaws | aws-java-sdk-codedeploy | 1.9.40 |
com.amazonaws | aws-java-sdk-cognitoidentity | 1.9.40 |
com.amazonaws | aws-java-sdk-cognitosync | 1.9.40 |
com.amazonaws | aws-java-sdk-config | 1.9.40 |
com.amazonaws | aws-java-sdk-core | 1.9.40 |
com.amazonaws | aws-java-sdk-datapipeline | 1.9.40 |
com.amazonaws | aws-java-sdk-directconnect | 1.9.40 |
com.amazonaws | aws-java-sdk-directory | 1.9.40 |
com.amazonaws | aws-java-sdk-dynamodb | 1.9.40 |
com.amazonaws | aws-java-sdk-ec2 | 1.9.40 |
com.amazonaws | aws-java-sdk-ecs | 1.9.40 |
com.amazonaws | aws-java-sdk-efs | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticache | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.9.40 |
com.amazonaws | aws-java-sdk-elastictranscoder | 1.9.40 |
com.amazonaws | aws-java-sdk-emr | 1.9.40 |
com.amazonaws | aws-java-sdk-glacier | 1.9.40 |
com.amazonaws | aws-java-sdk-iam | 1.9.40 |
com.amazonaws | aws-java-sdk-importexport | 1.9.40 |
com.amazonaws | aws-java-sdk-kinesis | 1.9.40 |
com.amazonaws | aws-java-sdk-kms | 1.9.40 |
com.amazonaws | aws-java-sdk-lambda | 1.9.40 |
com.amazonaws | aws-java-sdk-logs | 1.9.40 |
com.amazonaws | aws-java-sdk-machinelearning | 1.9.40 |
com.amazonaws | aws-java-sdk-opsworks | 1.9.40 |
com.amazonaws | aws-java-sdk-rds | 1.9.40 |
com.amazonaws | aws-java-sdk-redshift | 1.9.40 |
com.amazonaws | aws-java-sdk-route53 | 1.9.40 |
com.amazonaws | aws-java-sdk-s3 | 1.9.40 |
com.amazonaws | aws-java-sdk-ses | 1.9.40 |
com.amazonaws | aws-java-sdk-simpledb | 1.9.40 |
com.amazonaws | aws-java-sdk-simpleworkflow | 1.9.40 |
com.amazonaws | aws-java-sdk-sns | 1.9.40 |
com.amazonaws | aws-java-sdk-sqs | 1.9.40 |
com.amazonaws | aws-java-sdk-ssm | 1.9.40 |
com.amazonaws | aws-java-sdk-storagegateway | 1.9.40 |
com.amazonaws | aws-java-sdk-sts | 1.9.40 |
com.amazonaws | aws-java-sdk-support | 1.9.40 |
com.amazonaws | aws-java-sdk-swf-libraries | 1.9.40 |
com.amazonaws | aws-java-sdk-workspaces | 1.9.40 |
com.clearspring.analytics | stream | 2.7.0 |
com.databricks | Rserve | 1.8-3 |
com.databricks | jets3t | 0.7.1-0 |
com.databricks | spark-avro_2.10 | 2.0.1 |
com.databricks | spark-csv_2.10 | 1.3.2 |
com.databricks | spark-redshift_2.10 | 0.6.0 |
com.databricks.scalapb | compilerplugin_2.10 | 0.4.15-9 |
com.databricks.scalapb | scalapb-runtime_2.10 | 0.4.15-9 |
com.esotericsoftware.kryo | kryo | 2.21 |
com.esotericsoftware.minlog | minlog | 1.2 |
com.esotericsoftware.reflectasm | reflectasm-shaded | 1.07 |
com.fasterxml | classmate | 1.0.0 |
com.fasterxml.jackson.core | jackson-annotations | 2.4.5 |
com.fasterxml.jackson.core | jackson-core | 2.4.5 |
com.fasterxml.jackson.core | jackson-databind | 2.4.5 |
com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.4.5 |
com.fasterxml.jackson.module | jackson-module-scala_2.10 | 2.4.5 |
com.github.fommil | jniloader | 1.1 |
com.github.fommil.netlib | core | 1.1.2 |
com.github.fommil.netlib | native_ref-java | 1.1 |
com.github.fommil.netlib | native_ref-java-natives | 1.1 |
com.github.fommil.netlib | native_system-java | 1.1 |
com.github.fommil.netlib | native_system-java-natives | 1.1 |
com.github.fommil.netlib | netlib-native_ref-linux-x86_64-natives | 1.1 |
com.github.fommil.netlib | netlib-native_system-linux-x86_64-natives | 1.1 |
com.github.rwl | jtransforms | 2.4.0 |
com.github.scopt | scopt_2.10 | 3.3.0 |
com.google.api-client | google-api-client | 1.22.0 |
com.google.apis | google-api-services-admin-directory | directory_v1-rev69-1.22.0 |
com.google.code.findbugs | jsr305 | 2.0.1 |
com.google.guava | guava | 15.0 |
com.google.http-client | google-http-client | 1.22.0 |
com.google.http-client | google-http-client-jackson2 | 1.22.0 |
com.google.oauth-client | google-oauth-client | 1.22.0 |
com.google.protobuf | protobuf-java | 2.6.1 |
com.googlecode.javaewah | JavaEWAH | 0.3.2 |
com.h2database | h2 | 1.3.174 |
com.jcraft | jsch | 0.1.50 |
com.jolbox | bonecp | 0.8.0.RELEASE |
com.mchange | c3p0 | 0.9.5.1 |
com.mchange | mchange-commons-java | 0.2.10 |
com.ning | compress-lzf | 1.0.3 |
com.sun.istack | istack-commons-runtime | 2.16 |
com.sun.jersey | jersey-core | 1.9 |
com.sun.jersey | jersey-json | 1.9 |
com.sun.jersey | jersey-server | 1.9 |
com.sun.mail | gimap | 1.5.2 |
com.sun.mail | javax.mail | 1.5.2 |
com.sun.xml.bind | jaxb-core | 2.2.7 |
com.sun.xml.bind | jaxb-impl | 2.2.7 |
com.sun.xml.fastinfoset | FastInfoset | 1.2.12 |
com.thoughtworks.paranamer | paranamer | 2.6 |
com.trueaccord.lenses | lenses_2.10 | 0.3 |
com.twitter | chill-java | 0.5.0 |
com.twitter | chill_2.10 | 0.5.0 |
com.twitter | parquet-hadoop-bundle | 1.6.0 |
com.twitter | util-app_2.10 | 6.23.0 |
com.twitter | util-core_2.10 | 6.23.0 |
com.twitter | util-jvm_2.10 | 6.23.0 |
com.typesafe | config | 1.2.1 |
com.typesafe | scalalogging-slf4j_2.10 | 1.1.0 |
com.typesafe.akka | akka-actor_2.10 | 2.3.11 |
com.typesafe.akka | akka-remote_2.10 | 2.3.11 |
com.typesafe.akka | akka-slf4j_2.10 | 2.3.11 |
com.univocity | univocity-parsers | 1.5.1 |
com.zaxxer | HikariCP | 2.4.1 |
commons-beanutils | commons-beanutils | 1.9.2 |
commons-beanutils | commons-beanutils-core | 1.8.0 |
commons-cli | commons-cli | 1.2 |
commons-codec | commons-codec | 1.8 |
commons-collections | commons-collections | 3.2.2 |
commons-configuration | commons-configuration | 1.6 |
commons-dbcp | commons-dbcp | 1.4 |
commons-digester | commons-digester | 1.8.1 |
commons-el | commons-el | 1.0 |
commons-fileupload | commons-fileupload | 1.3 |
commons-httpclient | commons-httpclient | 3.1 |
commons-io | commons-io | 2.5 |
commons-lang | commons-lang | 2.6 |
commons-logging | commons-logging | 1.1.1 |
commons-net | commons-net | 2.2 |
commons-pool | commons-pool | 1.5.4 |
commons-validator | commons-validator | 1.5.0 |
dom4j | dom4j | 1.6.1 |
hsqldb | hsqldb | 1.8.0.10 |
info.ganglia.gmetric4j | gmetric4j | 1.0.7 |
io.dropwizard.metrics | metrics-core | 3.1.2 |
io.dropwizard.metrics | metrics-ganglia | 3.1.2 |
io.dropwizard.metrics | metrics-graphite | 3.1.2 |
io.dropwizard.metrics | metrics-healthchecks | 3.1.2 |
io.dropwizard.metrics | metrics-jetty8 | 3.1.2 |
io.dropwizard.metrics | metrics-json | 3.1.2 |
io.dropwizard.metrics | metrics-jvm | 3.1.2 |
io.dropwizard.metrics | metrics-log4j | 3.1.2 |
io.dropwizard.metrics | metrics-servlets | 3.1.2 |
io.jsonwebtoken | jjwt | 0.6.0 |
io.netty | netty | 3.8.0.Final |
io.netty | netty-all | 4.0.29.Final |
io.prometheus | simpleclient | 0.0.16 |
io.prometheus | simpleclient_common | 0.0.16 |
io.prometheus | simpleclient_dropwizard | 0.0.16 |
io.prometheus | simpleclient_pushgateway | 0.0.16 |
io.prometheus | simpleclient_servlet | 0.0.16 |
io.prometheus.client | model | 0.0.2 |
io.prometheus.jmx | collector | 0.7 |
javax.activation | activation | 1.1 |
javax.el | javax.el-api | 2.2.4 |
javax.jdo | jdo-api | 3.0.1 |
javax.servlet | javax.servlet-api | 3.0.1 |
javax.transaction | jta | 1.1 |
javax.validation | validation-api | 1.1.0.Final |
javax.xml.bind | jaxb-api | 2.2.7 |
javolution | javolution | 5.5.1 |
jline | jline | 2.11 |
jline | jline | 2.12 |
joda-time | joda-time | 2.8.1 |
log4j | apache-log4j-extras | 1.2.17 |
log4j | log4j | 1.2.17 |
mysql | mysql-connector-java | 5.1.27 |
net.hydromatic | eigenbase-properties | 1.1.5 |
net.java.dev.jets3t | jets3t | 0.7.1 |
net.jpountz.lz4 | lz4 | 1.3.0 |
net.liftweb | lift-json_2.10 | 2.6.1 |
net.razorvine | pyrolite | 4.9 |
net.sf.jpam | jpam | 1.1 |
net.sf.opencsv | opencsv | 2.3 |
net.sf.py4j | py4j | 0.9 |
net.sf.supercsv | super-csv | 2.2.0 |
net.sourceforge.f2j | arpack_combined_all | 0.1 |
org.acplt | oncrpc | 1.0.7 |
org.antlr | ST4 | 4.0.4 |
org.antlr | antlr-runtime | 3.4 |
org.antlr | antlr4-runtime | 4.5.3 |
org.antlr | stringtemplate | 3.2.1 |
org.apache.ant | ant | 1.9.2 |
org.apache.ant | ant-jsch | 1.9.2 |
org.apache.ant | ant-launcher | 1.9.2 |
org.apache.avro | avro | 1.7.7 |
org.apache.avro | avro-ipc | 1.7.7 |
org.apache.avro | avro-ipc-tests | 1.7.7 |
org.apache.avro | avro-mapred-hadoop1 | 1.7.7 |
org.apache.calcite | calcite-avatica | 1.2.0-incubating |
org.apache.calcite | calcite-core | 1.2.0-incubating |
org.apache.calcite | calcite-linq4j | 1.2.0-incubating |
org.apache.commons | commons-compress | 1.9 |
org.apache.commons | commons-csv | 1.1 |
org.apache.commons | commons-lang3 | 3.4 |
org.apache.commons | commons-math | 2.1 |
org.apache.commons | commons-math3 | 3.4.1 |
org.apache.curator | curator-client | 2.4.0 |
org.apache.curator | curator-framework | 2.4.0 |
org.apache.curator | curator-recipes | 2.4.0 |
org.apache.derby | derby | 10.10.2.0 |
org.apache.geronimo.specs | geronimo-jms_1.1_spec | 1.1.1 |
org.apache.hadoop | hadoop-client | 1.2.1-0 |
org.apache.hadoop | hadoop-core | 1.2.1-0 |
org.apache.httpcomponents | httpclient | 4.4.1 |
org.apache.httpcomponents | httpcore | 4.4.1 |
org.apache.ivy | ivy | 2.4.0 |
org.apache.mesos | mesos-shaded-protobuf | 0.21.1 |
org.apache.parquet | parquet-column | 1.7.0 |
org.apache.parquet | parquet-common | 1.7.0 |
org.apache.parquet | parquet-encoding | 1.7.0 |
org.apache.parquet | parquet-format | 2.3.0-incubating |
org.apache.parquet | parquet-generator | 1.7.0 |
org.apache.parquet | parquet-hadoop | 1.7.0 |
org.apache.parquet | parquet-jackson | 1.7.0 |
org.apache.poi | poi | 3.9 |
org.apache.poi | poi-ooxml | 3.9 |
org.apache.poi | poi-ooxml-schemas | 3.9 |
org.apache.thrift | libfb303 | 0.9.2 |
org.apache.thrift | libthrift | 0.9.2 |
org.apache.xbean | xbean-asm5-shaded | 4.4 |
org.apache.xmlbeans | xmlbeans | 2.3.0 |
org.apache.zookeeper | zookeeper | 3.4.5 |
org.codehaus.jackson | jackson-core-asl | 1.8.8 |
org.codehaus.jackson | jackson-jaxrs | 1.8.3 |
org.codehaus.jackson | jackson-mapper-asl | 1.8.8 |
org.codehaus.jackson | jackson-xc | 1.8.3 |
org.codehaus.janino | commons-compiler | 2.7.8 |
org.codehaus.janino | janino | 2.7.8 |
org.codehaus.jettison | jettison | 1.1 |
org.datanucleus | datanucleus-api-jdo | 3.2.6 |
org.datanucleus | datanucleus-core | 3.2.10 |
org.datanucleus | datanucleus-rdbms | 3.2.9 |
org.eclipse.jetty | jetty-client | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-continuation | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-http | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-io | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-security | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-server | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-servlet | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-servlets | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-util | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-websocket | 8.1.14.v20131031 |
org.eclipse.jetty.orbit | javax.servlet | 3.0.0.v201112011016 |
org.fusesource.jansi | jansi | 1.4 |
org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
org.hawkular.agent | prometheus-scraper | 0.20.1.Final |
org.hibernate | hibernate-validator | 5.1.1.Final |
org.iq80.snappy | snappy | 0.2 |
org.jboss.logging | jboss-logging | 3.1.3.GA |
org.jdbi | jdbi | 2.63.1 |
org.joda | joda-convert | 1.7 |
org.jodd | jodd-core | 3.5.2 |
org.jpmml | pmml-agent | 1.1.15 |
org.jpmml | pmml-model | 1.1.15 |
org.jpmml | pmml-schema | 1.1.15 |
org.json | json | 20090211 |
org.json4s | json4s-ast_2.10 | 3.2.10 |
org.json4s | json4s-core_2.10 | 3.2.10 |
org.json4s | json4s-jackson_2.10 | 3.2.10 |
org.mockito | mockito-all | 1.9.5 |
org.objenesis | objenesis | 1.2 |
org.postgresql | postgresql | 9.4-1204-jdbc41 |
org.roaringbitmap | RoaringBitmap | 0.5.11 |
org.rosuda.REngine | REngine | 2.1.0 |
org.scala-lang | jline | 2.10.6 |
org.scala-lang | scala-compiler_2.10 | 2.10.6 |
org.scala-lang | scala-library_2.10 | 2.10.6 |
org.scala-lang | scala-reflect_2.10 | 2.10.6 |
org.scala-lang | scalap_2.10 | 2.10.6 |
org.scala-sbt | test-interface | 1.0 |
org.scalacheck | scalacheck_2.10 | 1.12.5 |
org.scalamacros | quasiquotes_2.10 | 2.0.0 |
org.scalamock | scalamock-core_2.10 | 3.2.1 |
org.scalamock | scalamock-scalatest-support_2.10 | 3.2.1 |
org.scalanlp | breeze-macros_2.10 | 0.11.2 |
org.scalanlp | breeze_2.10 | 0.11.2 |
org.scalatest | scalatest_2.10 | 2.2.4 |
org.slf4j | jcl-over-slf4j | 1.7.10 |
org.slf4j | jul-to-slf4j | 1.7.10 |
org.slf4j | slf4j-api | 1.7.5 |
org.slf4j | slf4j-log4j12 | 1.7.5 |
org.spark-project.hive | hive-beeline | 1.2.1.spark |
org.spark-project.hive | hive-cli | 1.2.1.spark |
org.spark-project.hive | hive-exec | 1.2.1.spark |
org.spark-project.hive | hive-jdbc | 1.2.1.spark |
org.spark-project.hive | hive-metastore | 1.2.1.spark |
org.spark-project.hive | hive-service | 1.2.1.spark |
org.spark-project.spark | unused | 1.0.0 |
org.spire-math | spire-macros_2.10 | 0.7.4 |
org.spire-math | spire_2.10 | 0.7.4 |
org.springframework | spring-core | 4.1.4.RELEASE |
org.springframework | spring-test | 4.1.4.RELEASE |
org.tachyonproject | tachyon-client | 0.8.2 |
org.tachyonproject | tachyon-underfs-hdfs | 0.8.2 |
org.tachyonproject | tachyon-underfs-local | 0.8.2 |
org.tachyonproject | tachyon-underfs-s3 | 0.8.2 |
org.uncommons.maths | uncommons-maths | 1.2.2a |
org.xerial | sqlite-jdbc | 3.8.11.2 |
org.xerial.snappy | snappy-java | 1.1.2.6 |
org.yaml | snakeyaml | 1.16 |
oro | oro | 2.0.8 |
stax | stax-api | 1.0.1 |
xmlenc | xmlenc | 0.52 |
Pre-installed Java and Scala libraries (Hadoop 2 cluster version)
Group ID | Artifact ID | Version |
|---|---|---|
antlr | antlr | 2.7.7 |
asm | asm | 3.1 |
com.amazonaws | amazon-sqs-java-messaging-lib | 1.0.0 |
com.amazonaws | aws-java-sdk | 1.9.40 |
com.amazonaws | aws-java-sdk-autoscaling | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudformation | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudfront | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudhsm | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudsearch | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudtrail | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudwatch | 1.9.40 |
com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.9.40 |
com.amazonaws | aws-java-sdk-codedeploy | 1.9.40 |
com.amazonaws | aws-java-sdk-cognitoidentity | 1.9.40 |
com.amazonaws | aws-java-sdk-cognitosync | 1.9.40 |
com.amazonaws | aws-java-sdk-config | 1.9.40 |
com.amazonaws | aws-java-sdk-core | 1.9.40 |
com.amazonaws | aws-java-sdk-datapipeline | 1.9.40 |
com.amazonaws | aws-java-sdk-directconnect | 1.9.40 |
com.amazonaws | aws-java-sdk-directory | 1.9.40 |
com.amazonaws | aws-java-sdk-dynamodb | 1.9.40 |
com.amazonaws | aws-java-sdk-ec2 | 1.9.40 |
com.amazonaws | aws-java-sdk-ecs | 1.9.40 |
com.amazonaws | aws-java-sdk-efs | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticache | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.9.40 |
com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.9.40 |
com.amazonaws | aws-java-sdk-elastictranscoder | 1.9.40 |
com.amazonaws | aws-java-sdk-emr | 1.9.40 |
com.amazonaws | aws-java-sdk-glacier | 1.9.40 |
com.amazonaws | aws-java-sdk-iam | 1.9.40 |
com.amazonaws | aws-java-sdk-importexport | 1.9.40 |
com.amazonaws | aws-java-sdk-kinesis | 1.9.40 |
com.amazonaws | aws-java-sdk-kms | 1.9.40 |
com.amazonaws | aws-java-sdk-lambda | 1.9.40 |
com.amazonaws | aws-java-sdk-logs | 1.9.40 |
com.amazonaws | aws-java-sdk-machinelearning | 1.9.40 |
com.amazonaws | aws-java-sdk-opsworks | 1.9.40 |
com.amazonaws | aws-java-sdk-rds | 1.9.40 |
com.amazonaws | aws-java-sdk-redshift | 1.9.40 |
com.amazonaws | aws-java-sdk-route53 | 1.9.40 |
com.amazonaws | aws-java-sdk-s3 | 1.9.40 |
com.amazonaws | aws-java-sdk-ses | 1.9.40 |
com.amazonaws | aws-java-sdk-simpledb | 1.9.40 |
com.amazonaws | aws-java-sdk-simpleworkflow | 1.9.40 |
com.amazonaws | aws-java-sdk-sns | 1.9.40 |
com.amazonaws | aws-java-sdk-sqs | 1.9.40 |
com.amazonaws | aws-java-sdk-ssm | 1.9.40 |
com.amazonaws | aws-java-sdk-storagegateway | 1.9.40 |
com.amazonaws | aws-java-sdk-sts | 1.9.40 |
com.amazonaws | aws-java-sdk-support | 1.9.40 |
com.amazonaws | aws-java-sdk-swf-libraries | 1.9.40 |
com.amazonaws | aws-java-sdk-workspaces | 1.9.40 |
com.clearspring.analytics | stream | 2.7.0 |
com.databricks | Rserve | 1.8-3 |
com.databricks | jets3t | 0.7.1-0 |
com.databricks | spark-avro_2.10 | 2.0.1 |
com.databricks | spark-csv_2.10 | 1.3.2 |
com.databricks | spark-redshift_2.10 | 0.6.0 |
com.databricks.scalapb | compilerplugin_2.10 | 0.4.15-9 |
com.databricks.scalapb | scalapb-runtime_2.10 | 0.4.15-9 |
com.esotericsoftware.kryo | kryo | 2.21 |
com.esotericsoftware.minlog | minlog | 1.2 |
com.esotericsoftware.reflectasm | reflectasm-shaded | 1.07 |
com.fasterxml | classmate | 1.0.0 |
com.fasterxml.jackson.core | jackson-annotations | 2.4.5 |
com.fasterxml.jackson.core | jackson-core | 2.4.5 |
com.fasterxml.jackson.core | jackson-databind | 2.4.5 |
com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.4.5 |
com.fasterxml.jackson.module | jackson-module-scala_2.10 | 2.4.5 |
com.github.fommil | jniloader | 1.1 |
com.github.fommil.netlib | core | 1.1.2 |
com.github.fommil.netlib | native_ref-java | 1.1 |
com.github.fommil.netlib | native_ref-java-natives | 1.1 |
com.github.fommil.netlib | native_system-java | 1.1 |
com.github.fommil.netlib | native_system-java-natives | 1.1 |
com.github.fommil.netlib | netlib-native_ref-linux-x86_64-natives | 1.1 |
com.github.fommil.netlib | netlib-native_system-linux-x86_64-natives | 1.1 |
com.github.rwl | jtransforms | 2.4.0 |
com.github.scopt | scopt_2.10 | 3.3.0 |
com.google.api-client | google-api-client | 1.22.0 |
com.google.apis | google-api-services-admin-directory | directory_v1-rev69-1.22.0 |
com.google.code.findbugs | jsr305 | 2.0.1 |
com.google.code.gson | gson | 2.2.4 |
com.google.guava | guava | 15.0 |
com.google.http-client | google-http-client | 1.22.0 |
com.google.http-client | google-http-client-jackson2 | 1.22.0 |
com.google.oauth-client | google-oauth-client | 1.22.0 |
com.google.protobuf | protobuf-java | 2.6.1 |
com.googlecode.javaewah | JavaEWAH | 0.3.2 |
com.h2database | h2 | 1.3.174 |
com.jcraft | jsch | 0.1.50 |
com.jolbox | bonecp | 0.8.0.RELEASE |
com.mchange | c3p0 | 0.9.5.1 |
com.mchange | mchange-commons-java | 0.2.10 |
com.ning | compress-lzf | 1.0.3 |
com.sun.istack | istack-commons-runtime | 2.16 |
com.sun.jersey | jersey-client | 1.9 |
com.sun.jersey | jersey-core | 1.9 |
com.sun.jersey | jersey-json | 1.9 |
com.sun.jersey | jersey-server | 1.9 |
com.sun.mail | gimap | 1.5.2 |
com.sun.mail | javax.mail | 1.5.2 |
com.sun.xml.bind | jaxb-core | 2.2.7 |
com.sun.xml.bind | jaxb-impl | 2.2.7 |
com.sun.xml.fastinfoset | FastInfoset | 1.2.12 |
com.thoughtworks.paranamer | paranamer | 2.6 |
com.trueaccord.lenses | lenses_2.10 | 0.3 |
com.twitter | chill-java | 0.5.0 |
com.twitter | chill_2.10 | 0.5.0 |
com.twitter | parquet-hadoop-bundle | 1.6.0 |
com.twitter | util-app_2.10 | 6.23.0 |
com.twitter | util-core_2.10 | 6.23.0 |
com.twitter | util-jvm_2.10 | 6.23.0 |
com.typesafe | config | 1.2.1 |
com.typesafe | scalalogging-slf4j_2.10 | 1.1.0 |
com.typesafe.akka | akka-actor_2.10 | 2.3.11 |
com.typesafe.akka | akka-remote_2.10 | 2.3.11 |
com.typesafe.akka | akka-slf4j_2.10 | 2.3.11 |
com.univocity | univocity-parsers | 1.5.1 |
com.zaxxer | HikariCP | 2.4.1 |
commons-beanutils | commons-beanutils | 1.9.2 |
commons-beanutils | commons-beanutils-core | 1.8.0 |
commons-cli | commons-cli | 1.2 |
commons-codec | commons-codec | 1.8 |
commons-collections | commons-collections | 3.2.2 |
commons-configuration | commons-configuration | 1.6 |
commons-dbcp | commons-dbcp | 1.4 |
commons-digester | commons-digester | 1.8.1 |
commons-fileupload | commons-fileupload | 1.3 |
commons-httpclient | commons-httpclient | 3.1 |
commons-io | commons-io | 2.5 |
commons-lang | commons-lang | 2.6 |
commons-logging | commons-logging | 1.1.1 |
commons-net | commons-net | 2.2 |
commons-pool | commons-pool | 1.5.4 |
commons-validator | commons-validator | 1.5.0 |
dom4j | dom4j | 1.6.1 |
info.ganglia.gmetric4j | gmetric4j | 1.0.7 |
io.dropwizard.metrics | metrics-core | 3.1.2 |
io.dropwizard.metrics | metrics-ganglia | 3.1.2 |
io.dropwizard.metrics | metrics-graphite | 3.1.2 |
io.dropwizard.metrics | metrics-healthchecks | 3.1.2 |
io.dropwizard.metrics | metrics-jetty8 | 3.1.2 |
io.dropwizard.metrics | metrics-json | 3.1.2 |
io.dropwizard.metrics | metrics-jvm | 3.1.2 |
io.dropwizard.metrics | metrics-log4j | 3.1.2 |
io.dropwizard.metrics | metrics-servlets | 3.1.2 |
io.jsonwebtoken | jjwt | 0.6.0 |
io.netty | netty | 3.8.0.Final |
io.netty | netty-all | 4.0.29.Final |
io.prometheus | simpleclient | 0.0.16 |
io.prometheus | simpleclient_common | 0.0.16 |
io.prometheus | simpleclient_dropwizard | 0.0.16 |
io.prometheus | simpleclient_pushgateway | 0.0.16 |
io.prometheus | simpleclient_servlet | 0.0.16 |
io.prometheus.client | model | 0.0.2 |
io.prometheus.jmx | collector | 0.7 |
javax.activation | activation | 1.1 |
javax.el | javax.el-api | 2.2.4 |
javax.jdo | jdo-api | 3.0.1 |
javax.servlet | javax.servlet-api | 3.0.1 |
javax.servlet.jsp | jsp-api | 2.1 |
javax.transaction | jta | 1.1 |
javax.validation | validation-api | 1.1.0.Final |
javax.xml.bind | jaxb-api | 2.2.7 |
javolution | javolution | 5.5.1 |
jline | jline | 2.11 |
jline | jline | 2.12 |
joda-time | joda-time | 2.8.1 |
log4j | apache-log4j-extras | 1.2.17 |
log4j | log4j | 1.2.17 |
log4j | log4j | 1.2.17 |
mysql | mysql-connector-java | 5.1.27 |
net.hydromatic | eigenbase-properties | 1.1.5 |
net.java.dev.jets3t | jets3t | 0.7.1 |
net.jpountz.lz4 | lz4 | 1.3.0 |
net.liftweb | lift-json_2.10 | 2.6.1 |
net.razorvine | pyrolite | 4.9 |
net.sf.jpam | jpam | 1.1 |
net.sf.opencsv | opencsv | 2.3 |
net.sf.py4j | py4j | 0.9 |
net.sf.supercsv | super-csv | 2.2.0 |
net.sourceforge.f2j | arpack_combined_all | 0.1 |
org.acplt | oncrpc | 1.0.7 |
org.antlr | ST4 | 4.0.4 |
org.antlr | antlr-runtime | 3.4 |
org.antlr | antlr4-runtime | 4.5.3 |
org.antlr | stringtemplate | 3.2.1 |
org.apache.ant | ant | 1.9.2 |
org.apache.ant | ant-jsch | 1.9.2 |
org.apache.ant | ant-launcher | 1.9.2 |
org.apache.avro | avro | 1.7.7 |
org.apache.avro | avro-ipc | 1.7.7 |
org.apache.avro | avro-ipc-tests | 1.7.7 |
org.apache.avro | avro-mapred-hadoop2 | 1.7.7 |
org.apache.calcite | calcite-avatica | 1.2.0-incubating |
org.apache.calcite | calcite-core | 1.2.0-incubating |
org.apache.calcite | calcite-linq4j | 1.2.0-incubating |
org.apache.commons | commons-compress | 1.9 |
org.apache.commons | commons-csv | 1.1 |
org.apache.commons | commons-lang3 | 3.4 |
org.apache.commons | commons-math3 | 3.4.1 |
org.apache.curator | curator-client | 2.7.1 |
org.apache.curator | curator-framework | 2.7.1 |
org.apache.curator | curator-recipes | 2.7.1 |
org.apache.derby | derby | 10.10.2.0 |
org.apache.directory.api | api-asn1-api | 1.0.0-M20 |
org.apache.directory.api | api-util | 1.0.0-M20 |
org.apache.directory.server | apacheds-i18n | 2.0.0-M15 |
org.apache.directory.server | apacheds-kerberos-codec | 2.0.0-M15 |
org.apache.geronimo.specs | geronimo-jms_1.1_spec | 1.1.1 |
org.apache.hadoop | hadoop-annotations | 2.7.3 |
org.apache.hadoop | hadoop-auth | 2.7.3 |
org.apache.hadoop | hadoop-client | 2.7.3 |
org.apache.hadoop | hadoop-common | 2.7.3 |
org.apache.hadoop | hadoop-hdfs | 2.7.3 |
org.apache.hadoop | hadoop-mapreduce-client-app | 2.7.3 |
org.apache.hadoop | hadoop-mapreduce-client-common | 2.7.3 |
org.apache.hadoop | hadoop-mapreduce-client-core | 2.7.3 |
org.apache.hadoop | hadoop-mapreduce-client-jobclient | 2.7.3 |
org.apache.hadoop | hadoop-mapreduce-client-shuffle | 2.7.3 |
org.apache.hadoop | hadoop-yarn-api | 2.7.3 |
org.apache.hadoop | hadoop-yarn-client | 2.7.3 |
org.apache.hadoop | hadoop-yarn-common | 2.7.3 |
org.apache.hadoop | hadoop-yarn-server-common | 2.7.3 |
org.apache.htrace | htrace-core | 3.1.0-incubating |
org.apache.httpcomponents | httpclient | 4.4.1 |
org.apache.httpcomponents | httpcore | 4.4.1 |
org.apache.ivy | ivy | 2.4.0 |
org.apache.mesos | mesos-shaded-protobuf | 0.21.1 |
org.apache.parquet | parquet-column | 1.7.0 |
org.apache.parquet | parquet-common | 1.7.0 |
org.apache.parquet | parquet-encoding | 1.7.0 |
org.apache.parquet | parquet-format | 2.3.0-incubating |
org.apache.parquet | parquet-generator | 1.7.0 |
org.apache.parquet | parquet-hadoop | 1.7.0 |
org.apache.parquet | parquet-jackson | 1.7.0 |
org.apache.poi | poi | 3.9 |
org.apache.poi | poi-ooxml | 3.9 |
org.apache.poi | poi-ooxml-schemas | 3.9 |
org.apache.thrift | libfb303 | 0.9.2 |
org.apache.thrift | libthrift | 0.9.2 |
org.apache.xbean | xbean-asm5-shaded | 4.4 |
org.apache.xmlbeans | xmlbeans | 2.3.0 |
org.apache.zookeeper | zookeeper | 3.4.6 |
org.codehaus.jackson | jackson-core-asl | 1.8.8 |
org.codehaus.jackson | jackson-jaxrs | 1.9.13 |
org.codehaus.jackson | jackson-mapper-asl | 1.8.8 |
org.codehaus.jackson | jackson-xc | 1.9.13 |
org.codehaus.janino | commons-compiler | 2.7.8 |
org.codehaus.janino | janino | 2.7.8 |
org.codehaus.jettison | jettison | 1.1 |
org.datanucleus | datanucleus-api-jdo | 3.2.6 |
org.datanucleus | datanucleus-core | 3.2.10 |
org.datanucleus | datanucleus-rdbms | 3.2.9 |
org.eclipse.jetty | jetty-client | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-continuation | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-http | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-io | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-security | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-server | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-servlet | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-servlets | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-util | 8.1.14.v20131031 |
org.eclipse.jetty | jetty-websocket | 8.1.14.v20131031 |
org.eclipse.jetty.orbit | javax.servlet | 3.0.0.v201112011016 |
org.fusesource.jansi | jansi | 1.4 |
org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
org.hawkular.agent | prometheus-scraper | 0.20.1.Final |
org.hibernate | hibernate-validator | 5.1.1.Final |
org.iq80.snappy | snappy | 0.2 |
org.jboss.logging | jboss-logging | 3.1.3.GA |
org.jdbi | jdbi | 2.63.1 |
org.joda | joda-convert | 1.7 |
org.jodd | jodd-core | 3.5.2 |
org.jpmml | pmml-agent | 1.1.15 |
org.jpmml | pmml-model | 1.1.15 |
org.jpmml | pmml-schema | 1.1.15 |
org.json | json | 20090211 |
org.json4s | json4s-ast_2.10 | 3.2.10 |
org.json4s | json4s-core_2.10 | 3.2.10 |
org.json4s | json4s-jackson_2.10 | 3.2.10 |
org.mockito | mockito-all | 1.9.5 |
org.objenesis | objenesis | 1.2 |
org.postgresql | postgresql | 9.4-1204-jdbc41 |
org.roaringbitmap | RoaringBitmap | 0.5.11 |
org.rosuda.REngine | REngine | 2.1.0 |
org.scala-lang | jline | 2.10.6 |
org.scala-lang | scala-compiler_2.10 | 2.10.6 |
org.scala-lang | scala-library_2.10 | 2.10.6 |
org.scala-lang | scala-reflect_2.10 | 2.10.6 |
org.scala-lang | scalap_2.10 | 2.10.6 |
org.scala-sbt | test-interface | 1.0 |
org.scalacheck | scalacheck_2.10 | 1.12.5 |
org.scalamacros | quasiquotes_2.10 | 2.0.0 |
org.scalamock | scalamock-core_2.10 | 3.2.1 |
org.scalamock | scalamock-scalatest-support_2.10 | 3.2.1 |
org.scalanlp | breeze-macros_2.10 | 0.11.2 |
org.scalanlp | breeze_2.10 | 0.11.2 |
org.scalatest | scalatest_2.10 | 2.2.4 |
org.slf4j | jcl-over-slf4j | 1.7.10 |
org.slf4j | jul-to-slf4j | 1.7.10 |
org.slf4j | slf4j-api | 1.7.5 |
org.slf4j | slf4j-log4j12 | 1.7.5 |
org.slf4j | slf4j-log4j12 | 1.7.5 |
org.spark-project.hive | hive-beeline | 1.2.1.spark |
org.spark-project.hive | hive-cli | 1.2.1.spark |
org.spark-project.hive | hive-exec | 1.2.1.spark |
org.spark-project.hive | hive-jdbc | 1.2.1.spark |
org.spark-project.hive | hive-metastore | 1.2.1.spark |
org.spark-project.hive | hive-service | 1.2.1.spark |
org.spark-project.spark | unused | 1.0.0 |
org.spire-math | spire-macros_2.10 | 0.7.4 |
org.spire-math | spire_2.10 | 0.7.4 |
org.springframework | spring-core | 4.1.4.RELEASE |
org.springframework | spring-test | 4.1.4.RELEASE |
org.tachyonproject | tachyon-client | 0.8.2 |
org.tachyonproject | tachyon-underfs-hdfs | 0.8.2 |
org.tachyonproject | tachyon-underfs-local | 0.8.2 |
org.tachyonproject | tachyon-underfs-s3 | 0.8.2 |
org.uncommons.maths | uncommons-maths | 1.2.2a |
org.xerial | sqlite-jdbc | 3.8.11.2 |
org.xerial.snappy | snappy-java | 1.1.2.6 |
org.yaml | snakeyaml | 1.16 |
oro | oro | 2.0.8 |
stax | stax-api | 1.0.1 |
xerces | xercesImpl | 2.9.1 |
xml-apis | xml-apis | 1.3.04 |
xmlenc | xmlenc | 0.52 |