Spark Configuration

Spark configuration properties

To fine tune Spark jobs, you can provide custom Spark configuration properties at the bottom of the cluster creation page.

../../_images/spark-config-aws.png

Environment variables

In addition to Spark configuration properties, you can provide environment variables that you can access from scripts running on the cluster. See spark_env_vars in Create cluster request structure.

Note

This feature is supported only in the API.