# The Run object ## Attributes - `job_id` (int64) The canonical identifier of the job that contains this run. Example: `11223344` - `run_id` (int64) The canonical identifier of the run. This ID is unique across all runs of all jobs. Example: `455644833` - `creator_user_name` (string) The creator user name. This field won’t be included in the response if the user has already been deleted. Example: `user.name@databricks.com` - `number_in_job` (int64) A unique identifier for this job run. This is set to the same value as `run_id`. Example: `455644833` - `original_attempt_run_id` (int64) If this run is a retry of a prior run attempt, this field contains the run_id of the original attempt; otherwise, it is the same as the run_id. Example: `455644833` - `state` (object) Deprecated. Please use the `status` field instead. - `life_cycle_state` (string) A value indicating the run's current lifecycle state. This field is always available in the response. Note: Additional states might be introduced in future releases. Possible values: `PENDING`, `RUNNING`, `TERMINATING`, `TERMINATED`, `SKIPPED`, `INTERNAL_ERROR`, `BLOCKED`, `WAITING_FOR_RETRY`, `QUEUED` - `result_state` (string) A value indicating the run's result. This field is only available for terminal lifecycle states. Note: Additional states might be introduced in future releases. Possible values: `SUCCESS`, `FAILED`, `TIMEDOUT`, `CANCELED`, `MAXIMUM_CONCURRENT_RUNS_REACHED`, `UPSTREAM_CANCELED`, `UPSTREAM_FAILED`, `EXCLUDED`, `SUCCESS_WITH_FAILURES`, `DISABLED` - `state_message` (string) A descriptive message for the current state. This field is unstructured, and its exact format is subject to change. - `user_cancelled_or_timedout` (boolean) A value indicating whether a run was canceled manually by a user or by the scheduler because the run timed out. Default: `false` Example: `false` - `queue_reason` (string) The reason indicating why the run was queued. Example: `Queued due to reaching maximum concurrent runs of 1.` - `schedule` (object) The cron schedule that triggered this run if it was triggered by the periodic scheduler. - `quartz_cron_expression` (string) A Cron expression using Quartz syntax that describes the schedule for a job. See [Cron Trigger](http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html) for details. This field is required. Example: `20 30 * * * ?` - `timezone_id` (string) A Java timezone ID. The schedule for a job is resolved with respect to this timezone. See [Java TimeZone](https://docs.oracle.com/javase/7/docs/api/java/util/TimeZone.html) for details. This field is required. Example: `Europe/London` - `pause_status` (string) Indicate whether this schedule is paused or not. Possible values: `UNPAUSED`, `PAUSED` Default: `UNPAUSED` - `cluster_spec` (object) A snapshot of the job’s cluster specification when this run was created. - `existing_cluster_id` (string) If existing_cluster_id, the ID of an existing cluster that is used for all runs. When running jobs or tasks on an existing cluster, you may need to manually restart the cluster if it stops responding. We suggest running jobs and tasks on new clusters for greater reliability Example: `0923-164208-meows279` - `new_cluster` (object) If new_cluster, a description of a new cluster that is created for each run. - `apply_policy_default_values` (boolean) Default: `false` - `cluster_name` (string) Cluster name requested by the user. This doesn't have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs. - `spark_version` (string) The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available Spark versions can be retrieved by using the [clusters/sparkVersions](https://docs.databricks.com/api/workspace/clusters/sparkversions) API call. - `spark_conf` (object) An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions` respectively. - `aws_attributes` (object) Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. If this value is greater than 0, the cluster driver node in particular will be placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `0` - `availability` (string) Possible values: `SPOT`, `ON_DEMAND`, `SPOT_WITH_FALLBACK` Default: `SPOT_WITH_FALLBACK` - `zone_id` (string) Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like "us-west-2a". The provided availability zone must be in the same region as the deployment. For example, "us-west-2a" is not a valid zone id if the deployment resides in the "us-east-1" region. This is an optional field at cluster creation, and if not specified, the zone "auto" will be used. If the zone specified is "auto", will try to place cluster in a zone with high availability, and will retry placement in a different AZ if there is not enough capacity. The list of available zones as well as the default value can be found by using the `List Zones` method. - `instance_profile_arn` (string) Nodes for this cluster will only be placed on AWS instances with this instance profile. If ommitted, nodes will be placed on instances without an IAM instance profile. The instance profile must have previously been added to the environment by an account administrator. This feature may only be available to certain customer plans. - `spot_bid_price_percent` (int32) The bid price for AWS spot instances, as a percentage of the corresponding instance type's on-demand price. For example, if this field is set to 50, and the cluster needs a new `r3.xlarge` spot instance, then the bid price is half of the price of on-demand `r3.xlarge` instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand `r3.xlarge` instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000. Default: `100` - `ebs_volume_type` (string) The type of EBS volumes that will be launched with this cluster. Possible values: `GENERAL_PURPOSE_SSD`, `THROUGHPUT_OPTIMIZED_HDD` - `ebs_volume_count` (int32) The number of volumes launched for each instance. Users can choose up to 10 volumes. This feature is only enabled for supported node types. Legacy node types cannot specify custom EBS volumes. For node types with no instance store, at least one EBS volume needs to be specified; otherwise, cluster creation will fail. These EBS volumes will be mounted at `/ebs0`, `/ebs1`, and etc. Instance store volumes will be mounted at `/local_disk0`, `/local_disk1`, and etc. If EBS volumes are attached, will configure Spark to use only the EBS volumes for scratch storage because heterogenously sized scratch devices can lead to inefficient disk utilization. If no EBS volumes are attached, will configure Spark to use instance store volumes. Please note that if EBS volumes are specified, then the Spark configuration `spark.local.dir` will be overridden. Default: `0` - `ebs_volume_size` (int32) The size of each EBS volume (in GiB) launched for each instance. For general purpose SSD, this value must be within the range 100 - 4096. For throughput optimized HDD, this value must be within the range 500 - 4096. - `ebs_volume_iops` (int32) If using gp3 volumes, what IOPS to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `ebs_volume_throughput` (int32) If using gp3 volumes, what throughput to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `azure_attributes` (object) Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used. - `log_analytics_info` (object) Defines values necessary to configure and run Azure Log Analytics agent - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `availability` (string) Availability type used for all subsequent nodes past the `first_on_demand` ones. Note: If `first_on_demand` is zero, this availability type will be used for the entire cluster. Possible values: `SPOT_AZURE`, `ON_DEMAND_AZURE`, `SPOT_WITH_FALLBACK_AZURE` Default: `ON_DEMAND_AZURE` - `spot_bid_max_price` (double) The max bid price to be used for Azure spot instances. The Max price for the bid cannot be higher than the on-demand price of the instance. If not specified, the default value is -1, which specifies that the instance cannot be evicted on the basis of price, and only on the basis of availability. Further, the value should > 0 or -1. Default: `-1` - `capacity_reservation_group` (string, GA) The Azure capacity reservation group resource ID to use for launching VMs. When specified, VMs will be launched using the provided capacity reservation. Capacity reservations can only be specified when the workspace uses injected vnet (i.e. customer defined vnet not managed by databricks). Ensure the databricks-login-prod Enterprise Application is granted the following four permissions: 1. Microsoft.Compute/capacityReservationGroups/read 2. Microsoft.Compute/capacityReservationGroups/deploy/action 3. Microsoft.Compute/capacityReservationGroups/capacityReservations/read 4. Microsoft.Compute/capacityReservationGroups/capacityReservations/deploy/action Format: `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/capacityReservationGroups/{capacityReservationGroupName}` - `gcp_attributes` (object) Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used. - `use_preemptible_executors` (boolean) This field determines whether the spark executors will be scheduled to run on preemptible VMs (when set to true) versus standard compute engine VMs (when set to false; default). Note: Soon to be deprecated, use the 'availability' field instead. Default: `false` - `google_service_account` (string) If provided, the cluster will impersonate the google service account when accessing gcloud services (like GCS). The google service account must have previously been added to the environment by an account administrator. - `boot_disk_size` (int32) Boot disk size in GB - `availability` (string) This field determines whether the spark executors will be scheduled to run on preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable. Possible values: `PREEMPTIBLE_GCP`, `ON_DEMAND_GCP`, `PREEMPTIBLE_WITH_FALLBACK_GCP` Default: `ON_DEMAND_GCP` - `zone_id` (string) Identifier for the availability zone in which the cluster resides. This can be one of the following: - "HA" => High availability, spread nodes across availability zones for a deployment region [default]. - "AUTO" => picks an availability zone to schedule the cluster on. - A GCP availability zone => Pick One of the available zones for (machine type + region) from https://cloud.google.com/compute/docs/regions-zones. Default: `HA` - `local_ssd_count` (int32) If provided, each node (workers and driver) in the cluster will have this number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP documentation](https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds) for the supported number of local SSDs for each instance type. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `node_type_id` (string) This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the [clusters/listNodeTypes](https://docs.databricks.com/api/workspace/clusters/listnodetypes) API call. - `driver_node_type_id` (string) The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as `node_type_id` defined above. This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence. - `worker_node_type_flexibility` (object) Flexible node type configuration for worker nodes. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `driver_node_type_flexibility` (object) Flexible node type configuration for the driver node. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `ssh_public_keys` (array of string) SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name `ubuntu` on port `2200`. Up to 10 keys can be specified. - `custom_tags` (object) Additional tags for cluster resources. will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to `default_tags`. Notes: - Currently, allows at most 45 custom tags - Clusters can only reuse cloud resources if the resources' tags are a subset of the cluster tags - `cluster_log_conf` (object) The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every `5 mins`. The destination of driver logs is `$destination/$clusterId/driver`, while the destination of executor logs is `$destination/$clusterId/executor`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs" : { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ "s3": { "destination" : "s3://cluster_log_bucket/prefix", "region" : "us-west-2" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `volumes` (object, GA) destination needs to be provided, e.g. `{ "volumes": { "destination": "/Volumes/catalog/schema/volume/cluster_log" } }` - `spark_env_vars` (object) An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., `export X='Y'`) while launching the driver and workers. In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example below. This ensures that all default databricks managed environmental variables are included as well. Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m", "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS": "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}` - `autotermination_minutes` (int32) Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination. - `enable_elastic_disk` (boolean) Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. **AWS:** Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details. - `init_scripts` (array of object) The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If `cluster_log_conf` is specified, init script logs are sent to `//init_scripts`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs": { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `file` (object) destination needs to be provided, e.g. `{ "file": { "destination": "file:/my/local/file.sh" } }` - `gcs` (object) destination needs to be provided, e.g. `{ "gcs": { "destination": "gs://my-bucket/file.sh" } }` - `abfss` (object) destination needs to be provided, e.g. `abfss://@.dfs.core.windows.net/` - `workspace` (object) destination needs to be provided, e.g. `{ "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }` - `volumes` (object) destination needs to be provided. e.g. `{ \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }` - `docker_image` (object) Custom docker image BYOC - `url` (string) URL of the docker image. - `basic_auth` (object) Basic auth with username and password - `instance_pool_id` (string) The optional ID of the instance pool to which the cluster belongs. - `single_user_name` (string) Single user name if data_security_mode is `SINGLE_USER` - `policy_id` (string) The ID of the cluster policy used to create the cluster if applicable. - `enable_local_disk_encryption` (boolean) Whether to enable LUKS on cluster VMs' local disks - `driver_instance_pool_id` (string) The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned. - `workload_type` (object) - `clients` (object) defined what type of clients can use the cluster. E.g. Notebooks, Jobs - `data_security_mode` (string) Possible values: `NONE`, `SINGLE_USER`, `USER_ISOLATION`, `LEGACY_TABLE_ACL`, `LEGACY_PASSTHROUGH`, `LEGACY_SINGLE_USER`, `LEGACY_SINGLE_USER_STANDARD`, `DATA_SECURITY_MODE_STANDARD`, `DATA_SECURITY_MODE_DEDICATED`, `DATA_SECURITY_MODE_AUTO` - `runtime_engine` (string) Determines the cluster's runtime engine, either standard or Photon. This field is not compatible with legacy `spark_version` values that contain `-photon-`. Remove `-photon-` from the `spark_version` and set `runtime_engine` to `PHOTON`. If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used. Possible values: `NULL`, `STANDARD`, `PHOTON` - `kind` (string) Possible values: `COMPUTE_KIND_UNSPECIFIED`, `CLASSIC_PREVIEW` - `use_ml_runtime` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. `effective_spark_version` is determined by `spark_version` (DBR release), this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not. - `is_single_node` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. When set to true, will automatically set single node related `custom_tags`, `spark_conf`, and `num_workers` - `remote_disk_throughput` (int32) If set, what the configurable throughput (in Mb/s) for the remote disk is. Currently only supported for GCP HYPERDISK_BALANCED disks. - `total_initial_remote_disk_size` (int32) If set, what the total initial volume size (in GB) of the remote disks should be. Supported for GCP. - `dependency_mode` (string, Beta) Controls dependency configuration for the cluster. Possible values: `DEPENDENCY_MODE_UNSPECIFIED`, `DEPENDENCY_MODE_ENVIRONMENTS`, `DEPENDENCY_MODE_CLUSTER_LIBRARIES`, `DEPENDENCY_MODE_AUTO` - `num_workers` (int32) Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes. Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned. - `autoscale` (object) Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later. - `min_workers` (int32) The minimum number of workers to which the cluster can scale down when underutilized. It is also the initial number of workers the cluster will have after creation. - `max_workers` (int32) The maximum number of workers to which the cluster can scale up when overloaded. Note that `max_workers` must be strictly greater than `min_workers`. - `job_cluster_key` (string) If job_cluster_key, this task is executed reusing the cluster specified in `job.settings.job_clusters`. Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `libraries` (array of object) An optional list of libraries to be installed on the cluster. The default value is an empty list. - `jar` (string) URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "s3://my-bucket/library.jar" }`. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI. **Azure:** URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "abfss://my-bucket/library.jar" }`. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI. **GCP:** URI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "gs://my-bucket/library.jar" }`. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI. - `egg` (string) Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above. - `pypi` (object) Specification of a PyPi library to be installed. For example: `{ "package": "simplejson" }` - `package` (string) The name of the pypi package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0". - `repo` (string) The repository where the package can be found. If not specified, the default pip index is used. - `maven` (object) Specification of a maven library to be installed. For example: `{ "coordinates": "org.jsoup:jsoup:1.7.2" }` - `coordinates` (string) Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2". - `repo` (string) Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched. - `exclusions` (array of string) List of dependences to exclude. For example: `["slf4j:slf4j", "*:hadoop-client"]`. Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html. - `cran` (object) Specification of a CRAN library to be installed as part of the library - `package` (string) The name of the CRAN package to install. - `repo` (string) The repository where the package can be found. If not specified, the default CRAN repo is used. - `whl` (string) URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "s3://my-bucket/library.whl" }`. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI. **Azure:** URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "abfss://my-bucket/library.whl" }`. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI. **GCP:** URI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "gs://my-bucket/library.egg" }`. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI. - `requirements` (string) URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: `{ "requirements": "/Workspace/path/to/requirements.txt" }` or `{ "requirements" : "/Volumes/path/to/requirements.txt" }` - `cluster_instance` (object) The cluster used for this run. If the run is specified to use a new cluster, this field is set once the Jobs service has requested a cluster for the run. - `cluster_id` (string) The canonical identifier for the cluster used by a run. This field is always available for runs on existing clusters. For runs on new clusters, it becomes available once the cluster is created. This value can be used to view logs by browsing to `/#setting/sparkui/$cluster_id/driver-logs`. The logs continue to be available after the run completes. The response won’t include this field if the identifier is not available yet. Example: `0923-164208-meows279` - `spark_context_id` (string) The canonical identifier for the Spark context used by a run. This field is filled in once the run begins execution. This value can be used to view the Spark UI by browsing to `/#setting/sparkui/$cluster_id/$spark_context_id`. The Spark UI continues to be available after the run has completed. The response won’t include this field if the identifier is not available yet. - `job_parameters` (array of object) Job-level parameters used in the run - `name` (string) The name of the parameter Example: `table` - `default` (string) The optional default value of the parameter Example: `users` - `value` (string) The value used in the run Example: `customers` - `overriding_parameters` (object) The parameters used for this run. - `pipeline_params` (object) Controls whether the pipeline should perform a full refresh - `full_refresh` (boolean) If true, triggers a full refresh on the spark declarative pipeline. Default: `false` - `refresh_selection` (array of string, Beta) A list of tables to update without fullRefresh. - `full_refresh_selection` (array of string, Beta) A list of tables to update with fullRefresh. - `reset_checkpoint_selection` (array of string, Beta) A list of streaming flows to reset checkpoints without clearing data. - `refresh_flow_selection` (array of string, Beta) Flow names to selectively refresh. These are unioned with other selective refresh options (refresh_selection, full_refresh_selection) to determine the final set of flows to refresh. - `trigger` (string) Possible values: `PERIODIC`, `ONE_TIME`, `RETRY`, `RUN_JOB_TASK`, `FILE_ARRIVAL`, `CONTINUOUS`, `TABLE`, `CONTINUOUS_RESTART` - `trigger_info` (object) - `run_id` (int64) The run id of the Run Job task run - `run_name` (string) An optional name for the run. The maximum length is 4096 bytes in UTF-8 encoding. Default: `Untitled` Example: `A multitask job run` Constraints: `<= 4096 characters` - `run_page_url` (string) The URL to the detail page of the run. Example: `https://my-workspace.cloud.databricks.com/#job/11223344/run/123` - `run_type` (string) Possible values: `JOB_RUN`, `WORKFLOW_RUN`, `SUBMIT_RUN` - `tasks` (array of object) The list of tasks performed by the run. Each task has its own `run_id` which you can use to call `JobsGetOutput` to retrieve the run results. If more than 100 tasks are available, you can paginate through them using :method:jobs/getrun. Use the `next_page_token` field at the object root to determine if more results are available. Constraints: `<= 100 items` - `run_id` (int64) The ID of the task run. Example: `99887766` - `state` (object) Deprecated. Please use the `status` field instead. - `life_cycle_state` (string) A value indicating the run's current lifecycle state. This field is always available in the response. Note: Additional states might be introduced in future releases. Possible values: `PENDING`, `RUNNING`, `TERMINATING`, `TERMINATED`, `SKIPPED`, `INTERNAL_ERROR`, `BLOCKED`, `WAITING_FOR_RETRY`, `QUEUED` - `result_state` (string) A value indicating the run's result. This field is only available for terminal lifecycle states. Note: Additional states might be introduced in future releases. Possible values: `SUCCESS`, `FAILED`, `TIMEDOUT`, `CANCELED`, `MAXIMUM_CONCURRENT_RUNS_REACHED`, `UPSTREAM_CANCELED`, `UPSTREAM_FAILED`, `EXCLUDED`, `SUCCESS_WITH_FAILURES`, `DISABLED` - `state_message` (string) A descriptive message for the current state. This field is unstructured, and its exact format is subject to change. - `user_cancelled_or_timedout` (boolean) A value indicating whether a run was canceled manually by a user or by the scheduler because the run timed out. Default: `false` Example: `false` - `queue_reason` (string) The reason indicating why the run was queued. Example: `Queued due to reaching maximum concurrent runs of 1.` - `run_page_url` (string) - `cluster_instance` (object) The cluster used for this run. If the run is specified to use a new cluster, this field is set once the Jobs service has requested a cluster for the run. - `cluster_id` (string) The canonical identifier for the cluster used by a run. This field is always available for runs on existing clusters. For runs on new clusters, it becomes available once the cluster is created. This value can be used to view logs by browsing to `/#setting/sparkui/$cluster_id/driver-logs`. The logs continue to be available after the run completes. The response won’t include this field if the identifier is not available yet. Example: `0923-164208-meows279` - `spark_context_id` (string) The canonical identifier for the Spark context used by a run. This field is filled in once the run begins execution. This value can be used to view the Spark UI by browsing to `/#setting/sparkui/$cluster_id/$spark_context_id`. The Spark UI continues to be available after the run has completed. The response won’t include this field if the identifier is not available yet. - `attempt_number` (int32) The sequence number of this run attempt for a triggered job run. The initial attempt of a run has an attempt_number of 0. If the initial run attempt fails, and the job has a retry policy (`max_retries` > 0), subsequent runs are created with an `original_attempt_run_id` of the original attempt’s ID and an incrementing `attempt_number`. Runs are retried only until they succeed, and the maximum `attempt_number` is the same as the `max_retries` value for the job. Example: `0` - `git_source` (object) An optional specification for a remote Git repository containing the source code used by tasks. Version-controlled source code is supported by notebook, dbt, Python script, and SQL File tasks. If `git_source` is set, these tasks retrieve the file from the remote repository by default. However, this behavior can be overridden by setting `source` to `WORKSPACE` on the task. Note: dbt and SQL File tasks support only version-controlled sources. If dbt or SQL File tasks are used, `git_source` must be defined on the job. - `git_url` (string) URL of the repository to be cloned by this job. Example: `https://github.com/databricks/databricks-cli` Constraints: `<= 300 characters` - `git_provider` (string) Unique identifier of the service used to host the Git repository. The value is case insensitive. - `git_branch` (string) Name of the branch to be checked out and used by this job. This field cannot be specified in conjunction with git_tag or git_commit. Example: `main` Constraints: `<= 255 characters` - `git_tag` (string) Name of the tag to be checked out and used by this job. This field cannot be specified in conjunction with git_branch or git_commit. Example: `release-1.0.0` Constraints: `<= 255 characters` - `git_commit` (string) Commit to be checked out and used by this job. This field cannot be specified in conjunction with git_branch or git_tag. Example: `e0056d01` Constraints: `<= 64 characters` - `git_snapshot` (object) - `used_commit` (string) Commit that was used to execute the run. If git_branch was specified, this points to the HEAD of the branch at the time of the run; if git_tag was specified, this points to the commit the tag points to. Example: `4506fdf41e9fa98090570a34df7a5bce163ff15f` - `sparse_checkout` (object, GA) - `patterns` (array of string) List of patterns to include for sparse checkout. - `resolved_values` (object) Parameter values including resolved references - `notebook_task` (object) - `base_parameters` (object) - `spark_jar_task` (object) - `parameters` (array of string) - `spark_python_task` (object) - `spark_submit_task` (object) - `python_wheel_task` (object) - `parameters` (array of string) - `named_parameters` (object) - `dbt_task` (object) - `commands` (array of string) - `sql_task` (object) - `parameters` (object) - `run_job_task` (object) - `parameters` (object) - `job_parameters` (object) - `condition_task` (object) - `left` (string) - `right` (string) - `simulation_task` (object) - `parameters` (object) - `pipeline_task` (object, Beta) - `parameters` (object, Beta) Key/value-map of parameters passed to the pipeline execution. Limited to 10k characters in total. - `status` (object) - `state` (string) Possible values: `BLOCKED`, `PENDING`, `QUEUED`, `RUNNING`, `TERMINATING`, `TERMINATED`, `WAITING` - `termination_details` (object) If the run is in a TERMINATING or TERMINATED state, details about the reason for terminating the run. - `code` (string) Possible values: `SUCCESS`, `CANCELED`, `DRIVER_ERROR`, `CLUSTER_ERROR`, `REPOSITORY_CHECKOUT_FAILED`, `INVALID_CLUSTER_REQUEST`, `WORKSPACE_RUN_LIMIT_EXCEEDED`, `FEATURE_DISABLED`, `CLUSTER_REQUEST_LIMIT_EXCEEDED`, `STORAGE_ACCESS_ERROR`, `RUN_EXECUTION_ERROR`, `UNAUTHORIZED_ERROR`, `LIBRARY_INSTALLATION_ERROR`, `MAX_CONCURRENT_RUNS_EXCEEDED`, `MAX_SPARK_CONTEXTS_EXCEEDED`, `RESOURCE_NOT_FOUND`, `INVALID_RUN_CONFIGURATION`, `INTERNAL_ERROR`, `CLOUD_FAILURE`, `MAX_JOB_QUEUE_SIZE_EXCEEDED`, `SKIPPED`, `USER_CANCELED`, `BUDGET_POLICY_LIMIT_EXCEEDED`, `DISABLED`, `SUCCESS_WITH_FAILURES`, `BREAKING_CHANGE` - `type` (string) Possible values: `SUCCESS`, `INTERNAL_ERROR`, `CLIENT_ERROR`, `CLOUD_FAILURE` - `message` (string) A descriptive message with the termination details. This field is unstructured and the format might change. - `queue_details` (object) If the run was queued, details about the reason for queuing the run. - `code` (string) Possible values: `ACTIVE_RUNS_LIMIT_REACHED`, `MAX_CONCURRENT_RUNS_REACHED`, `ACTIVE_RUN_JOB_TASKS_LIMIT_REACHED` - `message` (string) A descriptive message with the queuing details. This field is unstructured, and its exact format is subject to change. - `effective_performance_target` (string) The actual performance target used by the serverless run during execution. This can differ from the client-set performance target on the request depending on whether the performance mode is supported by the job type. * `STANDARD`: Enables cost-efficient execution of serverless workloads. * `PERFORMANCE_OPTIMIZED`: Prioritizes fast startup and execution times through rapid scaling and optimized cluster performance. Possible values: `PERFORMANCE_TARGET_UNSPECIFIED`, `PERFORMANCE_OPTIMIZED`, `STANDARD` - `task_key` (string) A unique name for the task. This field is used to refer to this task from other tasks. This field is required and must be unique within its parent job. On Update or Reset, this field is used to reference the tasks to be updated or reset. Example: `Task_Key` Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `description` (string) An optional description for this task. Example: `This is the description for this task.` Constraints: `<= 1000 characters` - `depends_on` (array of object) An optional array of objects specifying the dependency graph of the task. All tasks specified in this field must complete successfully before executing this task. The key is `task_key`, and the value is the name assigned to the dependent task. - `task_key` (string) The name of the task this task depends on. Example: `Task_Key` Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `outcome` (string) Can only be specified on condition task dependencies. The outcome of the dependent task that must be met for this task to run. - `run_if` (string) An optional value indicating the condition that determines whether the task should be run once its dependencies have been completed. When omitted, defaults to `ALL_SUCCESS`. See :method:jobs/create for a list of possible values. Possible values: `ALL_SUCCESS`, `ALL_DONE`, `NONE_FAILED`, `AT_LEAST_ONE_SUCCESS`, `ALL_FAILED`, `AT_LEAST_ONE_FAILED` Example: `ALL_SUCCESS` - `timeout_seconds` (int32) An optional timeout applied to each run of this job task. A value of `0` means no timeout. Default: `0` Example: `86400` - `email_notifications` (object) An optional set of email addresses notified when the task run begins or completes. The default behavior is to not send any emails. - `on_start` (array of string) A list of email addresses to be notified when a run begins. If not specified on job creation, reset, or update, the list is empty, and notifications are not sent. Example: `user.name@databricks.com` - `on_success` (array of string) A list of email addresses to be notified when a run successfully completes. A run is considered to have completed successfully if it ends with a `TERMINATED` `life_cycle_state` and a `SUCCESS` result_state. If not specified on job creation, reset, or update, the list is empty, and notifications are not sent. Example: `user.name@databricks.com` - `on_failure` (array of string) A list of email addresses to be notified when a run unsuccessfully completes. A run is considered to have completed unsuccessfully if it ends with an `INTERNAL_ERROR` `life_cycle_state` or a `FAILED`, or `TIMED_OUT` result_state. If this is not specified on job creation, reset, or update the list is empty, and notifications are not sent. Example: `user.name@databricks.com` - `on_duration_warning_threshold_exceeded` (array of string) A list of email addresses to be notified when the duration of a run exceeds the threshold specified for the `RUN_DURATION_SECONDS` metric in the `health` field. If no rule for the `RUN_DURATION_SECONDS` metric is specified in the `health` field for the job, notifications are not sent. Example: `user.name@databricks.com` - `on_streaming_backlog_exceeded` (array of string, Public Preview) A list of email addresses to notify when any streaming backlog thresholds are exceeded for any stream. Streaming backlog thresholds can be set in the `health` field using the following metrics: `STREAMING_BACKLOG_BYTES`, `STREAMING_BACKLOG_RECORDS`, `STREAMING_BACKLOG_SECONDS`, or `STREAMING_BACKLOG_FILES`. Alerting is based on the 10-minute average of these metrics. If the issue persists, notifications are resent every 30 minutes. Example: `user.name@databricks.com` - `no_alert_for_skipped_runs` (boolean) If true, do not send email to recipients specified in `on_failure` if the run is skipped. This field is `deprecated`. Please use the `notification_settings.no_alert_for_skipped_runs` field. Default: `false` Example: `false` - `health` (object) - `rules` (array of object) - `metric` (string) Possible values: `RUN_DURATION_SECONDS`, `STREAMING_BACKLOG_BYTES`, `STREAMING_BACKLOG_RECORDS`, `STREAMING_BACKLOG_SECONDS`, `STREAMING_BACKLOG_FILES` Example: `RUN_DURATION_SECONDS` - `op` (string) Possible values: `GREATER_THAN` Example: `GREATER_THAN` - `value` (int64) Specifies the threshold value that the health metric should obey to satisfy the health rule. Example: `10` - `notification_settings` (object) Optional notification settings that are used when sending notifications to each of the `email_notifications` and `webhook_notifications` for this task run. - `no_alert_for_skipped_runs` (boolean) If true, do not send notifications to recipients specified in `on_failure` if the run is skipped. Default: `false` Example: `false` - `no_alert_for_canceled_runs` (boolean) If true, do not send notifications to recipients specified in `on_failure` if the run is canceled. Default: `false` Example: `false` - `alert_on_last_attempt` (boolean) If true, do not send notifications to recipients specified in `on_start` for the retried runs and do not send notifications to recipients specified in `on_failure` until the last retry of the run. Default: `false` Example: `false` - `webhook_notifications` (object) A collection of system notification IDs to notify when the run begins or completes. The default behavior is to not send any system notifications. Task webhooks respect the task notification settings. - `on_start` (array of object) An optional list of system notification IDs to call when the run starts. A maximum of 3 destinations can be specified for the `on_start` property. - `id` (string) - `on_success` (array of object) An optional list of system notification IDs to call when the run completes successfully. A maximum of 3 destinations can be specified for the `on_success` property. - `id` (string) - `on_failure` (array of object) An optional list of system notification IDs to call when the run fails. A maximum of 3 destinations can be specified for the `on_failure` property. - `id` (string) - `on_duration_warning_threshold_exceeded` (array of object) An optional list of system notification IDs to call when the duration of a run exceeds the threshold specified for the `RUN_DURATION_SECONDS` metric in the `health` field. A maximum of 3 destinations can be specified for the `on_duration_warning_threshold_exceeded` property. - `id` (string) - `on_streaming_backlog_exceeded` (array of object, Public Preview) An optional list of system notification IDs to call when any streaming backlog thresholds are exceeded for any stream. Streaming backlog thresholds can be set in the `health` field using the following metrics: `STREAMING_BACKLOG_BYTES`, `STREAMING_BACKLOG_RECORDS`, `STREAMING_BACKLOG_SECONDS`, or `STREAMING_BACKLOG_FILES`. Alerting is based on the 10-minute average of these metrics. If the issue persists, notifications are resent every 30 minutes. A maximum of 3 destinations can be specified for the `on_streaming_backlog_exceeded` property. - `id` (string) - `environment_key` (string) The key that references an environment spec in a job. This field is required for Python script, Python wheel and dbt tasks when using serverless compute. Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `disabled` (boolean, GA) An optional flag to disable the task. If set to true, the task will not run even if it is part of a job. Default: `false` - `compute` (object, Beta) Task level compute configuration. - `hardware_accelerator` (string, Beta) Hardware accelerator configuration for Serverless GPU workloads. Possible values: `GPU_1X_A10`, `GPU_8X_H100` - `notebook_task` (object) The task runs a notebook when the `notebook_task` field is present. - `notebook_path` (string) The path of the notebook to be run in the workspace or remote repository. For notebooks stored in the workspace, the path must be absolute and begin with a slash. For notebooks stored in a remote repository, the path must be relative. This field is required. Example: `/Users/user.name@databricks.com/notebook_to_run` - `base_parameters` (object) Base parameters to be used for each run of this job. If the run is initiated by a call to :method:jobs/run Now with parameters specified, the two parameters maps are merged. If the same key is specified in `base_parameters` and in `run-now`, the value from `run-now` is used. Use [Task parameter variables](https://docs.databricks.com/jobs.html#parameter-variables) to set parameters containing information about job runs. If the notebook takes a parameter that is not specified in the job’s `base_parameters` or the `run-now` override parameters, the default value from the notebook is used. Retrieve these parameters in a notebook using [dbutils.widgets.get](https://docs.databricks.com/dev-tools/databricks-utils.html#dbutils-widgets). The JSON representation of this field cannot exceed 1MB. - `source` (string) Optional location type of the notebook. When set to `WORKSPACE`, the notebook will be retrieved from the local workspace. When set to `GIT`, the notebook will be retrieved from a Git repository defined in `git_source`. If the value is empty, the task will use `GIT` if `git_source` is defined and `WORKSPACE` otherwise. * `WORKSPACE`: Notebook is located in workspace. * `GIT`: Notebook is located in cloud Git provider. Possible values: `WORKSPACE`, `GIT` Example: `WORKSPACE` - `warehouse_id` (string) Optional `warehouse_id` to run the notebook on a SQL warehouse. Classic SQL warehouses are NOT supported, please use serverless or pro SQL warehouses. Note that SQL warehouses only support SQL cells; if the notebook contains non-SQL cells, the run will fail. Example: `ab12cd34efgh567i` - `spark_jar_task` (object) The task runs a JAR when the `spark_jar_task` field is present. - `jar_uri` (string) Deprecated since 04/2016. For classic compute, provide a `jar` through the `libraries` field instead. For serverless compute, provide a `jar` though the `java_dependencies` field inside the `environments` list. See the examples of classic and serverless compute usage at the top of the page. - `main_class_name` (string) The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. The code must use `SparkContext.getOrCreate` to obtain a Spark context; otherwise, runs of the job fail. Example: `com.databricks.ComputeModels` - `parameters` (array of string) Parameters passed to the main method. Use [Task parameter variables](https://docs.databricks.com/jobs.html#parameter-variables) to set parameters containing information about job runs. - `run_as_repl` (boolean) Deprecated. A value of `false` is no longer supported. Default: `true` - `spark_python_task` (object) The task runs a Python file when the `spark_python_task` field is present. - `python_file` (string) The Python file to be executed. Cloud file URIs (such as dbfs:/, s3:/, adls:/, gcs:/) and workspace paths are supported. For python files stored in the workspace, the path must be absolute and begin with `/`. For files stored in a remote repository, the path must be relative. This field is required. Example: `dbfs:/path/to/file.py` - `parameters` (array of string) Command line parameters passed to the Python file. Use [Task parameter variables](https://docs.databricks.com/jobs.html#parameter-variables) to set parameters containing information about job runs. - `source` (string) Optional location type of the Python file. When set to `WORKSPACE` or not specified, the file will be retrieved from the local workspace or cloud location (if the `python_file` has a URI format). When set to `GIT`, the Python file will be retrieved from a Git repository defined in `git_source`. * `WORKSPACE`: The Python file is located in a workspace or at a cloud filesystem URI. * `GIT`: The Python file is located in a remote Git repository. Possible values: `WORKSPACE`, `GIT` Example: `WORKSPACE` - `spark_submit_task` (object) (Legacy) The task runs the spark-submit script when the spark_submit_task field is present. Databricks recommends using the spark_jar_task instead; see [Spark Submit task for jobs](https://docs.databricks.com/jobs/spark-submit). - `parameters` (array of string) Command-line parameters passed to spark submit. Use [Task parameter variables](https://docs.databricks.com/jobs.html#parameter-variables) to set parameters containing information about job runs. - `pipeline_task` (object) The task triggers a pipeline update when the `pipeline_task` field is present. Only pipelines configured to use triggered more are supported. - `pipeline_id` (string) The full name of the pipeline task to execute. Example: `a12cd3e4-0ab1-1abc-1a2b-1a2bcd3e4fg5` - `parameters` (object, Beta) Key/value-map of parameters passed to the pipeline execution. Limited to 10k characters in total. - `full_refresh` (boolean) If true, triggers a full refresh on the spark declarative pipeline. Default: `false` - `refresh_selection` (array of string, Beta) A list of tables to update without fullRefresh. - `full_refresh_selection` (array of string, Beta) A list of tables to update with fullRefresh. - `reset_checkpoint_selection` (array of string, Beta) A list of streaming flows to reset checkpoints without clearing data. - `refresh_flow_selection` (array of string, Beta) Flow names to selectively refresh. These are unioned with other selective refresh options (refresh_selection, full_refresh_selection) to determine the final set of flows to refresh. - `python_wheel_task` (object) The task runs a Python wheel when the `python_wheel_task` field is present. - `package_name` (string) Name of the package to execute - `entry_point` (string) Named entry point to use, if it does not exist in the metadata of the package it executes the function from the package directly using `$packageName.$entryPoint()` - `parameters` (array of string) Command-line parameters passed to Python wheel task. Leave it empty if `named_parameters` is not null. - `named_parameters` (object) Command-line parameters passed to Python wheel task in the form of `["--name=task", "--data=dbfs:/path/to/data.json"]`. Leave it empty if `parameters` is not null. - `dbt_task` (object) The task runs one or more dbt commands when the `dbt_task` field is present. The dbt task requires both Databricks SQL and the ability to use a serverless or a pro SQL warehouse. - `project_directory` (string) Path to the project directory. Optional for Git sourced tasks, in which case if no value is provided, the root of the Git repository is used. - `commands` (array of string) A list of dbt commands to execute. All commands must start with `dbt`. This parameter must not be empty. A maximum of up to 10 commands can be provided. - `schema` (string) Optional schema to write to. This parameter is only used when a warehouse_id is also provided. If not provided, the `default` schema is used. - `warehouse_id` (string) ID of the SQL warehouse to connect to. If provided, we automatically generate and provide the profile and connection details to dbt. It can be overridden on a per-command basis by using the `--profiles-dir` command line argument. Example: `30dade0507d960d1` - `profiles_directory` (string) Optional (relative) path to the profiles directory. Can only be specified if no warehouse_id is specified. If no warehouse_id is specified and this folder is unset, the root directory is used. - `catalog` (string) Optional name of the catalog to use. The value is the top level in the 3-level namespace of Unity Catalog (catalog / schema / relation). The catalog value can only be specified if a warehouse_id is specified. Requires dbt-databricks >= 1.1.1. Example: `main` - `source` (string) Optional location type of the project directory. When set to `WORKSPACE`, the project will be retrieved from the local workspace. When set to `GIT`, the project will be retrieved from a Git repository defined in `git_source`. If the value is empty, the task will use `GIT` if `git_source` is defined and `WORKSPACE` otherwise. * `WORKSPACE`: Project is located in workspace. * `GIT`: Project is located in cloud Git provider. Possible values: `WORKSPACE`, `GIT` Example: `WORKSPACE` - `sql_task` (object) The task runs a SQL query or file, or it refreshes a SQL alert or a legacy SQL dashboard when the `sql_task` field is present. - `parameters` (object) Parameters to be used for each run of this job. The SQL alert task does not support custom parameters. - `query` (object) If query, indicates that this job must execute a SQL query. - `query_id` (string) The canonical identifier of the SQL query. - `dashboard` (object) If dashboard, indicates that this job must refresh a SQL dashboard. - `dashboard_id` (string) The canonical identifier of the SQL dashboard. - `subscriptions` (array of object) If specified, dashboard snapshots are sent to subscriptions. - `custom_subject` (string) Subject of the email sent to subscribers of this task. - `pause_subscriptions` (boolean) If true, the dashboard snapshot is not taken, and emails are not sent to subscribers. Default: `false` - `alert` (object) If alert, indicates that this job must refresh a SQL alert. - `alert_id` (string) The canonical identifier of the SQL alert. - `subscriptions` (array of object) If specified, alert notifications are sent to subscribers. - `pause_subscriptions` (boolean) If true, the alert notifications are not sent to subscribers. Default: `false` - `file` (object) If file, indicates that this job runs a SQL file in a remote Git repository. - `path` (string) Path of the SQL file. Must be relative if the source is a remote Git repository and absolute for workspace paths. - `source` (string) Optional location type of the SQL file. When set to `WORKSPACE`, the SQL file will be retrieved from the local workspace. When set to `GIT`, the SQL file will be retrieved from a Git repository defined in `git_source`. If the value is empty, the task will use `GIT` if `git_source` is defined and `WORKSPACE` otherwise. * `WORKSPACE`: SQL file is located in workspace. * `GIT`: SQL file is located in cloud Git provider. Possible values: `WORKSPACE`, `GIT` Example: `WORKSPACE` - `warehouse_id` (string) The canonical identifier of the SQL warehouse. Recommended to use with serverless or pro SQL warehouses. Classic SQL warehouses are only supported for SQL alert, dashboard and query tasks and are limited to scheduled single-task jobs. - `run_job_task` (object) The task triggers another job when the `run_job_task` field is present. - `job_id` (int64) ID of the job to trigger. - `job_parameters` (object) Job-level parameters used to trigger the job. - `pipeline_params` (object) Controls whether the pipeline should perform a full refresh - `full_refresh` (boolean) If true, triggers a full refresh on the spark declarative pipeline. Default: `false` - `refresh_selection` (array of string, Beta) A list of tables to update without fullRefresh. - `full_refresh_selection` (array of string, Beta) A list of tables to update with fullRefresh. - `reset_checkpoint_selection` (array of string, Beta) A list of streaming flows to reset checkpoints without clearing data. - `refresh_flow_selection` (array of string, Beta) Flow names to selectively refresh. These are unioned with other selective refresh options (refresh_selection, full_refresh_selection) to determine the final set of flows to refresh. - `condition_task` (object) The task evaluates a condition that can be used to control the execution of other tasks when the `condition_task` field is present. The condition task does not require a cluster to execute and does not support retries or notifications. - `op` (string) * `EQUAL_TO`, `NOT_EQUAL` operators perform string comparison of their operands. This means that `“12.0” == “12”` will evaluate to `false`. * `GREATER_THAN`, `GREATER_THAN_OR_EQUAL`, `LESS_THAN`, `LESS_THAN_OR_EQUAL` operators perform numeric comparison of their operands. `“12.0” >= “12”` will evaluate to `true`, `“10.0” >= “12”` will evaluate to `false`. The boolean comparison to task values can be implemented with operators `EQUAL_TO`, `NOT_EQUAL`. If a task value was set to a boolean value, it will be serialized to `“true”` or `“false”` for the comparison. Possible values: `EQUAL_TO`, `GREATER_THAN`, `GREATER_THAN_OR_EQUAL`, `LESS_THAN`, `LESS_THAN_OR_EQUAL`, `NOT_EQUAL` - `left` (string) The left operand of the condition task. Can be either a string value or a job state or parameter reference. - `right` (string) The right operand of the condition task. Can be either a string value or a job state or parameter reference. - `outcome` (string) The condition expression evaluation result. Filled in if the task was successfully completed. Can be `"true"` or `"false"` - `for_each_task` (object) The task executes a nested task for every input provided when the `for_each_task` field is present. - `inputs` (string) Array for task to iterate on. This can be a JSON string or a reference to an array parameter. Constraints: `<= 5000 characters` - `concurrency` (int32) An optional maximum allowed number of concurrent runs of the task. Set this value if you want to be able to execute multiple runs of the task concurrently. Example: `20` Constraints: `[ 1 .. 100 ]` - `task` (object) Configuration for the task that will be run for each element in the array - `task_key` (string) A unique name for the task. This field is used to refer to this task from other tasks. This field is required and must be unique within its parent job. On Update or Reset, this field is used to reference the tasks to be updated or reset. Example: `Task_Key` Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `depends_on` (array of object) An optional array of objects specifying the dependency graph of the task. All tasks specified in this field must complete before executing this task. The task will run only if the `run_if` condition is true. The key is `task_key`, and the value is the name assigned to the dependent task. - `run_if` (string) An optional value specifying the condition determining whether the task is run once its dependencies have been completed. * `ALL_SUCCESS`: All dependencies have executed and succeeded * `AT_LEAST_ONE_SUCCESS`: At least one dependency has succeeded * `NONE_FAILED`: None of the dependencies have failed and at least one was executed * `ALL_DONE`: All dependencies have been completed * `AT_LEAST_ONE_FAILED`: At least one dependency failed * `ALL_FAILED`: ALl dependencies have failed Possible values: `ALL_SUCCESS`, `ALL_DONE`, `NONE_FAILED`, `AT_LEAST_ONE_SUCCESS`, `ALL_FAILED`, `AT_LEAST_ONE_FAILED` Default: `ALL_SUCCESS` Example: `ALL_SUCCESS` - `timeout_seconds` (int32) An optional timeout applied to each run of this job task. A value of `0` means no timeout. Default: `0` Example: `86400` - `health` (object) - `email_notifications` (object) An optional set of email addresses that is notified when runs of this task begin or complete as well as when this task is deleted. The default behavior is to not send any emails. - `notification_settings` (object) Optional notification settings that are used when sending notifications to each of the `email_notifications` and `webhook_notifications` for this task. - `webhook_notifications` (object) A collection of system notification IDs to notify when runs of this task begin or complete. The default behavior is to not send any system notifications. - `description` (string) An optional description for this task. Example: `This is the description for this task.` Constraints: `<= 1000 characters` - `environment_key` (string) The key that references an environment spec in a job. This field is required for Python script, Python wheel and dbt tasks when using serverless compute. Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `disabled` (boolean, GA) An optional flag to disable the task. If set to true, the task will not run even if it is part of a job. Default: `false` - `compute` (object, Beta) Task level compute configuration. - `notebook_task` (object) The task runs a notebook when the `notebook_task` field is present. - `spark_jar_task` (object) The task runs a JAR when the `spark_jar_task` field is present. - `spark_python_task` (object) The task runs a Python file when the `spark_python_task` field is present. - `spark_submit_task` (object) (Legacy) The task runs the spark-submit script when the spark_submit_task field is present. Databricks recommends using the spark_jar_task instead; see [Spark Submit task for jobs](https://docs.databricks.com/jobs/spark-submit). - `pipeline_task` (object) The task triggers a pipeline update when the `pipeline_task` field is present. Only pipelines configured to use triggered more are supported. - `python_wheel_task` (object) The task runs a Python wheel when the `python_wheel_task` field is present. - `dbt_task` (object) The task runs one or more dbt commands when the `dbt_task` field is present. The dbt task requires both Databricks SQL and the ability to use a serverless or a pro SQL warehouse. - `sql_task` (object) The task runs a SQL query or file, or it refreshes a SQL alert or a legacy SQL dashboard when the `sql_task` field is present. - `run_job_task` (object) The task triggers another job when the `run_job_task` field is present. - `condition_task` (object) The task evaluates a condition that can be used to control the execution of other tasks when the `condition_task` field is present. The condition task does not require a cluster to execute and does not support retries or notifications. - `for_each_task` (object) The task executes a nested task for every input provided when the `for_each_task` field is present. - `clean_rooms_notebook_task` (object, GA) The task runs a [clean rooms](https://docs.databricks.com/clean-rooms/index.html) notebook when the `clean_rooms_notebook_task` field is present. - `alert_task` (object, Public Preview) The task evaluates a alert and sends notifications to subscribers when the `alert_task` field is present. - `power_bi_task` (object, Public Preview) The task triggers a Power BI semantic model update when the `power_bi_task` field is present. - `dashboard_task` (object) The task refreshes a dashboard and sends a snapshot to subscribers. - `ai_runtime_task` (object, Public Preview) The task runs a multi-gpu compute workload on Databricks AI Runtime. Specify the accelerator type and count, the command to run, and where the workload's code and MLflow output are stored. - `existing_cluster_id` (string) If existing_cluster_id, the ID of an existing cluster that is used for all runs. When running jobs or tasks on an existing cluster, you may need to manually restart the cluster if it stops responding. We suggest running jobs and tasks on new clusters for greater reliability Example: `0923-164208-meows279` - `new_cluster` (object) If new_cluster, a description of a new cluster that is created for each run. - `job_cluster_key` (string) If job_cluster_key, this task is executed reusing the cluster specified in `job.settings.job_clusters`. Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `libraries` (array of object) An optional list of libraries to be installed on the cluster. The default value is an empty list. - `max_retries` (int32) An optional maximum number of times to retry an unsuccessful run. A run is considered to be unsuccessful if it completes with the `FAILED` result_state or `INTERNAL_ERROR` `life_cycle_state`. The value `-1` means to retry indefinitely and the value `0` means to never retry. Default: `0` Example: `10` - `min_retry_interval_millis` (int32) An optional minimal interval in milliseconds between the start of the failed run and the subsequent retry run. The default behavior is that unsuccessful runs are immediately retried. Example: `2000` - `retry_on_timeout` (boolean) An optional policy to specify whether to retry a job when it times out. The default behavior is to not retry on timeout. Default: `false` Example: `true` - `disable_auto_optimization` (boolean) An option to disable auto optimization in serverless Default: `false` Example: `true` - `clean_rooms_notebook_task` (object, GA) The task runs a [clean rooms](https://docs.databricks.com/clean-rooms/index.html) notebook when the `clean_rooms_notebook_task` field is present. - `clean_room_name` (string) The clean room that the notebook belongs to. - `notebook_name` (string) Name of the notebook being run. - `etag` (string) Checksum to validate the freshness of the notebook resource (i.e. the notebook being run is the latest version). It can be fetched by calling the :method:cleanroomassets/get API. - `notebook_base_parameters` (object) Base parameters to be used for the clean room notebook job. - `alert_task` (object, Public Preview) The task evaluates a alert and sends notifications to subscribers when the `alert_task` field is present. - `alert_id` (string, Public Preview) The alert_id is the canonical identifier of the alert. - `warehouse_id` (string, Public Preview) The warehouse_id identifies the warehouse settings used by the alert task. - `workspace_path` (string, Public Preview) The workspace_path is the path to the alert file in the workspace. The path: * must start with "/Workspace" * must be a normalized path. User has to select only one of alert_id or workspace_path to identify the alert. Example: `/Workspace/Users/user@company.com/example.dbalert.json` - `subscribers` (array of object, Public Preview) The subscribers receive alert evaluation result notifications after the alert task is completed. The number of subscriptions is limited to 100. - `user_name` (string, Public Preview) A valid workspace email address. Example: `user@databricks.com` - `destination_id` (string, Public Preview) Example: `ce8f875a-11ad-4117-bcc6-c5a542822aee` - `power_bi_task` (object, Public Preview) The task triggers a Power BI semantic model update when the `power_bi_task` field is present. - `tables` (array of object, Public Preview) The tables to be exported to Power BI - `name` (string, Public Preview) The table name in - `catalog` (string, Public Preview) The catalog name in - `schema` (string, Public Preview) The schema name in - `storage_mode` (string, Public Preview) The Power BI storage mode of the table Possible values: `DIRECT_QUERY`, `IMPORT`, `DUAL` - `warehouse_id` (string, Public Preview) The SQL warehouse ID to use as the Power BI data source - `power_bi_model` (object, Public Preview) The semantic model to update - `workspace_name` (string, Public Preview) The name of the Power BI workspace of the model - `model_name` (string, Public Preview) The name of the Power BI model - `storage_mode` (string, Public Preview) The default storage mode of the Power BI model Possible values: `DIRECT_QUERY`, `IMPORT`, `DUAL` - `authentication_method` (string, Public Preview) How the published Power BI model authenticates to Possible values: `OAUTH`, `PAT` - `overwrite_existing` (boolean, Public Preview) Whether to overwrite existing Power BI models - `connection_resource_name` (string, Public Preview) The resource name of the UC connection to authenticate from to Power BI - `refresh_after_update` (boolean, Public Preview) Whether the model should be refreshed after the update - `dashboard_task` (object) The task refreshes a dashboard and sends a snapshot to subscribers. - `subscription` (object) Optional: subscription configuration for sending the dashboard snapshot. - `subscribers` (array of object) The list of subscribers to send the snapshot of the dashboard to. - `paused` (boolean) When true, the subscription will not send emails. Example: `false` - `custom_subject` (string) Optional: Allows users to specify a custom subject line on the email sent to subscribers. Example: `Custom email subject` Constraints: `<= 250 characters` - `warehouse_id` (string) Optional: The warehouse id to execute the dashboard with for the schedule. If not specified, the default warehouse of the dashboard will be used. Example: `47bb1c472649e711` - `dashboard_id` (string) The identifier of the dashboard to refresh. Example: `01ef0cb45e2a1da4a61950e9b8789ce9` - `ai_runtime_task` (object, Public Preview) The task runs a multi-gpu compute workload on Databricks AI Runtime. Specify the accelerator type and count, the command to run, and where the workload's code and MLflow output are stored. - `experiment` (string, Public Preview) MLflow experiment name for this run. If an experiment with this name already exists under the calling user, the run is appended to it; otherwise a new experiment is created. To target a specific MLflow storage location (for example, when running as a service principal), set `mlflow_experiment_directory`. Example: `llama-fine-tune` - `deployments` (array of object, Public Preview) Deployment specs for this task. Exactly one deployment is currently supported (a single entry where every node runs the same command); this is a current-Preview constraint. Role-split workloads (driver + worker, parameter server, separate eval node, etc.) with multiple entries are the eventual intent but not yet supported. - `command_path` (string, Public Preview) Workspace path of the script to run on each node in this deployment. Upload the script to this path and supply the path here. When the task runs, the file at this path is run on each node; if it fails, the task fails with its exit code. Example script contents: # Plain Python: python train.py --epochs 10 # Multi-GPU via accelerate: accelerate launch train.py --config config.yaml # Distributed via torchrun: torchrun --nproc_per_node=8 train.py Example: `/Workspace/Users/alice@databricks.com/llama-fine-tune/command.sh` - `compute` (object, Public Preview) Compute resources allocated to each node in this deployment. - `name` (string, Public Preview) Optional human-readable name for this deployment (for example, `driver`, `worker`, `param_server`). Used for log and UI display. Distinct names are recommended so deployments can be told apart, but uniqueness is not enforced. Example: `worker` - `mlflow_run` (string, Public Preview) Optional display name for the MLflow run created under `experiment`. If omitted, MLflow generates a default name. Example: `llama-fine-tune-lora-r16` - `mlflow_experiment_directory` (string, Public Preview) Optional workspace directory under which the MLflow experiment named in `experiment` is created. Must start with `/Workspace`. Set this when running as a service principal that has no default user directory; for regular users the experiment defaults to the user's home directory. Example: `/Workspace/Users/alice@databricks.com/experiments` - `docker_image_url` (string, Beta) Optional Docker image URL for a custom container image. When set, the task runs on the specified container image instead of the default client image. Format: `{organization}/{repository}:{tag}` Example: `nvidia/cuda:13.3.0-cudnn-devel-ubuntu24.04` - `existing_cluster_id` (string) If existing_cluster_id, the ID of an existing cluster that is used for all runs. When running jobs or tasks on an existing cluster, you may need to manually restart the cluster if it stops responding. We suggest running jobs and tasks on new clusters for greater reliability Example: `0923-164208-meows279` - `new_cluster` (object) If new_cluster, a description of a new cluster that is created for each run. - `apply_policy_default_values` (boolean) Default: `false` - `cluster_name` (string) Cluster name requested by the user. This doesn't have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs. - `spark_version` (string) The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available Spark versions can be retrieved by using the [clusters/sparkVersions](https://docs.databricks.com/api/workspace/clusters/sparkversions) API call. - `spark_conf` (object) An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions` respectively. - `aws_attributes` (object) Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. If this value is greater than 0, the cluster driver node in particular will be placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `0` - `availability` (string) Possible values: `SPOT`, `ON_DEMAND`, `SPOT_WITH_FALLBACK` Default: `SPOT_WITH_FALLBACK` - `zone_id` (string) Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like "us-west-2a". The provided availability zone must be in the same region as the deployment. For example, "us-west-2a" is not a valid zone id if the deployment resides in the "us-east-1" region. This is an optional field at cluster creation, and if not specified, the zone "auto" will be used. If the zone specified is "auto", will try to place cluster in a zone with high availability, and will retry placement in a different AZ if there is not enough capacity. The list of available zones as well as the default value can be found by using the `List Zones` method. - `instance_profile_arn` (string) Nodes for this cluster will only be placed on AWS instances with this instance profile. If ommitted, nodes will be placed on instances without an IAM instance profile. The instance profile must have previously been added to the environment by an account administrator. This feature may only be available to certain customer plans. - `spot_bid_price_percent` (int32) The bid price for AWS spot instances, as a percentage of the corresponding instance type's on-demand price. For example, if this field is set to 50, and the cluster needs a new `r3.xlarge` spot instance, then the bid price is half of the price of on-demand `r3.xlarge` instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand `r3.xlarge` instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000. Default: `100` - `ebs_volume_type` (string) The type of EBS volumes that will be launched with this cluster. Possible values: `GENERAL_PURPOSE_SSD`, `THROUGHPUT_OPTIMIZED_HDD` - `ebs_volume_count` (int32) The number of volumes launched for each instance. Users can choose up to 10 volumes. This feature is only enabled for supported node types. Legacy node types cannot specify custom EBS volumes. For node types with no instance store, at least one EBS volume needs to be specified; otherwise, cluster creation will fail. These EBS volumes will be mounted at `/ebs0`, `/ebs1`, and etc. Instance store volumes will be mounted at `/local_disk0`, `/local_disk1`, and etc. If EBS volumes are attached, will configure Spark to use only the EBS volumes for scratch storage because heterogenously sized scratch devices can lead to inefficient disk utilization. If no EBS volumes are attached, will configure Spark to use instance store volumes. Please note that if EBS volumes are specified, then the Spark configuration `spark.local.dir` will be overridden. Default: `0` - `ebs_volume_size` (int32) The size of each EBS volume (in GiB) launched for each instance. For general purpose SSD, this value must be within the range 100 - 4096. For throughput optimized HDD, this value must be within the range 500 - 4096. - `ebs_volume_iops` (int32) If using gp3 volumes, what IOPS to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `ebs_volume_throughput` (int32) If using gp3 volumes, what throughput to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `azure_attributes` (object) Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used. - `log_analytics_info` (object) Defines values necessary to configure and run Azure Log Analytics agent - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `availability` (string) Availability type used for all subsequent nodes past the `first_on_demand` ones. Note: If `first_on_demand` is zero, this availability type will be used for the entire cluster. Possible values: `SPOT_AZURE`, `ON_DEMAND_AZURE`, `SPOT_WITH_FALLBACK_AZURE` Default: `ON_DEMAND_AZURE` - `spot_bid_max_price` (double) The max bid price to be used for Azure spot instances. The Max price for the bid cannot be higher than the on-demand price of the instance. If not specified, the default value is -1, which specifies that the instance cannot be evicted on the basis of price, and only on the basis of availability. Further, the value should > 0 or -1. Default: `-1` - `capacity_reservation_group` (string, GA) The Azure capacity reservation group resource ID to use for launching VMs. When specified, VMs will be launched using the provided capacity reservation. Capacity reservations can only be specified when the workspace uses injected vnet (i.e. customer defined vnet not managed by databricks). Ensure the databricks-login-prod Enterprise Application is granted the following four permissions: 1. Microsoft.Compute/capacityReservationGroups/read 2. Microsoft.Compute/capacityReservationGroups/deploy/action 3. Microsoft.Compute/capacityReservationGroups/capacityReservations/read 4. Microsoft.Compute/capacityReservationGroups/capacityReservations/deploy/action Format: `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/capacityReservationGroups/{capacityReservationGroupName}` - `gcp_attributes` (object) Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used. - `use_preemptible_executors` (boolean) This field determines whether the spark executors will be scheduled to run on preemptible VMs (when set to true) versus standard compute engine VMs (when set to false; default). Note: Soon to be deprecated, use the 'availability' field instead. Default: `false` - `google_service_account` (string) If provided, the cluster will impersonate the google service account when accessing gcloud services (like GCS). The google service account must have previously been added to the environment by an account administrator. - `boot_disk_size` (int32) Boot disk size in GB - `availability` (string) This field determines whether the spark executors will be scheduled to run on preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable. Possible values: `PREEMPTIBLE_GCP`, `ON_DEMAND_GCP`, `PREEMPTIBLE_WITH_FALLBACK_GCP` Default: `ON_DEMAND_GCP` - `zone_id` (string) Identifier for the availability zone in which the cluster resides. This can be one of the following: - "HA" => High availability, spread nodes across availability zones for a deployment region [default]. - "AUTO" => picks an availability zone to schedule the cluster on. - A GCP availability zone => Pick One of the available zones for (machine type + region) from https://cloud.google.com/compute/docs/regions-zones. Default: `HA` - `local_ssd_count` (int32) If provided, each node (workers and driver) in the cluster will have this number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP documentation](https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds) for the supported number of local SSDs for each instance type. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `node_type_id` (string) This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the [clusters/listNodeTypes](https://docs.databricks.com/api/workspace/clusters/listnodetypes) API call. - `driver_node_type_id` (string) The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as `node_type_id` defined above. This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence. - `worker_node_type_flexibility` (object) Flexible node type configuration for worker nodes. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `driver_node_type_flexibility` (object) Flexible node type configuration for the driver node. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `ssh_public_keys` (array of string) SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name `ubuntu` on port `2200`. Up to 10 keys can be specified. - `custom_tags` (object) Additional tags for cluster resources. will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to `default_tags`. Notes: - Currently, allows at most 45 custom tags - Clusters can only reuse cloud resources if the resources' tags are a subset of the cluster tags - `cluster_log_conf` (object) The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every `5 mins`. The destination of driver logs is `$destination/$clusterId/driver`, while the destination of executor logs is `$destination/$clusterId/executor`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs" : { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ "s3": { "destination" : "s3://cluster_log_bucket/prefix", "region" : "us-west-2" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `volumes` (object, GA) destination needs to be provided, e.g. `{ "volumes": { "destination": "/Volumes/catalog/schema/volume/cluster_log" } }` - `spark_env_vars` (object) An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., `export X='Y'`) while launching the driver and workers. In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example below. This ensures that all default databricks managed environmental variables are included as well. Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m", "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS": "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}` - `autotermination_minutes` (int32) Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination. - `enable_elastic_disk` (boolean) Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. **AWS:** Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details. - `init_scripts` (array of object) The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If `cluster_log_conf` is specified, init script logs are sent to `//init_scripts`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs": { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `file` (object) destination needs to be provided, e.g. `{ "file": { "destination": "file:/my/local/file.sh" } }` - `gcs` (object) destination needs to be provided, e.g. `{ "gcs": { "destination": "gs://my-bucket/file.sh" } }` - `abfss` (object) destination needs to be provided, e.g. `abfss://@.dfs.core.windows.net/` - `workspace` (object) destination needs to be provided, e.g. `{ "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }` - `volumes` (object) destination needs to be provided. e.g. `{ \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }` - `docker_image` (object) Custom docker image BYOC - `url` (string) URL of the docker image. - `basic_auth` (object) Basic auth with username and password - `instance_pool_id` (string) The optional ID of the instance pool to which the cluster belongs. - `single_user_name` (string) Single user name if data_security_mode is `SINGLE_USER` - `policy_id` (string) The ID of the cluster policy used to create the cluster if applicable. - `enable_local_disk_encryption` (boolean) Whether to enable LUKS on cluster VMs' local disks - `driver_instance_pool_id` (string) The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned. - `workload_type` (object) - `clients` (object) defined what type of clients can use the cluster. E.g. Notebooks, Jobs - `data_security_mode` (string) Possible values: `NONE`, `SINGLE_USER`, `USER_ISOLATION`, `LEGACY_TABLE_ACL`, `LEGACY_PASSTHROUGH`, `LEGACY_SINGLE_USER`, `LEGACY_SINGLE_USER_STANDARD`, `DATA_SECURITY_MODE_STANDARD`, `DATA_SECURITY_MODE_DEDICATED`, `DATA_SECURITY_MODE_AUTO` - `runtime_engine` (string) Determines the cluster's runtime engine, either standard or Photon. This field is not compatible with legacy `spark_version` values that contain `-photon-`. Remove `-photon-` from the `spark_version` and set `runtime_engine` to `PHOTON`. If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used. Possible values: `NULL`, `STANDARD`, `PHOTON` - `kind` (string) Possible values: `COMPUTE_KIND_UNSPECIFIED`, `CLASSIC_PREVIEW` - `use_ml_runtime` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. `effective_spark_version` is determined by `spark_version` (DBR release), this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not. - `is_single_node` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. When set to true, will automatically set single node related `custom_tags`, `spark_conf`, and `num_workers` - `remote_disk_throughput` (int32) If set, what the configurable throughput (in Mb/s) for the remote disk is. Currently only supported for GCP HYPERDISK_BALANCED disks. - `total_initial_remote_disk_size` (int32) If set, what the total initial volume size (in GB) of the remote disks should be. Supported for GCP. - `dependency_mode` (string, Beta) Controls dependency configuration for the cluster. Possible values: `DEPENDENCY_MODE_UNSPECIFIED`, `DEPENDENCY_MODE_ENVIRONMENTS`, `DEPENDENCY_MODE_CLUSTER_LIBRARIES`, `DEPENDENCY_MODE_AUTO` - `num_workers` (int32) Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes. Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned. - `autoscale` (object) Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later. - `min_workers` (int32) The minimum number of workers to which the cluster can scale down when underutilized. It is also the initial number of workers the cluster will have after creation. - `max_workers` (int32) The maximum number of workers to which the cluster can scale up when overloaded. Note that `max_workers` must be strictly greater than `min_workers`. - `job_cluster_key` (string) If job_cluster_key, this task is executed reusing the cluster specified in `job.settings.job_clusters`. Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `libraries` (array of object) An optional list of libraries to be installed on the cluster. The default value is an empty list. - `jar` (string) URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "s3://my-bucket/library.jar" }`. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI. **Azure:** URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "abfss://my-bucket/library.jar" }`. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI. **GCP:** URI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: `{ "jar": "/Workspace/path/to/library.jar" }`, `{ "jar" : "/Volumes/path/to/library.jar" }` or `{ "jar": "gs://my-bucket/library.jar" }`. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI. - `egg` (string) Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above. - `pypi` (object) Specification of a PyPi library to be installed. For example: `{ "package": "simplejson" }` - `package` (string) The name of the pypi package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0". - `repo` (string) The repository where the package can be found. If not specified, the default pip index is used. - `maven` (object) Specification of a maven library to be installed. For example: `{ "coordinates": "org.jsoup:jsoup:1.7.2" }` - `coordinates` (string) Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2". - `repo` (string) Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched. - `exclusions` (array of string) List of dependences to exclude. For example: `["slf4j:slf4j", "*:hadoop-client"]`. Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html. - `cran` (object) Specification of a CRAN library to be installed as part of the library - `package` (string) The name of the CRAN package to install. - `repo` (string) The repository where the package can be found. If not specified, the default CRAN repo is used. - `whl` (string) URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "s3://my-bucket/library.whl" }`. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI. **Azure:** URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "abfss://my-bucket/library.whl" }`. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI. **GCP:** URI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: `{ "whl": "/Workspace/path/to/library.whl" }`, `{ "whl" : "/Volumes/path/to/library.whl" }` or `{ "whl": "gs://my-bucket/library.egg" }`. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI. - `requirements` (string) URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: `{ "requirements": "/Workspace/path/to/requirements.txt" }` or `{ "requirements" : "/Volumes/path/to/requirements.txt" }` - `max_retries` (int32) An optional maximum number of times to retry an unsuccessful run. A run is considered to be unsuccessful if it completes with the `FAILED` result_state or `INTERNAL_ERROR` `life_cycle_state`. The value `-1` means to retry indefinitely and the value `0` means to never retry. Default: `0` Example: `10` - `min_retry_interval_millis` (int32) An optional minimal interval in milliseconds between the start of the failed run and the subsequent retry run. The default behavior is that unsuccessful runs are immediately retried. Example: `2000` - `retry_on_timeout` (boolean) An optional policy to specify whether to retry a job when it times out. The default behavior is to not retry on timeout. Default: `false` Example: `true` - `disable_auto_optimization` (boolean) An option to disable auto optimization in serverless Default: `false` Example: `true` - `start_time` (int64) The time at which this run was started in epoch milliseconds (milliseconds since 1/1/1970 UTC). This may not be the time when the job task starts executing, for example, if the job is scheduled to run on a new cluster, this is the time the cluster creation call is issued. Example: `1625060460483` - `setup_duration` (int64) The time in milliseconds it took to set up the cluster. For runs that run on new clusters this is the cluster creation time, for runs that run on existing clusters this time should be very short. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `setup_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `execution_duration` (int64) The time in milliseconds it took to execute the commands in the JAR or notebook until they completed, failed, timed out, were cancelled, or encountered an unexpected error. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `execution_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `cleanup_duration` (int64) The time in milliseconds it took to terminate the cluster and clean up any associated artifacts. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `cleanup_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `end_time` (int64) The time at which this run ended in epoch milliseconds (milliseconds since 1/1/1970 UTC). This field is set to 0 if the job is still running. Example: `1625060863413` - `run_duration` (int64) The time in milliseconds it took the job run and all of its repairs to finish. Example: `110183` - `queue_duration` (int64) The time in milliseconds that the run has spent in the queue. Example: `1625060863413` - `description` (string) Description of the run - `attempt_number` (int32) The sequence number of this run attempt for a triggered job run. The initial attempt of a run has an attempt_number of 0. If the initial run attempt fails, and the job has a retry policy (`max_retries` > 0), subsequent runs are created with an `original_attempt_run_id` of the original attempt’s ID and an incrementing `attempt_number`. Runs are retried only until they succeed, and the maximum `attempt_number` is the same as the `max_retries` value for the job. Example: `0` - `job_clusters` (array of object) A list of job cluster specifications that can be shared and reused by tasks of this job. Libraries cannot be declared in a shared job cluster. You must declare dependent libraries in task settings. If more than 100 job clusters are available, you can paginate through them using :method:jobs/getrun. Constraints: `<= 100 items` - `job_cluster_key` (string) A unique name for the job cluster. This field is required and must be unique within the job. `JobTaskSettings` may refer to this field to determine which cluster to launch for the task execution. Example: `auto_scaling_cluster` Constraints: `[ 1 .. 100 ] characters`, `^[\w\-\_]+$` - `new_cluster` (object) If new_cluster, a description of a cluster that is created for each task. - `apply_policy_default_values` (boolean) Default: `false` - `cluster_name` (string) Cluster name requested by the user. This doesn't have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs. - `spark_version` (string) The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available Spark versions can be retrieved by using the [clusters/sparkVersions](https://docs.databricks.com/api/workspace/clusters/sparkversions) API call. - `spark_conf` (object) An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions` respectively. - `aws_attributes` (object) Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. If this value is greater than 0, the cluster driver node in particular will be placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `0` - `availability` (string) Possible values: `SPOT`, `ON_DEMAND`, `SPOT_WITH_FALLBACK` Default: `SPOT_WITH_FALLBACK` - `zone_id` (string) Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like "us-west-2a". The provided availability zone must be in the same region as the deployment. For example, "us-west-2a" is not a valid zone id if the deployment resides in the "us-east-1" region. This is an optional field at cluster creation, and if not specified, the zone "auto" will be used. If the zone specified is "auto", will try to place cluster in a zone with high availability, and will retry placement in a different AZ if there is not enough capacity. The list of available zones as well as the default value can be found by using the `List Zones` method. - `instance_profile_arn` (string) Nodes for this cluster will only be placed on AWS instances with this instance profile. If ommitted, nodes will be placed on instances without an IAM instance profile. The instance profile must have previously been added to the environment by an account administrator. This feature may only be available to certain customer plans. - `spot_bid_price_percent` (int32) The bid price for AWS spot instances, as a percentage of the corresponding instance type's on-demand price. For example, if this field is set to 50, and the cluster needs a new `r3.xlarge` spot instance, then the bid price is half of the price of on-demand `r3.xlarge` instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand `r3.xlarge` instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000. Default: `100` - `ebs_volume_type` (string) The type of EBS volumes that will be launched with this cluster. Possible values: `GENERAL_PURPOSE_SSD`, `THROUGHPUT_OPTIMIZED_HDD` - `ebs_volume_count` (int32) The number of volumes launched for each instance. Users can choose up to 10 volumes. This feature is only enabled for supported node types. Legacy node types cannot specify custom EBS volumes. For node types with no instance store, at least one EBS volume needs to be specified; otherwise, cluster creation will fail. These EBS volumes will be mounted at `/ebs0`, `/ebs1`, and etc. Instance store volumes will be mounted at `/local_disk0`, `/local_disk1`, and etc. If EBS volumes are attached, will configure Spark to use only the EBS volumes for scratch storage because heterogenously sized scratch devices can lead to inefficient disk utilization. If no EBS volumes are attached, will configure Spark to use instance store volumes. Please note that if EBS volumes are specified, then the Spark configuration `spark.local.dir` will be overridden. Default: `0` - `ebs_volume_size` (int32) The size of each EBS volume (in GiB) launched for each instance. For general purpose SSD, this value must be within the range 100 - 4096. For throughput optimized HDD, this value must be within the range 500 - 4096. - `ebs_volume_iops` (int32) If using gp3 volumes, what IOPS to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `ebs_volume_throughput` (int32) If using gp3 volumes, what throughput to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used. - `azure_attributes` (object) Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used. - `log_analytics_info` (object) Defines values necessary to configure and run Azure Log Analytics agent - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `availability` (string) Availability type used for all subsequent nodes past the `first_on_demand` ones. Note: If `first_on_demand` is zero, this availability type will be used for the entire cluster. Possible values: `SPOT_AZURE`, `ON_DEMAND_AZURE`, `SPOT_WITH_FALLBACK_AZURE` Default: `ON_DEMAND_AZURE` - `spot_bid_max_price` (double) The max bid price to be used for Azure spot instances. The Max price for the bid cannot be higher than the on-demand price of the instance. If not specified, the default value is -1, which specifies that the instance cannot be evicted on the basis of price, and only on the basis of availability. Further, the value should > 0 or -1. Default: `-1` - `capacity_reservation_group` (string, GA) The Azure capacity reservation group resource ID to use for launching VMs. When specified, VMs will be launched using the provided capacity reservation. Capacity reservations can only be specified when the workspace uses injected vnet (i.e. customer defined vnet not managed by databricks). Ensure the databricks-login-prod Enterprise Application is granted the following four permissions: 1. Microsoft.Compute/capacityReservationGroups/read 2. Microsoft.Compute/capacityReservationGroups/deploy/action 3. Microsoft.Compute/capacityReservationGroups/capacityReservations/read 4. Microsoft.Compute/capacityReservationGroups/capacityReservations/deploy/action Format: `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/capacityReservationGroups/{capacityReservationGroupName}` - `gcp_attributes` (object) Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used. - `use_preemptible_executors` (boolean) This field determines whether the spark executors will be scheduled to run on preemptible VMs (when set to true) versus standard compute engine VMs (when set to false; default). Note: Soon to be deprecated, use the 'availability' field instead. Default: `false` - `google_service_account` (string) If provided, the cluster will impersonate the google service account when accessing gcloud services (like GCS). The google service account must have previously been added to the environment by an account administrator. - `boot_disk_size` (int32) Boot disk size in GB - `availability` (string) This field determines whether the spark executors will be scheduled to run on preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable. Possible values: `PREEMPTIBLE_GCP`, `ON_DEMAND_GCP`, `PREEMPTIBLE_WITH_FALLBACK_GCP` Default: `ON_DEMAND_GCP` - `zone_id` (string) Identifier for the availability zone in which the cluster resides. This can be one of the following: - "HA" => High availability, spread nodes across availability zones for a deployment region [default]. - "AUTO" => picks an availability zone to schedule the cluster on. - A GCP availability zone => Pick One of the available zones for (machine type + region) from https://cloud.google.com/compute/docs/regions-zones. Default: `HA` - `local_ssd_count` (int32) If provided, each node (workers and driver) in the cluster will have this number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP documentation](https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds) for the supported number of local SSDs for each instance type. - `first_on_demand` (int32) The first `first_on_demand` nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, `first_on_demand` nodes will be placed on on-demand instances and the remainder will be placed on `availability` instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster. Default: `1` - `node_type_id` (string) This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the [clusters/listNodeTypes](https://docs.databricks.com/api/workspace/clusters/listnodetypes) API call. - `driver_node_type_id` (string) The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as `node_type_id` defined above. This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence. - `worker_node_type_flexibility` (object) Flexible node type configuration for worker nodes. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `driver_node_type_flexibility` (object) Flexible node type configuration for the driver node. - `alternate_node_type_ids` (array of string) A list of node type IDs to use as fallbacks when the primary node type is unavailable. - `ssh_public_keys` (array of string) SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name `ubuntu` on port `2200`. Up to 10 keys can be specified. - `custom_tags` (object) Additional tags for cluster resources. will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to `default_tags`. Notes: - Currently, allows at most 45 custom tags - Clusters can only reuse cloud resources if the resources' tags are a subset of the cluster tags - `cluster_log_conf` (object) The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every `5 mins`. The destination of driver logs is `$destination/$clusterId/driver`, while the destination of executor logs is `$destination/$clusterId/executor`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs" : { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ "s3": { "destination" : "s3://cluster_log_bucket/prefix", "region" : "us-west-2" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `volumes` (object, GA) destination needs to be provided, e.g. `{ "volumes": { "destination": "/Volumes/catalog/schema/volume/cluster_log" } }` - `spark_env_vars` (object) An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., `export X='Y'`) while launching the driver and workers. In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example below. This ensures that all default databricks managed environmental variables are included as well. Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m", "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS": "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}` - `autotermination_minutes` (int32) Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination. - `enable_elastic_disk` (boolean) Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. **AWS:** Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details. - `init_scripts` (array of object) The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If `cluster_log_conf` is specified, init script logs are sent to `//init_scripts`. - `dbfs` (object) destination needs to be provided. e.g. `{ "dbfs": { "destination" : "dbfs:/home/cluster_log" } }` - `s3` (object) destination and either the region or endpoint need to be provided. e.g. `{ \"s3\": { \"destination\": \"s3://cluster_log_bucket/prefix\", \"region\": \"us-west-2\" } }` Cluster iam role is used to access s3, please make sure the cluster iam role in `instance_profile_arn` has permission to write data to the s3 destination. - `file` (object) destination needs to be provided, e.g. `{ "file": { "destination": "file:/my/local/file.sh" } }` - `gcs` (object) destination needs to be provided, e.g. `{ "gcs": { "destination": "gs://my-bucket/file.sh" } }` - `abfss` (object) destination needs to be provided, e.g. `abfss://@.dfs.core.windows.net/` - `workspace` (object) destination needs to be provided, e.g. `{ "workspace": { "destination": "/cluster-init-scripts/setup-datadog.sh" } }` - `volumes` (object) destination needs to be provided. e.g. `{ \"volumes\" : { \"destination\" : \"/Volumes/my-init.sh\" } }` - `docker_image` (object) Custom docker image BYOC - `url` (string) URL of the docker image. - `basic_auth` (object) Basic auth with username and password - `instance_pool_id` (string) The optional ID of the instance pool to which the cluster belongs. - `single_user_name` (string) Single user name if data_security_mode is `SINGLE_USER` - `policy_id` (string) The ID of the cluster policy used to create the cluster if applicable. - `enable_local_disk_encryption` (boolean) Whether to enable LUKS on cluster VMs' local disks - `driver_instance_pool_id` (string) The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned. - `workload_type` (object) - `clients` (object) defined what type of clients can use the cluster. E.g. Notebooks, Jobs - `data_security_mode` (string) Possible values: `NONE`, `SINGLE_USER`, `USER_ISOLATION`, `LEGACY_TABLE_ACL`, `LEGACY_PASSTHROUGH`, `LEGACY_SINGLE_USER`, `LEGACY_SINGLE_USER_STANDARD`, `DATA_SECURITY_MODE_STANDARD`, `DATA_SECURITY_MODE_DEDICATED`, `DATA_SECURITY_MODE_AUTO` - `runtime_engine` (string) Determines the cluster's runtime engine, either standard or Photon. This field is not compatible with legacy `spark_version` values that contain `-photon-`. Remove `-photon-` from the `spark_version` and set `runtime_engine` to `PHOTON`. If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used. Possible values: `NULL`, `STANDARD`, `PHOTON` - `kind` (string) Possible values: `COMPUTE_KIND_UNSPECIFIED`, `CLASSIC_PREVIEW` - `use_ml_runtime` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. `effective_spark_version` is determined by `spark_version` (DBR release), this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not. - `is_single_node` (boolean) This field can only be used when `kind = CLASSIC_PREVIEW`. When set to true, will automatically set single node related `custom_tags`, `spark_conf`, and `num_workers` - `remote_disk_throughput` (int32) If set, what the configurable throughput (in Mb/s) for the remote disk is. Currently only supported for GCP HYPERDISK_BALANCED disks. - `total_initial_remote_disk_size` (int32) If set, what the total initial volume size (in GB) of the remote disks should be. Supported for GCP. - `dependency_mode` (string, Beta) Controls dependency configuration for the cluster. Possible values: `DEPENDENCY_MODE_UNSPECIFIED`, `DEPENDENCY_MODE_ENVIRONMENTS`, `DEPENDENCY_MODE_CLUSTER_LIBRARIES`, `DEPENDENCY_MODE_AUTO` - `num_workers` (int32) Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes. Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned. - `autoscale` (object) Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later. - `min_workers` (int32) The minimum number of workers to which the cluster can scale down when underutilized. It is also the initial number of workers the cluster will have after creation. - `max_workers` (int32) The maximum number of workers to which the cluster can scale up when overloaded. Note that `max_workers` must be strictly greater than `min_workers`. - `git_source` (object) An optional specification for a remote Git repository containing the source code used by tasks. Version-controlled source code is supported by notebook, dbt, Python script, and SQL File tasks. If `git_source` is set, these tasks retrieve the file from the remote repository by default. However, this behavior can be overridden by setting `source` to `WORKSPACE` on the task. Note: dbt and SQL File tasks support only version-controlled sources. If dbt or SQL File tasks are used, `git_source` must be defined on the job. - `git_url` (string) URL of the repository to be cloned by this job. Example: `https://github.com/databricks/databricks-cli` Constraints: `<= 300 characters` - `git_provider` (string) Unique identifier of the service used to host the Git repository. The value is case insensitive. - `git_branch` (string) Name of the branch to be checked out and used by this job. This field cannot be specified in conjunction with git_tag or git_commit. Example: `main` Constraints: `<= 255 characters` - `git_tag` (string) Name of the tag to be checked out and used by this job. This field cannot be specified in conjunction with git_branch or git_commit. Example: `release-1.0.0` Constraints: `<= 255 characters` - `git_commit` (string) Commit to be checked out and used by this job. This field cannot be specified in conjunction with git_branch or git_tag. Example: `e0056d01` Constraints: `<= 64 characters` - `git_snapshot` (object) - `used_commit` (string) Commit that was used to execute the run. If git_branch was specified, this points to the HEAD of the branch at the time of the run; if git_tag was specified, this points to the commit the tag points to. Example: `4506fdf41e9fa98090570a34df7a5bce163ff15f` - `sparse_checkout` (object, GA) - `patterns` (array of string) List of patterns to include for sparse checkout. - `repair_history` (array of object) The repair history of the run. - `type` (string) The repair history item type. Indicates whether a run is the original run or a repair run. Possible values: `ORIGINAL`, `REPAIR` - `start_time` (int64) The start time of the (repaired) run. Example: `1625060460483` - `end_time` (int64) The end time of the (repaired) run. Example: `1625060863413` - `state` (object) Deprecated. Please use the `status` field instead. - `life_cycle_state` (string) A value indicating the run's current lifecycle state. This field is always available in the response. Note: Additional states might be introduced in future releases. Possible values: `PENDING`, `RUNNING`, `TERMINATING`, `TERMINATED`, `SKIPPED`, `INTERNAL_ERROR`, `BLOCKED`, `WAITING_FOR_RETRY`, `QUEUED` - `result_state` (string) A value indicating the run's result. This field is only available for terminal lifecycle states. Note: Additional states might be introduced in future releases. Possible values: `SUCCESS`, `FAILED`, `TIMEDOUT`, `CANCELED`, `MAXIMUM_CONCURRENT_RUNS_REACHED`, `UPSTREAM_CANCELED`, `UPSTREAM_FAILED`, `EXCLUDED`, `SUCCESS_WITH_FAILURES`, `DISABLED` - `state_message` (string) A descriptive message for the current state. This field is unstructured, and its exact format is subject to change. - `user_cancelled_or_timedout` (boolean) A value indicating whether a run was canceled manually by a user or by the scheduler because the run timed out. Default: `false` Example: `false` - `queue_reason` (string) The reason indicating why the run was queued. Example: `Queued due to reaching maximum concurrent runs of 1.` - `id` (int64) The ID of the repair. Only returned for the items that represent a repair in `repair_history`. Example: `734650698524280` - `task_run_ids` (array of int64) The run IDs of the task runs that ran as part of this repair history item. - `status` (object) - `state` (string) Possible values: `BLOCKED`, `PENDING`, `QUEUED`, `RUNNING`, `TERMINATING`, `TERMINATED`, `WAITING` - `termination_details` (object) If the run is in a TERMINATING or TERMINATED state, details about the reason for terminating the run. - `code` (string) Possible values: `SUCCESS`, `CANCELED`, `DRIVER_ERROR`, `CLUSTER_ERROR`, `REPOSITORY_CHECKOUT_FAILED`, `INVALID_CLUSTER_REQUEST`, `WORKSPACE_RUN_LIMIT_EXCEEDED`, `FEATURE_DISABLED`, `CLUSTER_REQUEST_LIMIT_EXCEEDED`, `STORAGE_ACCESS_ERROR`, `RUN_EXECUTION_ERROR`, `UNAUTHORIZED_ERROR`, `LIBRARY_INSTALLATION_ERROR`, `MAX_CONCURRENT_RUNS_EXCEEDED`, `MAX_SPARK_CONTEXTS_EXCEEDED`, `RESOURCE_NOT_FOUND`, `INVALID_RUN_CONFIGURATION`, `INTERNAL_ERROR`, `CLOUD_FAILURE`, `MAX_JOB_QUEUE_SIZE_EXCEEDED`, `SKIPPED`, `USER_CANCELED`, `BUDGET_POLICY_LIMIT_EXCEEDED`, `DISABLED`, `SUCCESS_WITH_FAILURES`, `BREAKING_CHANGE` - `type` (string) Possible values: `SUCCESS`, `INTERNAL_ERROR`, `CLIENT_ERROR`, `CLOUD_FAILURE` - `message` (string) A descriptive message with the termination details. This field is unstructured and the format might change. - `queue_details` (object) If the run was queued, details about the reason for queuing the run. - `code` (string) Possible values: `ACTIVE_RUNS_LIMIT_REACHED`, `MAX_CONCURRENT_RUNS_REACHED`, `ACTIVE_RUN_JOB_TASKS_LIMIT_REACHED` - `message` (string) A descriptive message with the queuing details. This field is unstructured, and its exact format is subject to change. - `effective_performance_target` (string) The actual performance target used by the serverless run during execution. This can differ from the client-set performance target on the request depending on whether the performance mode is supported by the job type. * `STANDARD`: Enables cost-efficient execution of serverless workloads. * `PERFORMANCE_OPTIMIZED`: Prioritizes fast startup and execution times through rapid scaling and optimized cluster performance. Possible values: `PERFORMANCE_TARGET_UNSPECIFIED`, `PERFORMANCE_OPTIMIZED`, `STANDARD` - `status` (object) - `state` (string) Possible values: `BLOCKED`, `PENDING`, `QUEUED`, `RUNNING`, `TERMINATING`, `TERMINATED`, `WAITING` - `termination_details` (object) If the run is in a TERMINATING or TERMINATED state, details about the reason for terminating the run. - `code` (string) Possible values: `SUCCESS`, `CANCELED`, `DRIVER_ERROR`, `CLUSTER_ERROR`, `REPOSITORY_CHECKOUT_FAILED`, `INVALID_CLUSTER_REQUEST`, `WORKSPACE_RUN_LIMIT_EXCEEDED`, `FEATURE_DISABLED`, `CLUSTER_REQUEST_LIMIT_EXCEEDED`, `STORAGE_ACCESS_ERROR`, `RUN_EXECUTION_ERROR`, `UNAUTHORIZED_ERROR`, `LIBRARY_INSTALLATION_ERROR`, `MAX_CONCURRENT_RUNS_EXCEEDED`, `MAX_SPARK_CONTEXTS_EXCEEDED`, `RESOURCE_NOT_FOUND`, `INVALID_RUN_CONFIGURATION`, `INTERNAL_ERROR`, `CLOUD_FAILURE`, `MAX_JOB_QUEUE_SIZE_EXCEEDED`, `SKIPPED`, `USER_CANCELED`, `BUDGET_POLICY_LIMIT_EXCEEDED`, `DISABLED`, `SUCCESS_WITH_FAILURES`, `BREAKING_CHANGE` - `type` (string) Possible values: `SUCCESS`, `INTERNAL_ERROR`, `CLIENT_ERROR`, `CLOUD_FAILURE` - `message` (string) A descriptive message with the termination details. This field is unstructured and the format might change. - `queue_details` (object) If the run was queued, details about the reason for queuing the run. - `code` (string) Possible values: `ACTIVE_RUNS_LIMIT_REACHED`, `MAX_CONCURRENT_RUNS_REACHED`, `ACTIVE_RUN_JOB_TASKS_LIMIT_REACHED` - `message` (string) A descriptive message with the queuing details. This field is unstructured, and its exact format is subject to change. - `job_run_id` (int64) ID of the job run that this run belongs to. For legacy and single-task job runs the field is populated with the job run ID. For task runs, the field is populated with the ID of the job run that the task run belongs to. - `has_more` (boolean) Indicates if the run has more array properties (`tasks`, `job_clusters`) that are not shown. They can be accessed via :method:jobs/getrun endpoint. It is only relevant for API 2.2 :method:jobs/listruns requests with `expand_tasks=true`. Example: `true` - `effective_performance_target` (string) The actual performance target used by the serverless run during execution. This can differ from the client-set performance target on the request depending on whether the performance mode is supported by the job type. * `STANDARD`: Enables cost-efficient execution of serverless workloads. * `PERFORMANCE_OPTIMIZED`: Prioritizes fast startup and execution times through rapid scaling and optimized cluster performance. Possible values: `PERFORMANCE_TARGET_UNSPECIFIED`, `PERFORMANCE_OPTIMIZED`, `STANDARD` - `start_time` (int64) The time at which this run was started in epoch milliseconds (milliseconds since 1/1/1970 UTC). This may not be the time when the job task starts executing, for example, if the job is scheduled to run on a new cluster, this is the time the cluster creation call is issued. Example: `1625060460483` - `setup_duration` (int64) The time in milliseconds it took to set up the cluster. For runs that run on new clusters this is the cluster creation time, for runs that run on existing clusters this time should be very short. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `setup_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `execution_duration` (int64) The time in milliseconds it took to execute the commands in the JAR or notebook until they completed, failed, timed out, were cancelled, or encountered an unexpected error. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `execution_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `cleanup_duration` (int64) The time in milliseconds it took to terminate the cluster and clean up any associated artifacts. The duration of a task run is the sum of the `setup_duration`, `execution_duration`, and the `cleanup_duration`. The `cleanup_duration` field is set to 0 for multitask job runs. The total duration of a multitask job run is the value of the `run_duration` field. Example: `0` - `end_time` (int64) The time at which this run ended in epoch milliseconds (milliseconds since 1/1/1970 UTC). This field is set to 0 if the job is still running. Example: `1625060863413` - `run_duration` (int64) The time in milliseconds it took the job run and all of its repairs to finish. Example: `110183` - `queue_duration` (int64) The time in milliseconds that the run has spent in the queue. Example: `1625060863413` ## Example ```json { "job_id": 11223344, "run_id": 455644833, "creator_user_name": "user.name@databricks.com", "number_in_job": 455644833, "original_attempt_run_id": 455644833, "state": { "life_cycle_state": "string", "result_state": "string", "state_message": "string", "user_cancelled_or_timedout": false, "queue_reason": "Queued due to reaching maximum concurrent runs of 1." }, "schedule": { "quartz_cron_expression": "20 30 * * * ?", "timezone_id": "Europe/London", "pause_status": "string" }, "cluster_spec": { "existing_cluster_id": "0923-164208-meows279", "new_cluster": {}, "job_cluster_key": "string", "libraries": [ {} ] }, "cluster_instance": { "cluster_id": "0923-164208-meows279", "spark_context_id": "string" }, "job_parameters": [ { "name": "table", "default": "users", "value": "customers" } ], "overriding_parameters": { "pipeline_params": {} }, "trigger": "string", "trigger_info": { "run_id": 0 }, "run_name": "A multitask job run", "run_page_url": "https://my-workspace.cloud.databricks.com/#job/11223344/run/123", "run_type": "string", "tasks": [ { "attempt_number": 0, "cleanup_duration": 0, "cluster_instance": { "cluster_id": "0923-164208-meows279", "spark_context_id": "4348585301701786933" }, "description": "Ingests order data", "end_time": 1629989930171, "execution_duration": 0, "job_cluster_key": "auto_scaling_cluster", "libraries": [ { "jar": "dbfs:/mnt/databricks/OrderIngest.jar" } ], "run_id": 2112892, "run_if": "ALL_SUCCESS", "run_page_url": "https://my-workspace.cloud.databricks.com/#job/39832/run/20", "setup_duration": 0, "spark_jar_task": { "main_class_name": "com.databricks.OrdersIngest" }, "start_time": 1629989929660, "state": { "life_cycle_state": "INTERNAL_ERROR", "result_state": "FAILED", "state_message": "Library installation failed for library due to user error. Error messages:\n'Manage' permissions are required to install libraries on a cluster", "user_cancelled_or_timedout": false }, "task_key": "Orders_Ingest" }, { "attempt_number": 0, "cleanup_duration": 0, "cluster_instance": { "cluster_id": "0923-164208-meows279" }, "depends_on": [ { "task_key": "Orders_Ingest" }, { "task_key": "Sessionize" } ], "description": "Matches orders with user sessions", "end_time": 1629989930238, "execution_duration": 0, "new_cluster": { "autoscale": { "max_workers": 16, "min_workers": 2 }, "node_type_id": null, "spark_conf": { "spark.speculation": true }, "spark_version": "7.3.x-scala2.12" }, "notebook_task": { "notebook_path": "/Users/user.name@databricks.com/Match", "source": "WORKSPACE" }, "run_id": 2112897, "run_if": "ALL_SUCCESS", "run_page_url": "https://my-workspace.cloud.databricks.com/#job/39832/run/21", "setup_duration": 0, "start_time": 0, "state": { "life_cycle_state": "SKIPPED", "state_message": "An upstream task failed.", "user_cancelled_or_timedout": false }, "task_key": "Match" }, { "attempt_number": 0, "cleanup_duration": 0, "cluster_instance": { "cluster_id": "0923-164208-meows279", "spark_context_id": "4348585301701786933" }, "description": "Extracts session data from events", "end_time": 1629989930144, "execution_duration": 0, "existing_cluster_id": "0923-164208-meows279", "libraries": [ { "jar": "dbfs:/mnt/databricks/Sessionize.jar" } ], "run_id": 2112902, "run_if": "ALL_SUCCESS", "run_page_url": "https://my-workspace.cloud.databricks.com/#job/39832/run/22", "setup_duration": 0, "spark_jar_task": { "main_class_name": "com.databricks.Sessionize" }, "start_time": 1629989929668, "state": { "life_cycle_state": "INTERNAL_ERROR", "result_state": "FAILED", "state_message": "Library installation failed for library due to user error. Error messages:\n'Manage' permissions are required to install libraries on a cluster", "user_cancelled_or_timedout": false }, "task_key": "Sessionize" } ], "description": "string", "attempt_number": 0, "job_clusters": [ { "job_cluster_key": "auto_scaling_cluster", "new_cluster": { "autoscale": { "max_workers": 16, "min_workers": 2 }, "node_type_id": null, "spark_conf": { "spark.speculation": true }, "spark_version": "7.3.x-scala2.12" } } ], "git_source": { "git_url": "https://github.com/databricks/databricks-cli", "git_provider": "string", "git_branch": "main", "git_tag": "release-1.0.0", "git_commit": "e0056d01", "git_snapshot": {}, "sparse_checkout": {} }, "repair_history": [ { "type": "string", "start_time": 1625060460483, "end_time": 1625060863413, "state": {}, "id": 734650698524280, "task_run_ids": [ 1106460542112844, 988297789683452 ], "status": {}, "effective_performance_target": "string" } ], "status": { "state": "string", "termination_details": {}, "queue_details": {} }, "job_run_id": 0, "has_more": true, "effective_performance_target": "string", "start_time": 1625060460483, "setup_duration": 0, "execution_duration": 0, "cleanup_duration": 0, "end_time": 1625060863413, "run_duration": 110183, "queue_duration": 1625060863413 } ```