Pass context about job runs into job tasks
You can use dynamic value references to pass context about a job or task run such as the job or task name, the identifier of a run, or the start time of a job run. Dynamic value references are templated variables that are replaced with the appropriate values when the job task runs.
When a job runs, the task parameter variable surrounded by double curly braces is replaced and appended to an optional string value included as part of the value. For example, to pass a parameter named MyJobId
with a value of my-job-6
for any run of job ID 6, add the following task parameter:
{
"MyJobID": "my-job-{{job.id}}"
}
The contents of the double curly braces are not evaluated as expressions, so you cannot do operations or functions in double-curly braces.
User-provided identifiers, for example, task names, task value keys, or job parameter names containing special characters must be escaped by surrounding the identifiers with backticks ( ` `
). Only alphanumeric and underscore characters can be used without escaping.
{
"VariableWithSpecialChars": "{{job.parameters.`param$@`}}"
}
Syntax errors in references (for example, a missing brace) are ignored and the value is treated as a literal string. For example, {{my.value}
is passed as the string "{{my.value}"
. However, entering an invalid reference that belongs to a known namespace (for example, {{job.naem}}
) is not allowed. An error message is displayed if an invalid reference belonging to a known namespace is entered in the UI.
After a task completes, you can see resolved values for parameters under Parameters on the run details page.
Supported value references
The following dynamic value references are supported:
Reference |
Description |
---|---|
|
The unique identifier assigned to the job. |
|
The name of the job at the time of the job run. |
|
The unique identifier assigned to the job run. |
|
The number of repair attempts on the current job run. |
|
A value based on the time (in UTC timezone) that the job run started. The return value is based on the |
|
The value of the job-level parameter with the key |
|
The trigger type of the job run. The possible values are |
|
If a file arrival trigger is configured for this job, the value of the storage location. |
|
A value based on the time (in UTC timezone) that the job run was triggered, rounded down to the closest minute for jobs with a cron schedule. The return value is based on the |
|
The name of the current task. |
|
The unique identifier of the current task run. |
|
The number of times the current task was run (including retries and repairs). |
|
The notebook path of the current notebook task. |
|
The unique identifier assigned to the task run for |
|
The result state of task |
|
The error code for task |
|
The number of times the task |
|
The path to the notebook for the notebook task |
|
The task value with the key |
|
The unique identifier assigned to the workspace. |
|
The URL of the workspace. |
You can set these references with any task when you Create a job, Edit a job, or Run a job with different parameters.
You can also pass parameters between tasks in a job with task values. See Share information between tasks in a Databricks job.
Options for date and time values
Use the following arguments to specify the return value from time based parameter variables. All return values are based on a timestamp in UTC timezone.
Argument |
Description |
---|---|
|
Returns a digit from 1 to 7, representing the day of the week of the timestamp. |
|
Returns |
|
Returns the date in ISO format. |
|
Returns the date and time in ISO format. |
|
Returns the year part of the timestamp. |
|
Returns the month part of the timestamp. |
|
Returns the day part of the timestamp. |
|
Returns the hour part of the timestamp. |
|
Returns the minute part of the timestamp. |
|
Returns the second part of the timestamp. |
|
Returns the timestamp in milliseconds. |
Deprecated parameter variables
The following parameter variables are deprecated. Although they are still supported, any new jobs or updates to existing jobs should use the [supported value references>(#supported-references). The recommended replacement reference is included in the description of each variable.
Variable |
Description |
---|---|
|
The unique identifier assigned to a job. Use |
|
The unique identifier assigned to a task run. Use |
|
The date a task run started. The format is yyyy-MM-dd in UTC timezone. Use |
|
The timestamp of the run’s start of execution after the cluster is created and ready. The format is milliseconds since UNIX epoch in UTC timezone, as returned by |
|
The number of retries that have been attempted to run a task if the first attempt fails. The value is 0 for the first attempt and increments with each retry. Use |
|
The unique identifier assigned to the run of a job with multiple tasks. Use |
|
The unique name assigned to a task that’s part of a job with multiple tasks. Use |