Statement Execution

View as Markdown

Manage Databricks SQL assets, including warehouses, dashboards, queries and query history, and alerts

Cancel StatementPOST /api/2.0/sql/statements/{statement_id}/cancelExecute StatementPOST /api/2.0/sql/statementsGet Result DataGET /api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index}Get Statement ResultGET /api/2.0/sql/statements/{statement_id}

Cancel Statement GA

POST /api/2.0/sql/statements/{statement_id}/cancel

Requests that an executing statement be canceled. Callers must poll for status to see the terminal state. Cancel response is empty; receiving response indicates successful receipt.

API scopes: sql

Parameters

statement_idstringpath

The statement ID is returned upon successfully submitting a SQL statement, and is a required reference for all subsequent calls.

View as Markdown

Response

{}

Execute Statement GA

POST /api/2.0/sql/statements

Execute a SQL statement and optionally await its results for a specified time.

Use case: small result sets with INLINE + JSON_ARRAY

For flows that generate small and predictable result sets (<= 25 MiB), INLINE responses of JSON_ARRAY result data are typically the simplest way to execute and fetch result data.

Use case: large result sets with EXTERNAL_LINKS

Using EXTERNAL_LINKS to fetch result data allows you to fetch large result sets efficiently. The main differences from using INLINE disposition are that the result data is accessed with URLs, and that there are 3 supported formats: JSON_ARRAY, ARROW_STREAM and CSV compared to only JSON_ARRAY with INLINE.

** URLs**

External links point to data stored within your workspace's internal storage, in the form of a URL. The URLs are valid for only a short period, <= 15 minutes. Alongside each external_link is an expiration field indicating the time at which the URL is no longer valid. In EXTERNAL_LINKS mode, chunks can be resolved and fetched multiple times and in parallel.

---

Warning: Databricks strongly recommends that you protect the URLs that are returned by the EXTERNAL_LINKS disposition.

When you use the EXTERNAL_LINKS disposition, a short-lived, URL is generated, which can be used to download the results directly from . As a short-lived is embedded in this URL, you should protect the URL.

Because URLs are already generated with embedded temporary s, you must not set an Authorization header in the download requests.

The EXTERNAL_LINKS disposition can be disabled upon request by creating a support case.

See also Security best practices.

---

StatementResponse contains statement_id and status; other fields might be absent or present depending on context. If the SQL warehouse fails to execute the provided statement, a 200 response is returned with status.state set to FAILED (in contrast to a failure when accepting the request, which results in a non-200 response). Details of the error can be found at status.error in case of execution failures.

API scopes: sql

AWS

Execute a SQL statement and optionally await its results for a specified time.

Use case: small result sets with INLINE + JSON_ARRAY

For flows that generate small and predictable result sets (<= 25 MiB), INLINE responses of JSON_ARRAY result data are typically the simplest way to execute and fetch result data.

Use case: large result sets with EXTERNAL_LINKS

Using EXTERNAL_LINKS to fetch result data allows you to fetch large result sets efficiently. The main differences from using INLINE disposition are that the result data is accessed with presigned URLs, and that there are 3 supported formats: JSON_ARRAY, ARROW_STREAM and CSV compared to only JSON_ARRAY with INLINE.

Presigned URLs

External links point to data stored within your workspace's internal storage, in the form of a presigned URL. The URLs are valid for only a short period, <= 15 minutes. Alongside each external_link is an expiration field indicating the time at which the URL is no longer valid. In EXTERNAL_LINKS mode, chunks can be resolved and fetched multiple times and in parallel.

---

Warning: Databricks strongly recommends that you protect the URLs that are returned by the EXTERNAL_LINKS disposition.

When you use the EXTERNAL_LINKS disposition, a short-lived, presigned URL is generated, which can be used to download the results directly from Amazon S3. As a short-lived access credential is embedded in this presigned URL, you should protect the URL.

Because presigned URLs are already generated with embedded temporary access credentials, you must not set an Authorization header in the download requests.

The EXTERNAL_LINKS disposition can be disabled upon request by creating a support case. See Support.

See also Security best practices.

---

StatementResponse contains statement_id and status; other fields might be absent or present depending on context. If the SQL warehouse fails to execute the provided statement, a 200 response is returned with status.state set to FAILED (in contrast to a failure when accepting the request, which results in a non-200 response). Details of the error can be found at status.error in case of execution failures.

Azure

Execute a SQL statement and optionally await its results for a specified time.

Use case: small result sets with INLINE + JSON_ARRAY

For flows that generate small and predictable result sets (<= 25 MiB), INLINE responses of JSON_ARRAY result data are typically the simplest way to execute and fetch result data.

Use case: large result sets with EXTERNAL_LINKS

Using EXTERNAL_LINKS to fetch result data allows you to fetch large result sets efficiently. The main differences from using INLINE disposition are that the result data is accessed with shared access signature (SAS) URLs, and that there are 3 supported formats: JSON_ARRAY, ARROW_STREAM and CSV compared to only JSON_ARRAY with INLINE.

Shared Access Signature URLs

External links point to data stored within your workspace's internal storage, in the form of a SAS URL. The URLs are valid for only a short period, <= 15 minutes. Alongside each external_link is an expiration field indicating the time at which the URL is no longer valid. In EXTERNAL_LINKS mode, chunks can be resolved and fetched multiple times and in parallel.

---

Warning: Databricks strongly recommends that you protect the URLs that are returned by the EXTERNAL_LINKS disposition.

When you use the EXTERNAL_LINKS disposition, a short-lived, SAS URL is generated, which can be used to download the results directly from Azure storage. As a short-lived SAS token is embedded in this SAS URL, you should protect the URL.

Because SAS URLs are already generated with embedded temporary SAS tokens, you must not set an Authorization header in the download requests.

The EXTERNAL_LINKS disposition can be disabled upon request by creating a support case.

See also Security best practices.

---

StatementResponse contains statement_id and status; other fields might be absent or present depending on context. If the SQL warehouse fails to execute the provided statement, a 200 response is returned with status.state set to FAILED (in contrast to a failure when accepting the request, which results in a non-200 response). Details of the error can be found at status.error in case of execution failures.

GCP

Execute a SQL statement and optionally await its results for a specified time.

Use case: small result sets with INLINE + JSON_ARRAY

For flows that generate small and predictable result sets (<= 25 MiB), INLINE responses of JSON_ARRAY result data are typically the simplest way to execute and fetch result data.

Use case: large result sets with EXTERNAL_LINKS

Using EXTERNAL_LINKS to fetch result data allows you to fetch large result sets efficiently. The main differences from using INLINE disposition are that the result data is accessed with signed URLs, and that there are 3 supported formats: JSON_ARRAY, ARROW_STREAM and CSV compared to only JSON_ARRAY with INLINE.

Signed URLs

External links point to data stored within your workspace's internal storage, in the form of a signed URL. The URLs are valid for only a short period, <= 15 minutes. Alongside each external_link is an expiration field indicating the time at which the URL is no longer valid. In EXTERNAL_LINKS mode, chunks can be resolved and fetched multiple times and in parallel.

---

Warning: Databricks strongly recommends that you protect the URLs that are returned by the EXTERNAL_LINKS disposition.

When you use the EXTERNAL_LINKS disposition, a short-lived, signed URL is generated, which can be used to download the results directly from Google Cloud Storage. As a short-lived access credential is embedded in this signed URL, you should protect the URL.

Because signed URLs are already generated with embedded temporary access credentials, you must not set an Authorization header in the download requests.

The EXTERNAL_LINKS disposition can be disabled upon request by creating a support case. See Support.

See also Security best practices.

---

StatementResponse contains statement_id and status; other fields might be absent or present depending on context. If the SQL warehouse fails to execute the provided statement, a 200 response is returned with status.state set to FAILED (in contrast to a failure when accepting the request, which results in a non-200 response). Details of the error can be found at status.error in case of execution failures.

Request body

statementstring

The SQL statement to execute. The statement can optionally be parameterized, see parameters. The maximum query text size is 16 MiB.

Example: SELECT * FROM range(10)

warehouse_idstring

Warehouse upon which to execute a statement. See also What are SQL warehouses?

catalogstring

Sets default catalog for statement execution, similar to USE CATALOG in SQL.

schemastring

Sets default schema for statement execution, similar to USE SCHEMA in SQL.

row_limitint64

Applies the given row limit to the statement's result set, but unlike the LIMIT clause in SQL, it also sets the truncated field in the response to indicate whether the result was trimmed due to the limit or not.

byte_limitint64

Applies the given byte limit to the statement's result size. Byte counts are based on internal data representations and might not match the final size in the requested format. If the result was truncated due to the byte limit, then truncated in the response is set to true. When using EXTERNAL_LINKS disposition, a default byte_limit of 100 GiB is applied if byte_limit is not explicitly set.

formatstring

Statement execution supports three result formats: JSON_ARRAY (default), ARROW_STREAM, and CSV.

Important: The formats ARROW_STREAM and CSV are supported only with EXTERNAL_LINKS disposition. JSON_ARRAY is supported in INLINE and EXTERNAL_LINKS disposition.

When specifying format=JSON_ARRAY, result data will be formatted as an array of arrays of values, where each value is either the string representation of a value, or null. For example, the output of SELECT concat('id-', id) AS strCol, id AS intCol, null AS nullCol FROM range(3) would look like this:

 [
 [ "id-1", "1", null ],
 [ "id-2", "2", null ],
 [ "id-3", "3", null ],
 ]

When specifying format=JSON_ARRAY and disposition=EXTERNAL_LINKS, each chunk in the result contains compact JSON with no indentation or extra whitespace.

When specifying format=ARROW_STREAM and disposition=EXTERNAL_LINKS, each chunk in the result will be formatted as Apache Arrow Stream. See the Apache Arrow streaming format.

When specifying format=CSV and disposition=EXTERNAL_LINKS, each chunk in the result will be a CSV according to RFC 4180 standard. All the columns values will have string representation similar to the JSON_ARRAY format, and null values will be encoded as “null”. Only the first chunk in the result would contain a header row with column names. For example, the output of SELECT concat('id-', id) AS strCol, id AS intCol, null as nullCol FROM range(3) would look like this:

strCol,intCol,nullCol
id-1,1,null
id-2,2,null
id-3,3,null

Default: JSON_ARRAY

Values: FORMAT_UNSPECIFIED, JSON_ARRAY, ARROW_STREAM, CSV

dispositionstring

The fetch disposition provides two modes of fetching results: INLINE and EXTERNAL_LINKS.

Statements executed with INLINE disposition will return result data inline, in JSON_ARRAY format, in a series of chunks. If a given statement produces a result set with a size larger than 25 MiB, that statement execution is aborted, and no result set will be available.

NOTE Byte limits are computed based upon internal representations of the result set data, and might not match the sizes visible in JSON responses.

Statements executed with EXTERNAL_LINKS disposition will return result data as external links: URLs that point to cloud storage internal to the workspace. Using EXTERNAL_LINKS disposition allows statements to generate arbitrarily sized result sets for fetching up to 100 GiB. The resulting links have two important properties:

1. They point to resources external to the <Databricks> compute; therefore any associated authentication information (typically a personal access token, OAuth token, or similar) must be removed when fetching from these links.

2. These are URLs with a specific expiration, indicated in the response. The behavior when attempting to use an expired link is cloud specific.

AWS

The fetch disposition provides two modes of fetching results: INLINE and EXTERNAL_LINKS.

Statements executed with INLINE disposition will return result data inline, in JSON_ARRAY format, in a series of chunks. If a given statement produces a result set with a size larger than 25 MiB, that statement execution is aborted, and no result set will be available.

NOTE Byte limits are computed based upon internal representations of the result set data, and might not match the sizes visible in JSON responses.

Statements executed with EXTERNAL_LINKS disposition will return result data as external links: URLs that point to cloud storage internal to the workspace. Using EXTERNAL_LINKS disposition allows statements to generate arbitrarily sized result sets for fetching up to 100 GiB. The resulting links have two important properties:

1. They point to resources external to the <Databricks> compute; therefore any associated authentication information (typically a personal access token, OAuth token, or similar) must be removed when fetching from these links.

2. These are presigned URLs with a specific expiration, indicated in the response. The behavior when attempting to use an expired link is cloud specific.

Azure

The fetch disposition provides two modes of fetching results: INLINE and EXTERNAL_LINKS.

Statements executed with INLINE disposition will return result data inline, in JSON_ARRAY format, in a series of chunks. If a given statement produces a result set with a size larger than 25 MiB, that statement execution is aborted, and no result set will be available.

NOTE Byte limits are computed based upon internal representations of the result set data, and might not match the sizes visible in JSON responses.

Statements executed with EXTERNAL_LINKS disposition will return result data as external links: URLs that point to cloud storage internal to the workspace. Using EXTERNAL_LINKS disposition allows statements to generate arbitrarily sized result sets for fetching up to 100 GiB. The resulting links have two important properties:

1. They point to resources external to the <Databricks> compute; therefore any associated authentication information (typically a personal access token, OAuth token, or similar) must be removed when fetching from these links.

2. These are SAS URLs with a specific expiration, indicated in the response. The behavior when attempting to use an expired link is cloud specific.

GCP

The fetch disposition provides two modes of fetching results: INLINE and EXTERNAL_LINKS.

Statements executed with INLINE disposition will return result data inline, in JSON_ARRAY format, in a series of chunks. If a given statement produces a result set with a size larger than 25 MiB, that statement execution is aborted, and no result set will be available.

NOTE Byte limits are computed based upon internal representations of the result set data, and might not match the sizes visible in JSON responses.

Statements executed with EXTERNAL_LINKS disposition will return result data as external links: URLs that point to cloud storage internal to the workspace. Using EXTERNAL_LINKS disposition allows statements to generate arbitrarily sized result sets for fetching up to 100 GiB. The resulting links have two important properties:

1. They point to resources external to the <Databricks> compute; therefore any associated authentication information (typically a personal access token, OAuth token, or similar) must be removed when fetching from these links.

2. These are signed URLs with a specific expiration, indicated in the response. The behavior when attempting to use an expired link is cloud specific.

Default: INLINE

Values: FETCH_DISPOSITION_UNSPECIFIED, INLINE, EXTERNAL_LINKS

wait_timeoutstring

The time in seconds the call will wait for the statement's result set as Ns, where N can be set to 0 or to a value between 5 and 50.

When set to 0s, the statement will execute in asynchronous mode and the call will not wait for the execution to finish. In this case, the call returns directly with PENDING state and a statement ID which can be used for polling with statementexecution/getStatement.

When set between 5 and 50 seconds, the call will behave synchronously up to this timeout and wait for the statement execution to finish. If the execution finishes within this time, the call returns immediately with a manifest and result data (or a FAILED state in case of an execution error). If the statement takes longer to execute, on_wait_timeout determines what should happen after the timeout is reached.

Default: 10s

on_wait_timeoutstring

When wait_timeout > 0s, the call will block up to the specified time. If the statement execution doesn't finish within this time, on_wait_timeout determines whether the execution should continue or be canceled. When set to CONTINUE, the statement execution continues asynchronously and the call returns a statement ID which can be used for polling with statementexecution/getStatement. When set to CANCEL, the statement execution is canceled and the call returns with a CANCELED state.

Default: CONTINUE

Values: TIMEOUT_ACTION_UNSPECIFIED, CONTINUE, CANCEL

parametersarray of object

A list of parameters to pass into a SQL statement containing parameter markers. A parameter consists of a name, a value, and optionally a type. To represent a NULL value, the value field may be omitted or set to null explicitly. If the type field is omitted, the value is interpreted as a string.

If the type is given, parameters will be checked for type correctness according to the given type. A value is correct if the provided string can be converted to the requested type using the cast function. The exact semantics are described in the section cast function of the SQL language reference.

For example, the following statement contains two parameters, my_name and my_date:

    SELECT * FROM my_table WHERE name = :my_name AND date = :my_date

The parameters can be passed in the request body as follows:

{ ..., "statement": "SELECT * FROM my_table WHERE name = :my_name AND date = :my_date", "parameters": [ { "name": "my_name", "value": "the name" }, { "name": "my_date", "value": "2020-01-01", "type": "DATE" } ] }

Currently, positional parameters denoted by a ? marker are not supported by the Databricks SQL Statement Execution API.

Also see the section Parameter markers of the SQL language reference.

Show child attributesHide child attributes

namestring

The name of a parameter marker to be substituted in the statement.

valuestring

The value to substitute, represented as a string. If omitted, the value is interpreted as NULL.

typestring

The data type, given as a string. For example: INT, STRING, DECIMAL(10,2). If no type is given the type is assumed to be STRING. Complex types, such as ARRAY, MAP, and STRUCT are not supported. For valid types, refer to the section Data types of the SQL language reference.

query_tagsarray of objectPublic Preview

An array of query tags to annotate a SQL statement. A query tag consists of a non-empty key and, optionally, a value. To represent a NULL value, either omit the value field or manually set it to null or white space. Refer to the SQL language reference for the format specification of query tags. There's no significance to the order of tags. Only one value per key will be recorded. A sequence in excess of 20 query tags will be coerced to 20. Example:

 {
   ...,
   "query_tags": [
     { "key": "team", "value": "eng" },
     { "key": "some key only tag" }
   ]
 }

Show child attributesHide child attributes

keystringPublic Preview

valuestringPublic Preview

Response

statement_idstring

The statement ID is returned upon successfully submitting a SQL statement, and is a required reference for all subsequent calls.

statusobject

Show child attributesHide child attributes

statestring

Statement execution state:

• PENDING: waiting for warehouse
• RUNNING: running
• SUCCEEDED: execution was successful, result data available for fetch
• FAILED: execution failed; reason for failure described in accompanying error message
• CANCELED: user canceled; can come from explicit cancel call, or timeout with on_wait_timeout=CANCEL
• CLOSED: execution successful, and statement closed; result no longer available for fetch

Values: STATE_UNSPECIFIED, PENDING, RUNNING, SUCCEEDED, FAILED, CANCELED, CLOSED

errorobject

Show child attributesHide child attributes

error_codestring

Values: UNKNOWN, INTERNAL_ERROR, TEMPORARILY_UNAVAILABLE, IO_ERROR, BAD_REQUEST, SERVICE_UNDER_MAINTENANCE, WORKSPACE_TEMPORARILY_UNAVAILABLE, DEADLINE_EXCEEDED, CANCELLED, RESOURCE_EXHAUSTED, ABORTED, NOT_FOUND, ALREADY_EXISTS, UNAUTHENTICATED

messagestring

A brief summary of the error condition.

sql_statestring

SQLSTATE error code returned when the statement execution fails. Only populated when the statement status is FAILED.

manifestobject

Show child attributesHide child attributes

formatstring

Values: FORMAT_UNSPECIFIED, JSON_ARRAY, ARROW_STREAM, CSV

schemaobject

Show child attributesHide child attributes

column_countint32

columnsarray of object

Show child attributesHide child attributes

namestring

The name of the column.

type_textstring

The full SQL type specification.

type_namestring

The name of the base data type. This doesn't include details for complex types such as STRUCT, MAP or ARRAY.

Values: BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, STRING, BINARY, DECIMAL, INTERVAL, ARRAY, STRUCT, MAP, CHAR, NULL, USER_DEFINED_TYPE

positionint32

The ordinal position of the column (starting at position 0).

type_precisionint32

Specifies the number of digits in a number. This applies to the DECIMAL type.

type_scaleint32

Specifies the number of digits to the right of the decimal point in a number. This applies to the DECIMAL type.

type_interval_typestring

The format of the interval type.

total_chunk_countint32

The total number of chunks that the result set has been divided into.

chunksarray of object

Array of result set chunk metadata.

Show child attributesHide child attributes

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

total_row_countint64

The total number of rows in the result set.

total_byte_countint64

The total number of bytes in the result set. This field is not available when using INLINE disposition.

truncatedboolean

Indicates whether the result is truncated due to row_limit or byte_limit.

resultobject

Show child attributesHide child attributes

external_linksarray of object

Show child attributesHide child attributes

external_linkstring

A URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

AWS

A presigned URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

Azure

A SAS URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

GCP

A signed URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

expirationstring

Indicates the date-time that the given external link will expire and becomes invalid, after which point a new external_link must be requested.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

data_arrayarray of array of object

The JSON_ARRAY format is an array of arrays of values, where each non-null value is formatted as a string. Null values are encoded as JSON null.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

View as Markdown

Request

Asynchronous executionDefault (wait up to 10s, fallback to async)EXTERNAL_LINKS with ARROW_STREAMEXTERNAL_LINKS with CSVEXTERNAL_LINKS with JSON_ARRAYNamed parametersSynchronous executionWait up to 30s, fallback to async

{
  "statement": "SELECT * FROM range(100)",
  "wait_timeout": "0s",
  "warehouse_id": "abcdef0123456789"
}

Response

Statement failed with syntax errrorStatement runningStatement succeeded with EXTERNAL_LINKS and CSVStatement succeeeded with INLINE data

{
  "statement_id": "01ee48eb-5124-1922-bb90-f98c82f024fe",
  "status": {
    "error": {
      "error_code": "BAD_REQUEST",
      "message": "[PARSE_SYNTAX_ERROR] Syntax error at or near ..."
    },
    "state": "FAILED"
  }
}

Get Result Data GA

GET /api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index}

After the statement execution has SUCCEEDED, this request can be used to fetch any chunk by index. Whereas the first chunk with chunk_index=0 is typically fetched with statementexecution/executeStatement or statementexecution/getStatement, this request can be used to fetch subsequent chunks. The response structure is identical to the nested result element described in the statementexecution/getStatement request, and similarly includes the next_chunk_index and next_chunk_internal_link fields for simple iteration through the result set. Depending on disposition, the response returns chunks of data either inline, or as links.

API scopes: sql

Parameters

statement_idstringpath

The statement ID is returned upon successfully submitting a SQL statement, and is a required reference for all subsequent calls.

chunk_indexint32path

Response

external_linksarray of object

Show child attributesHide child attributes

external_linkstring

A URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

AWS

A presigned URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

Azure

A SAS URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

GCP

A signed URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

expirationstring

Indicates the date-time that the given external link will expire and becomes invalid, after which point a new external_link must be requested.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

data_arrayarray of array of object

The JSON_ARRAY format is an array of arrays of values, where each non-null value is formatted as a string. Null values are encoded as JSON null.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

View as Markdown

Response

EXTERNAL_LINKS, has next chunkEXTERNAL_LINKS, no next chunkINLINE, JSON_ARRAY, has next chunkINLINE, JSON_ARRAY, no next chunk

{
  "external_links": [
    {
      "byte_count": 24486486,
      "chunk_index": 0,
      "expiration": "2023-01-30T22:23:23.140Z",
      "external_link": "https://someplace.cloud-provider.com/very/long/path/...",
      "next_chunk_index": 1,
      "next_chunk_internal_link": "/api/2.0/sql/statements/01ee4b3f-4f56-1648-825a-c261d31be5f1/result/chunks/1",
      "row_count": 100,
      "row_offset": 0
    }
  ]
}

Get Statement Result GA

GET /api/2.0/sql/statements/{statement_id}

This request can be used to poll for the statement's status. StatementResponse contains statement_id and status; other fields might be absent or present depending on context. When the status.state field is SUCCEEDED it will also return the result manifest and the first chunk of the result data. When the statement is in the terminal states CANCELED, CLOSED or FAILED, it returns HTTP 200 with the state set. After at least 12 hours in terminal state, the statement is removed from the warehouse and further calls will receive an HTTP 404 response.

NOTE This call currently might take up to 5 seconds to get the latest status and result.

API scopes: sql

Parameters

statement_idstringpath

The statement ID is returned upon successfully submitting a SQL statement, and is a required reference for all subsequent calls.

Response

statement_idstring

The statement ID is returned upon successfully submitting a SQL statement, and is a required reference for all subsequent calls.

statusobject

Show child attributesHide child attributes

statestring

Statement execution state:

• PENDING: waiting for warehouse
• RUNNING: running
• SUCCEEDED: execution was successful, result data available for fetch
• FAILED: execution failed; reason for failure described in accompanying error message
• CANCELED: user canceled; can come from explicit cancel call, or timeout with on_wait_timeout=CANCEL
• CLOSED: execution successful, and statement closed; result no longer available for fetch

Values: STATE_UNSPECIFIED, PENDING, RUNNING, SUCCEEDED, FAILED, CANCELED, CLOSED

errorobject

Show child attributesHide child attributes

error_codestring

Values: UNKNOWN, INTERNAL_ERROR, TEMPORARILY_UNAVAILABLE, IO_ERROR, BAD_REQUEST, SERVICE_UNDER_MAINTENANCE, WORKSPACE_TEMPORARILY_UNAVAILABLE, DEADLINE_EXCEEDED, CANCELLED, RESOURCE_EXHAUSTED, ABORTED, NOT_FOUND, ALREADY_EXISTS, UNAUTHENTICATED

messagestring

A brief summary of the error condition.

sql_statestring

SQLSTATE error code returned when the statement execution fails. Only populated when the statement status is FAILED.

manifestobject

Show child attributesHide child attributes

formatstring

Values: FORMAT_UNSPECIFIED, JSON_ARRAY, ARROW_STREAM, CSV

schemaobject

Show child attributesHide child attributes

column_countint32

columnsarray of object

Show child attributesHide child attributes

namestring

The name of the column.

type_textstring

The full SQL type specification.

type_namestring

The name of the base data type. This doesn't include details for complex types such as STRUCT, MAP or ARRAY.

Values: BOOLEAN, BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, DATE, TIMESTAMP, STRING, BINARY, DECIMAL, INTERVAL, ARRAY, STRUCT, MAP, CHAR, NULL, USER_DEFINED_TYPE

positionint32

The ordinal position of the column (starting at position 0).

type_precisionint32

Specifies the number of digits in a number. This applies to the DECIMAL type.

type_scaleint32

Specifies the number of digits to the right of the decimal point in a number. This applies to the DECIMAL type.

type_interval_typestring

The format of the interval type.

total_chunk_countint32

The total number of chunks that the result set has been divided into.

chunksarray of object

Array of result set chunk metadata.

Show child attributesHide child attributes

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

total_row_countint64

The total number of rows in the result set.

total_byte_countint64

The total number of bytes in the result set. This field is not available when using INLINE disposition.

truncatedboolean

Indicates whether the result is truncated due to row_limit or byte_limit.

resultobject

Show child attributesHide child attributes

external_linksarray of object

Show child attributesHide child attributes

external_linkstring

A URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

AWS

A presigned URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

Azure

A SAS URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

GCP

A signed URL pointing to a chunk of result data, hosted by an external service, with a short expiration time (<= 15 minutes). As this URL contains a temporary credential, it should be considered sensitive and the client should not expose this URL in a log.

expirationstring

Indicates the date-time that the given external link will expire and becomes invalid, after which point a new external_link must be requested.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

data_arrayarray of array of object

The JSON_ARRAY format is an array of arrays of values, where each non-null value is formatted as a string. Null values are encoded as JSON null.

chunk_indexint32

The position within the sequence of result set chunks.

row_offsetint64

The starting row offset within the result set.

row_countint64

The number of rows within the result chunk.

byte_countint64

The number of bytes in the result chunk. This field is not available when using INLINE disposition.

next_chunk_indexint32

When fetching, provides the chunk_index for the next chunk. If absent, indicates there are no more chunks. The next chunk can be fetched with a statementexecution/getstatementresultchunkn request.

next_chunk_internal_linkstring

When fetching, provides a link to fetch the next chunk. If absent, indicates there are no more chunks. This link is an absolute path to be joined with your $DATABRICKS_HOST, and should be treated as an opaque link. This is an alternative to using next_chunk_index.

View as Markdown

Response

Statement failed with syntax errrorStatement runningStatement succeeded with EXTERNAL_LINKS and CSVStatement succeeeded with INLINE data

{
  "statement_id": "01ee48eb-5124-1922-bb90-f98c82f024fe",
  "status": {
    "error": {
      "error_code": "BAD_REQUEST",
      "message": "[PARSE_SYNTAX_ERROR] Syntax error at or near ..."
    },
    "state": "FAILED"
  }
}