Refresh

View as Markdown

The Refresh object gives information on a refresh of the data quality monitoring pipeline.

GetGET /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}ListGET /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshesCreatePOST /api/data-quality/v1/monitors/{refresh.object_type}/{refresh.object_id}/refreshesUpdatePATCH /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}DeleteDELETE /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}

Refresh object

The Refresh object gives information on a refresh of the data quality monitoring pipeline.

object_typestringPublic Preview

The type of the monitored object. Can be one of the following: schema or table.

object_idstringPublic Preview

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64Public Preview

Unique id of the refresh operation.

statestringPublic Preview

The current state of the refresh.

Values: MONITOR_REFRESH_STATE_UNKNOWN, MONITOR_REFRESH_STATE_PENDING, MONITOR_REFRESH_STATE_RUNNING, MONITOR_REFRESH_STATE_SUCCESS, MONITOR_REFRESH_STATE_FAILED, MONITOR_REFRESH_STATE_CANCELED

messagestringPublic Preview

An optional message to give insight into the current state of the refresh (e.g. FAILURE messages).

start_time_msint64Public Preview

Time when the refresh started (milliseconds since 1/1/1970 UTC).

end_time_msint64Public Preview

Time when the refresh ended (milliseconds since 1/1/1970 UTC).

triggerstringPublic Preview

What triggered the refresh.

Values: MONITOR_REFRESH_TRIGGER_UNKNOWN, MONITOR_REFRESH_TRIGGER_MANUAL, MONITOR_REFRESH_TRIGGER_SCHEDULE, MONITOR_REFRESH_TRIGGER_DATA_CHANGE

View as Markdown

The Refresh object

{
  "object_type": "string",
  "object_id": "string",
  "refresh_id": 0,
  "state": "string",
  "message": "string",
  "start_time_ms": 0,
  "end_time_ms": 0,
  "trigger": "string"
}

Get Public Preview

GET /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}

Get data quality monitor refresh. The call must be made in the same workspace as where the monitor was created.

For the table object_type, the caller must have either of the following sets of permissions:

1. MANAGE and USE_CATALOG on the table's parent catalog.
2. USE_CATALOG on the table's parent catalog, and MANAGE and USE_SCHEMA on the table's parent schema.
3. USE_CATALOG on the table's parent catalog, USE_SCHEMA on the table's parent schema, and SELECT on the table.

For the schema object_type, the caller must have either of the following sets of permissions:

1. MANAGE and USE_CATALOG on the schema's parent catalog.
2. USE_CATALOG on the schema's parent catalog, and USE_SCHEMA on the schema.

API scopes: dataquality

Parameters

object_typestringpath

The type of the monitored object. Can be one of the following: schema or table.

object_idstringpath

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64path

Unique id of the refresh operation.

Response

Returns the Refresh object.

View as Markdown

Response

{
  "object_type": "string",
  "object_id": "string",
  "refresh_id": 0,
  "state": "string",
  "message": "string",
  "start_time_ms": 0,
  "end_time_ms": 0,
  "trigger": "string"
}

List Public Preview

GET /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes

List data quality monitor refreshes. The call must be made in the same workspace as where the monitor was created.

For the table object_type, the caller must have either of the following sets of permissions:

1. MANAGE and USE_CATALOG on the table's parent catalog.
2. USE_CATALOG on the table's parent catalog, and MANAGE and USE_SCHEMA on the table's parent schema.
3. USE_CATALOG on the table's parent catalog, USE_SCHEMA on the table's parent schema, and SELECT on the table.

For the schema object_type, the caller must have either of the following sets of permissions:

1. MANAGE and USE_CATALOG on the schema's parent catalog.
2. USE_CATALOG on the schema's parent catalog, and USE_SCHEMA on the schema.

API scopes: dataquality

Parameters

object_typestringpath

The type of the monitored object. Can be one of the following: schema or table.

object_idstringpath

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

page_tokenstringquery

page_sizeint32query

Default: 25

Constraints: <= 100

Response

Returns a list of Refresh objects.

View as Markdown

Response

{
  "refreshes": [
    {
      "object_type": "string",
      "object_id": "string",
      "refresh_id": 0,
      "state": "string",
      "message": "string",
      "start_time_ms": 0,
      "end_time_ms": 0,
      "trigger": "string"
    }
  ],
  "next_page_token": "string"
}

Create Public Preview

POST /api/data-quality/v1/monitors/{refresh.object_type}/{refresh.object_id}/refreshes

Creates a refresh. Currently only supported for the table object_type. The call must be made in the same workspace as where the monitor was created.

The caller must have either of the following sets of permissions:

1. MANAGE and USE_CATALOG on the table's parent catalog.
2. USE_CATALOG on the table's parent catalog, and MANAGE and USE_SCHEMA on the table's parent schema.
3. USE_CATALOG on the table's parent catalog, USE_SCHEMA on the table's parent schema, and MANAGE on the table.

API scopes: dataquality

Parameters

refreshobjectpath

The refresh to create

Show child attributesHide child attributes

object_typestring

The type of the monitored object. Can be one of the following: schema or table.

object_idstring

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64

Unique id of the refresh operation.

statestring

The current state of the refresh.

Values: MONITOR_REFRESH_STATE_UNKNOWN, MONITOR_REFRESH_STATE_PENDING, MONITOR_REFRESH_STATE_RUNNING, MONITOR_REFRESH_STATE_SUCCESS, MONITOR_REFRESH_STATE_FAILED, MONITOR_REFRESH_STATE_CANCELED

messagestring

An optional message to give insight into the current state of the refresh (e.g. FAILURE messages).

start_time_msint64

Time when the refresh started (milliseconds since 1/1/1970 UTC).

end_time_msint64

Time when the refresh ended (milliseconds since 1/1/1970 UTC).

triggerstring

What triggered the refresh.

Values: MONITOR_REFRESH_TRIGGER_UNKNOWN, MONITOR_REFRESH_TRIGGER_MANUAL, MONITOR_REFRESH_TRIGGER_SCHEDULE, MONITOR_REFRESH_TRIGGER_DATA_CHANGE

Response

Returns the Refresh object.

View as Markdown

Response

{
  "object_type": "string",
  "object_id": "string",
  "refresh_id": 0,
  "state": "string",
  "message": "string",
  "start_time_ms": 0,
  "end_time_ms": 0,
  "trigger": "string"
}

Update Public Preview

PATCH /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}

(Unimplemented) Update a refresh

API scopes: dataquality

Parameters

object_typestringpath

The type of the monitored object. Can be one of the following: schema or table.

object_idstringpath

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64path

Unique id of the refresh operation.

update_maskstringquery

The field mask to specify which fields to update.

Request body

refreshobject

The refresh to update.

Show child attributesHide child attributes

object_typestring

The type of the monitored object. Can be one of the following: schema or table.

object_idstring

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64

Unique id of the refresh operation.

statestring

The current state of the refresh.

Values: MONITOR_REFRESH_STATE_UNKNOWN, MONITOR_REFRESH_STATE_PENDING, MONITOR_REFRESH_STATE_RUNNING, MONITOR_REFRESH_STATE_SUCCESS, MONITOR_REFRESH_STATE_FAILED, MONITOR_REFRESH_STATE_CANCELED

messagestring

An optional message to give insight into the current state of the refresh (e.g. FAILURE messages).

start_time_msint64

Time when the refresh started (milliseconds since 1/1/1970 UTC).

end_time_msint64

Time when the refresh ended (milliseconds since 1/1/1970 UTC).

triggerstring

What triggered the refresh.

Values: MONITOR_REFRESH_TRIGGER_UNKNOWN, MONITOR_REFRESH_TRIGGER_MANUAL, MONITOR_REFRESH_TRIGGER_SCHEDULE, MONITOR_REFRESH_TRIGGER_DATA_CHANGE

Response

Returns the Refresh object.

View as Markdown

Request

{
  "refresh": {
    "object_type": "string",
    "object_id": "string",
    "refresh_id": 0,
    "state": "string",
    "message": "string",
    "start_time_ms": 0,
    "end_time_ms": 0,
    "trigger": "string"
  }
}

Response

{
  "object_type": "string",
  "object_id": "string",
  "refresh_id": 0,
  "state": "string",
  "message": "string",
  "start_time_ms": 0,
  "end_time_ms": 0,
  "trigger": "string"
}

Delete Public Preview

DELETE /api/data-quality/v1/monitors/{object_type}/{object_id}/refreshes/{refresh_id}

(Unimplemented) Delete a refresh

API scopes: dataquality

Parameters

object_typestringpath

The type of the monitored object. Can be one of the following: schema or table.

object_idstringpath

The UUID of the request object. It is schema_id for schema, and table_id for table.

Find the schema_id from either:

1. The schema_id of the Schemas resource.
2. In Catalog Explorer > select the schema > go to the Details tab > the Schema ID field.

Find the table_id from either:

1. The table_id of the Tables resource.
2. In Catalog Explorer > select the table > go to the Details tab > the Table ID field.

refresh_idint64path

Unique id of the refresh operation.

View as Markdown

Response

{}