Microsoft Graph activity logs

Preview

Lakewatch is in Private Preview. To request access, reach out to your Databricks account team.

Overview

Microsoft Graph activity logs record each HTTP request made to Microsoft Graph in your tenant, with the request method and URI, the response status, and the calling application or user. Lakewatch ingests the MicrosoftGraphActivityLogs diagnostic category from cloud storage using Auto Loader and schematizes it with the built-in azure_graph_activity parser. This is a cloud storage source, not an API connector.

Onboard Microsoft Graph activity logs

To ingest Microsoft Graph activity logs, create a cloud-storage datasource for them and attach the built-in azure_graph_activity parser so events are written to a schematized parsed table. For prerequisites, Unity Catalog access, and the full datasource setup steps, see Cloud storage.

The parser claims only events whose category is MicrosoftGraphActivityLogs. If one storage location mixes several Azure log categories, attach every matching Azure parser to the same datasource, with the generic azure_monitor_activity parser last. See Ingest several Azure log categories from one location.

Parsed schema and observables

The built-in azure_graph_activity parser schematizes Microsoft Graph activity logs into a parsed table. Every parsed table also carries the Lakewatch-managed columns:

• lw_id: A non-null string Lakewatch uses as the primary key, supporting deduplication across incremental reads.
• time: A non-null event timestamp assigned by the parser.
• lw_meta: Standard Lakewatch processing metadata.
• lw_observables: Values extracted from fields marked as security observables in the parser.

The full event payload is kept in the properties column as a VARIANT, so fields that the parser doesn't promote to their own columns remain queryable. For more information, see the Parser reference.

Parsed table schema

The azure_graph_activity parser writes the following columns (generated from the parser definition):

Column	Data type
time	TIMESTAMP
resourceId	STRING
operationName	STRING
operationVersion	STRING
category	STRING
tenantId	STRING
resultSignature	STRING
durationMs	STRING
callerIpAddress	STRING
correlationId	STRING
level	STRING
location	STRING
requestMethod	STRING
requestUri	STRING
responseStatusCode	BIGINT
responseSizeBytes	BIGINT
apiVersion	STRING
roles	STRING
scopes	STRING
signInActivityId	STRING
requestId	STRING
appId	STRING
servicePrincipalId	STRING
userId	STRING
userPrincipalObjectId	STRING
identityProvider	STRING
ipAddress	STRING
userAgent	STRING
properties	VARIANT

Column	Data type
time	TIMESTAMP
resourceId	STRING
operationName	STRING
operationVersion	STRING
category	STRING
tenantId	STRING
resultSignature	STRING
durationMs	STRING
callerIpAddress	STRING
correlationId	STRING
level	STRING
location	STRING
requestMethod	STRING
requestUri	STRING
responseStatusCode	BIGINT
responseSizeBytes	BIGINT
apiVersion	STRING
roles	STRING
scopes	STRING
signInActivityId	STRING
requestId	STRING
appId	STRING
servicePrincipalId	STRING
userId	STRING
userPrincipalObjectId	STRING
identityProvider	STRING
ipAddress	STRING
userAgent	STRING
properties	VARIANT

Parser observables

The parser extracts the following fields into lw_observables (generated from the parser definition):

Field	Observable
callerIpAddress	ip
correlationId	trace_id
requestUri	url
signInActivityId	trace_id
requestId	trace_id
appId	actor_id
servicePrincipalId	actor_id
userId	actor_id
userPrincipalObjectId	actor_id
ipAddress	ip

Field	Observable
callerIpAddress	ip
correlationId	trace_id
requestUri	url
signInActivityId	trace_id
requestId	trace_id
appId	actor_id
servicePrincipalId	actor_id
userId	actor_id
userPrincipalObjectId	actor_id
ipAddress	ip

The following VARIANT fields are scanned for observables by matching nested key names; the observable types extracted from each:

Field	Observable types scanned
properties	trace_id

Field	Observable types scanned
properties	trace_id