Microsoft Entra ID sign-in logs

Preview

Lakewatch is in Private Preview. To request access, reach out to your Databricks account team.

Overview

Microsoft Entra ID sign-in logs record authentication activity in your tenant. Lakewatch ingests all five sign-in diagnostic categories from cloud storage using Auto Loader and schematizes them with the built-in azure_signin parser:

SignInLogs: Interactive user sign-ins.
NonInteractiveUserSignInLogs: Sign-ins performed by a client on behalf of a user, such as token refreshes.
ServicePrincipalSignInLogs: Sign-ins by applications and service principals.
ManagedIdentitySignInLogs: Sign-ins by managed identities.
MicrosoftServicePrincipalSignInLogs: Sign-ins by Microsoft first-party service principals.

Service principal and managed identity sign-ins carry no user principal name. Use the appId, servicePrincipalId, and userId columns, which are extracted as actor_id observables, to correlate them. This is a cloud storage source, not an API connector.

To ingest Microsoft Entra ID sign-in logs, create a cloud-storage datasource for them and attach the built-in azure_signin parser so events are written to a schematized parsed table. For prerequisites, Unity Catalog access, and the full datasource setup steps, see Cloud storage.

The parser claims only events whose category is SignInLogs, NonInteractiveUserSignInLogs, ServicePrincipalSignInLogs, ManagedIdentitySignInLogs or MicrosoftServicePrincipalSignInLogs. If one storage location mixes several Azure log categories, attach every matching Azure parser to the same datasource, with the generic azure_monitor_activity parser last. See Ingest several Azure log categories from one location.

Parsed schema and observables

The built-in azure_signin parser schematizes Microsoft Entra ID sign-in logs into a parsed table. Every parsed table also carries the Lakewatch-managed columns:

lw_id: A non-null string Lakewatch uses as the primary key, supporting deduplication across incremental reads.
time: A non-null event timestamp assigned by the parser.
lw_meta: Standard Lakewatch processing metadata.
lw_observables: Values extracted from fields marked as security observables in the parser.

The full event payload is kept in the properties column as a VARIANT, so fields that the parser doesn't promote to their own columns remain queryable. For more information, see the Parser reference.

Parsed table schema

The azure_signin parser writes the following columns (generated from the parser definition):

Column	Data type
`time`	`TIMESTAMP`
`resourceId`	`STRING`
`operationName`	`STRING`
`operationVersion`	`STRING`
`category`	`STRING`
`tenantId`	`STRING`
`resultType`	`STRING`
`resultSignature`	`STRING`
`resultDescription`	`STRING`
`durationMs`	`DOUBLE`
`callerIpAddress`	`STRING`
`correlationId`	`STRING`
`identity`	`STRING`
`level`	`STRING`
`location`	`STRING`
`signInId`	`STRING`
`userPrincipalName`	`STRING`
`userDisplayName`	`STRING`
`userId`	`STRING`
`appId`	`STRING`
`appDisplayName`	`STRING`
`servicePrincipalId`	`STRING`
`ipAddress`	`STRING`
`clientAppUsed`	`STRING`
`userAgent`	`STRING`
`locationCity`	`STRING`
`locationState`	`STRING`
`locationCountryOrRegion`	`STRING`
`locationLatitude`	`DOUBLE`
`locationLongitude`	`DOUBLE`
`deviceId`	`STRING`
`deviceOperatingSystem`	`STRING`
`deviceBrowser`	`STRING`
`deviceIsCompliant`	`BOOLEAN`
`deviceIsManaged`	`BOOLEAN`
`statusErrorCode`	`BIGINT`
`conditionalAccessStatus`	`STRING`
`isInteractive`	`BOOLEAN`
`riskLevelDuringSignIn`	`STRING`
`riskLevelAggregated`	`STRING`
`riskState`	`STRING`
`riskDetail`	`STRING`
`properties`	`VARIANT`

Column	Data type
`time`	`TIMESTAMP`
`resourceId`	`STRING`
`operationName`	`STRING`
`operationVersion`	`STRING`
`category`	`STRING`
`tenantId`	`STRING`
`resultType`	`STRING`
`resultSignature`	`STRING`
`resultDescription`	`STRING`
`durationMs`	`DOUBLE`
`callerIpAddress`	`STRING`
`correlationId`	`STRING`
`identity`	`STRING`
`level`	`STRING`
`location`	`STRING`
`signInId`	`STRING`
`userPrincipalName`	`STRING`
`userDisplayName`	`STRING`
`userId`	`STRING`
`appId`	`STRING`
`appDisplayName`	`STRING`
`servicePrincipalId`	`STRING`
`ipAddress`	`STRING`
`clientAppUsed`	`STRING`
`userAgent`	`STRING`
`locationCity`	`STRING`
`locationState`	`STRING`
`locationCountryOrRegion`	`STRING`
`locationLatitude`	`DOUBLE`
`locationLongitude`	`DOUBLE`
`deviceId`	`STRING`
`deviceOperatingSystem`	`STRING`
`deviceBrowser`	`STRING`
`deviceIsCompliant`	`BOOLEAN`
`deviceIsManaged`	`BOOLEAN`
`statusErrorCode`	`BIGINT`
`conditionalAccessStatus`	`STRING`
`isInteractive`	`BOOLEAN`
`riskLevelDuringSignIn`	`STRING`
`riskLevelAggregated`	`STRING`
`riskState`	`STRING`
`riskDetail`	`STRING`
`properties`	`VARIANT`

Parser observables

The parser extracts the following fields into lw_observables (generated from the parser definition):

Field	Observable
`callerIpAddress`	`ip`
`correlationId`	`trace_id`
`userPrincipalName`	`username`
`userPrincipalName`	`email`
`userDisplayName`	`username`
`userId`	`actor_id`
`appId`	`actor_id`
`servicePrincipalId`	`actor_id`
`ipAddress`	`ip`

The following VARIANT fields are scanned for observables by matching nested key names; the observable types extracted from each:

Field	Observable types scanned
`properties`	`ip`, `username`, `actor_id`, `trace_id`, `url`

Overview​

Onboard Microsoft Entra ID sign-in logs​

Parsed schema and observables​

Overview

Onboard Microsoft Entra ID sign-in logs

Parsed schema and observables