Akamai WAF logs

Beta

Lakewatch is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage workspace-level previews.

Lakewatch provides an API connector to ingest WAF logs from Akamai. This article covers prerequisites, configuration instructions, connector schemas, and security observables.

Prerequisites

The connector requires:

1. Serverless Lakeflow pipelines turned on in your workspace.
2. Unity Catalog, with privileges to create or use a Unity Catalog connection. See Connection permissions.
3. The Lakeflow Connect for Akamai WAF preview turned on in your workspace by a workspace admin. See Turn on the connector preview.
4. If your workspace's serverless network policy uses restricted access, the source's API domains must be on the policy's Allowed domains list. See Allow the source API domains in your network policy.
5. An Akamai Control Center administrator must enable SIEM Integration and provision the SIEM user.
6. The user associated with the API credentials must have the Manage SIEM role for every group containing a security configuration you want to ingest. See Akamai SIEM Integration.
7. SIEM Integration and data collection must be turned on for each security configuration, and the configuration must be active on the production network. See Get started with the SIEM Integration API.

Known limitations

• SCD type 2 is not supported, since Akamai WAF logs are append-only.
• The connector authenticates using long-lived EdgeGrid credentials that default to 2 years. After they expire, create new credentials and update the connection.
• Historical lookback can't exceed the 12 hours Akamai retains WAF logs. See Choose how far back to ingest.

Configure Akamai for ingestion

Configure Akamai to enable authentication from Databricks.

Fetch Web Security Configuration IDs

1. Sign in to Akamai Control Center as an Admin.
2. Click the menu icon in the top left of the page to open the navigation pane.
3. In the navigation pane, click Your Services > Web & Data Center Security > Security Configurations > Web Security.
4. Review the list of security configurations. Each configuration has a dropdown. Note the Contract and Group ID under the dropdown. You need these values when you create a user with the MANAGE_SIEM role.
5. For each security configuration that you want to ingest logs for:
   1. Click the version under Last Created, Staging, or Production.
   2. Go to Advanced Settings > Logging > Data Collection For SIEM integration > Web Security Configuration ID.
   3. Note the Web Security Configuration ID. Keep these IDs for both the MANAGE_SIEM permission setup and the datasource's Security Configuration ID(s) field.

Create a user with the MANAGE_SIEM role

1. Sign in to Akamai Control Center as an Admin.
2. Click the menu icon in the top left of the page to open the navigation pane.
3. In the navigation pane, click Your Services > Account Admin > Identity & Access.
4. Click Create User.
5. Enter the Email and Verify email values.
6. Under Assign roles, select MANAGE_SIEM for each of the groups used to define the Web Security Configuration IDs from the previous step.

Create an API client

1. Sign in to Akamai Control Center as the user you created in the previous step.
2. Click the menu icon in the top left of the page to open the navigation pane.
3. In the navigation pane, click Your Services > Account Admin > Identity & Access.
4. Click Create API Client > Advanced.
5. Under APIs, select Select APIs.
6. Click Reset API selection.
7. Search for the SIEM API.
8. Under Access level, select READ-ONLY, then click Submit.
9. Enter a unique Name for the API client, then click Create API Client.
10. Copy the client_secret, host, access_token, and client_token values. Akamai shows these credentials only one time. Use them to create a Unity Catalog connection in Databricks.

tip

When creating a connection through the Lakewatch UI, you need the Host, Client Token, Client Secret, and Access Token gathered above.

Next steps

After completing the configuration above, create a datasource with the Akamai WAF connector in the Lakewatch UI:

1. Create or select a Unity Catalog connection using the Host, Client Token, Client Secret, and Access Token.
2. In the datasource form's required Security Configuration ID(s) field, enter the Web Security Configuration IDs as comma-separated integers, for example, 12345, 67890. These IDs select which configurations to ingest and are separate from the connection credentials.

See API connectors for the remaining datasource settings.

---

Connector and Lakewatch schemas

The connector ingests the following source table from the default source schema:

Source table	Akamai data	Built-in Lakewatch parser
akamai_waf_events	WAF security events from the Akamai SIEM Integration API	akamai_waf

Source table	Akamai data	Built-in Lakewatch parser
akamai_waf_events	WAF security events from the Akamai SIEM Integration API	akamai_waf

The connector writes each source table directly to the destination configured for its built-in parser. By default, the destination table is named <parser>_<datasource>, where <datasource> is the datasource name. For example, with the default datasource name akamai_waf, the table is akamai_waf_akamai_waf. You can set a different destination during datasource creation. There is no separate raw staging table or later parsing step.

Lakewatch-managed columns

The connector adds lw_id, a non-null string that Lakewatch uses for stable deduplication and time ordering. It derives lw_id from the event timestamp and _databricks_primary_key. The connector derives:

• _databricks_primary_key from a stable hash of the source event.
• time from httpMessage.start, interpreted as Unix epoch seconds. The connector uses time as the cursor field for incremental ingestion.

For Lakewatch datasources, the connector also adds:

• lw_meta: The datasource and connection identity, parser name and version, and ingestion time.
• lw_observables: Security observables extracted by the connector.

For more information, see Connector output tables.

Connector output table schema

The destination includes the source and event columns below, together with the lw_meta and lw_observables columns described above.

The following schema combines the standard Lakeflow Connect fields with the Lakewatch-only lw_id column. For field definitions, see the Akamai SIEM Integration API.

Schema

Column	Data type
lw_id	STRING
_databricks_primary_key	LONG
time	TIMESTAMP
format	STRING
type	STRING
version	STRING
attackData	STRUCT<appliedAction: STRING, clientIP: STRING, configId: STRING, policyId: STRING, ruleActions: STRING, ruleData: STRING, ruleMessages: STRING, ruleSelectors: STRING, ruleTags: STRING, ruleVersions: STRING, rules: STRING, apiId: STRING, apiKey: STRING, clientReputation: STRING, slowPostAction: STRING, slowPostRate: STRING, decodedRules: ARRAY<STRUCT<rule: STRING, ruleAction: STRING, ruleData: STRING, ruleMessage: STRING, ruleSelector: STRING, ruleTag: STRING, ruleVersion: STRING>>>
botData	STRUCT<botScore: STRING, responseSegment: STRING>
clientData	STRUCT<telemetryType: STRING, appBundleId: STRING, appVersion: STRING, sdkVersion: STRING>
custom	STRING
geo	STRUCT<asn: STRING, city: STRING, continent: STRING, country: STRING, regionCode: STRING>
httpMessage	STRUCT<bytes: STRING, host: STRING, method: STRING, path: STRING, port: STRING, protocol: STRING, query: STRING, requestHeaders: STRING, requestId: STRING, responseHeaders: STRING, start: STRING, status: STRING, tls: STRING>
identity	STRUCT<ja4: STRING, tlsFingerprintV2: STRING, tlsFingerprintV3: STRING>
userRiskData	STRUCT<allow: STRING, emailDomain: STRING, general: STRING, originUserId: STRING, risk: STRING, score: STRING, status: STRING, trust: STRING, username: STRING, uuid: STRING>

Column	Data type
lw_id	STRING
_databricks_primary_key	LONG
time	TIMESTAMP
format	STRING
type	STRING
version	STRING
attackData	STRUCT<appliedAction: STRING, clientIP: STRING, configId: STRING, policyId: STRING, ruleActions: STRING, ruleData: STRING, ruleMessages: STRING, ruleSelectors: STRING, ruleTags: STRING, ruleVersions: STRING, rules: STRING, apiId: STRING, apiKey: STRING, clientReputation: STRING, slowPostAction: STRING, slowPostRate: STRING, decodedRules: ARRAY<STRUCT<rule: STRING, ruleAction: STRING, ruleData: STRING, ruleMessage: STRING, ruleSelector: STRING, ruleTag: STRING, ruleVersion: STRING>>>
botData	STRUCT<botScore: STRING, responseSegment: STRING>
clientData	STRUCT<telemetryType: STRING, appBundleId: STRING, appVersion: STRING, sdkVersion: STRING>
custom	STRING
geo	STRUCT<asn: STRING, city: STRING, continent: STRING, country: STRING, regionCode: STRING>
httpMessage	STRUCT<bytes: STRING, host: STRING, method: STRING, path: STRING, port: STRING, protocol: STRING, query: STRING, requestHeaders: STRING, requestId: STRING, responseHeaders: STRING, start: STRING, status: STRING, tls: STRING>
identity	STRUCT<ja4: STRING, tlsFingerprintV2: STRING, tlsFingerprintV3: STRING>
userRiskData	STRUCT<allow: STRING, emailDomain: STRING, general: STRING, originUserId: STRING, risk: STRING, score: STRING, status: STRING, trust: STRING, username: STRING, uuid: STRING>

Security observables

The connector extracts the following fields into lw_observables.

Observables

Field	Observable
attackData.clientIP	ip
httpMessage.host	hostname
httpMessage.requestId	trace_id
userRiskData.emailDomain	domain
userRiskData.originUserId	actor_id
userRiskData.username	username
userRiskData.uuid	actor_id

Field	Observable
attackData.clientIP	ip
httpMessage.host	hostname
httpMessage.requestId	trace_id
userRiskData.emailDomain	domain
userRiskData.originUserId	actor_id
userRiskData.username	username
userRiskData.uuid	actor_id