How do I analyze AWS WAF logs in CloudWatch?

Resolution

To analyze and filter specific log requests in CloudWatch, use CloudWatch Logs Insights or the CloudWatch query generator.

Use CloudWatch Log Insights to analyze AWS WAF access logs

You can use CloudWatch Log Insights from within the CloudWatch console or in the Log Insights tab in AWS WAF.

From AWS WAF

Complete the following steps:

1. Open the AWS WAF console.
2. In the navigation pane, choose AWS WAF, and then choose Web ACLs.
3. For Region, select the AWS Region that contains your web access control list (web ACL).
   Note: If your web ACL is set up for Amazon CloudFront, then select Global.
4. Select your web ACL.
5. In the navigation pane, choose Log Insights.
6. Select the log groups that you want to query.
7. In the Query editor, enter your query. Use query syntax to design your queries. You can also select queries from the Most frequently used queries list.
8. Choose Run query.
9. To view your results, in the navigation pane, choose Logs.

From CloudWatch

Complete the following steps:

1. Open the CloudWatch console.
2. In the navigation pane, choose Logs, and then choose Log Insights.
3. For Select log group(s), select one or more log groups to query from the dropdown list. Or, choose Browse log groups, and then select your query.
4. (Optional) Choose a time range for the period that you want to query.
5. Use query syntax to design your queries.
6. To view your results, choose Run query.

You can use the following example queries to filter out specific information with CloudWatch Logs insights.

Top client IPs

To count the top client IPs that access your logs, run the following query:

fields httpRequest.clientIp
| stats count(*) as requestCount by httpRequest.clientIp
| sort requestCount desc

Top countries

To count the top countries that access your logs, run the following query:

stats count(*) as RequestCount by httpRequest.country as Country
| sort RequestCount desc

Top hosts

To count the top hosts that access your logs, run the following query:

parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
|stats count(*) as RequestCount by Host
| sort RequestCount desc

Top methods

To count the top methods that access your logs, run the following query:

stats count(*)as RequestCount by httpRequest.httpMethod as Method
| sort RequestCount desc

Top user agents

To count the top user agents that access your logs, run the following query:

parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/
| stats count(*) as RequestCount by UserAgent
| sort RequestCount desc

Top terminating rules

To count the top terminating rules in your logs, run the following query:

stats count(*) as RequestCount by terminatingRuleId
| sort RequestCount desc

Filter blocked requests

To filter for all blocked requests, and their terminating rule, URI path, and client IP, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter action = "BLOCK"
| sort @timestamp desc

Filter by host

To filter your logs by a specific host, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI
| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| filter Host = "www.example.com"

Note: Replace www.example.com with the name of your host.

Filter on a specific string

To filter your logs by a specific string, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method,httpRequest.uri as URI
| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}])/
| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}])/
| filter @message like "{jndi:ldap"
| sort action, URI desc

Note: Replace {jndi:ldap. with your string.

Filter on POST requests

To filter for POST requests, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method, httpRequest.uri as URI
| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/
| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| filter httpRequest.httpMethod ="POST"
| display Rule, action, Country, ClientIP, Method, URI, Host, UserAgent
| sort Rule, action desc

Filter by country

To filter out requests that don't originate from a specific country, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI
| parse @message /\{"name":"[Hh]ost\",\"value":\"(?[^"}]*)/
| parse @message /\{"name":"[Uu]ser\-[Aa]gent\",\"value\"\:\"(?[^"}]*)/
| filter Country != "US"
| sort Country, action desc

Note: Replace US with the country code that you want to filter out.

Filter requests blocked by rate-based rules

To filter your logs for requests blocked by a rate-based rule, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule, httpRequest.country as Country
| filter action = "BLOCK"
| filter terminatingRuleType = "RATE_BASED"
| sort @timestamp desc

Cross-Site Scripting (XSS) or SQL Injection

To find patterns that cause XSS or SQL Injection in the terminating rule for a custom rule or AWS Managed Rule Group, run this query. The query returns entries with a timestamp, client IP address, country of origin, match details, and the request ID:

fields @timestamp
| parse @message ',"terminatingRuleMatchDetails":[*],' as terminatingRuleMatchData
| filter (terminatingRuleMatchData like /XSS/ or terminatingRuleMatchData like /SQL/)
| display @timestamp, httpRequest.clientIp, httpRequest.country, terminatingRuleMatchData, httpRequest.requestId

Filter requests counted by a specific rule in a rule group

To filter log entries for requests that are counted by a specific rule in a rule group and then terminated by the default action, run this query:

fields @timestamp
| filter (@message like 'excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"NoUserAgent_HEADER"}]}' and @message like 'terminatingRuleId":"Default_Action"')
| parse @message '"ruleId":*}]}' as ruleMatchDetails
| display @timestamp, httpRequest.clientIp, httpRequest.country, ruleMatchDetails, httpRequest.requestId

Note: Replace ruleId with your rule ID.

Filter requests with a CAPTCHA that's not valid

To filter for the top 100 requests with a CAPTCHA that's not valid, run the following query. This query returns the time that the request was made, the IP address, the request ID, the response code, and the entire message:

fields @timestamp, httpRequest.clientIp, httpRequest.requestId, captchaResponse.failureReason, @message
|filter captchaResponse.failureReason ='TOKEN_MISSING'
| sort @timestamp desc
| limit 100

Note: Replace 100 with the number of requests that you want to filter for.

Use CloudWatch query generator to analyze AWS WAF access logs

To use generative AI to analyze your access logs, run the query generator in CloudWatch.