Log Analysis

When to use this skill

• Error debugging: analyze the root cause of application errors

• Performance analysis: analyze response times and throughput

• Security audit: detect anomalous access patterns

• Incident response: investigate the root cause during an outage

Instructions

Step 1: Locate Log Files

# Common log locations

/var/log/                    # System logs

/var/log/nginx/              # Nginx logs

/var/log/apache2/            # Apache logs

./logs/                      # Application logs

Step 2: Search for Error Patterns

Common error search:

# Search ERROR-level logs

grep -i "error\|exception\|fail" application.log

# Recent errors (last 100 lines)

tail -100 application.log | grep -i error

# Errors with timestamps

grep -E "^\[.*ERROR" application.log

HTTP error codes:

# 5xx server errors

grep -E "HTTP/[0-9.]+ 5[0-9]{2}" access.log

# 4xx client errors

grep -E "HTTP/[0-9.]+ 4[0-9]{2}" access.log

# Specific error code

grep "HTTP/1.1\" 500" access.log

Step 3: Pattern Analysis

Time-based analysis:

# Error count by time window

grep -i error application.log | cut -d' ' -f1,2 | sort | uniq -c | sort -rn

# Logs for a specific time window

grep "2025-01-05 14:" application.log

IP-based analysis:

# Request count by IP

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

# Activity for a specific IP

grep "192.168.1.100" access.log

Step 4: Performance Analysis

Response time analysis:

# Extract response times from Nginx logs

awk '{print $NF}' access.log | sort -n | tail -20

# Slow requests (>= 1 second)

awk '$NF > 1.0 {print $0}' access.log

Traffic volume analysis:

# Requests per minute

awk '{print $4}' access.log | cut -d: -f1,2,3 | uniq -c

# Requests per endpoint

awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20

Step 5: Security Analysis

Suspicious patterns:

# SQL injection attempts

grep -iE "(union|select|insert|update|delete|drop).*--" access.log

# XSS attempts

grep -iE "<script|javascript:|onerror=" access.log

# Directory traversal

grep -E "\.\./" access.log

# Brute force attack

grep -E "POST.*/login" access.log | awk '{print $1}' | sort | uniq -c | sort -rn

Output format

Analysis report structure

# Log analysis report

## Summary

- Analysis window: YYYY-MM-DD HH:MM ~ YYYY-MM-DD HH:MM

- Total log lines: X,XXX

- Error count: XXX

- Warning count: XXX

## Error analysis

| Error type | Occurrences | Last seen |

|----------|-----------|----------|

| Error A  | 150       | 2025-01-05 14:30 |

| Error B  | 45        | 2025-01-05 14:25 |

## Recommended actions

1. [Action 1]

2. [Action 2]

Best practices

• Set time range: clearly define the time window to analyze

• Save patterns: script common grep patterns

• Check context: review logs around the error too (-A, -B options)

• Log rotation: search compressed logs with zgrep as well

Constraints

Required Rules (MUST)

• Perform read-only operations only

• Mask sensitive information (passwords, tokens)

Prohibited (MUST NOT)

• Do not modify log files

• Do not expose sensitive information externally

References

• grep manual

• awk guide

• Log analysis best practices

Examples

Example 1: Basic usage

Example 2: Advanced usage