SKILL.md
$27
- Enable or disable security audit logging on a cluster
- Select which security events to record (authentication, access, config changes)
- Create filter policies to reduce audit log noise
- Query audit logs for failed authentication attempts
- Investigate unauthorized access or privilege escalation incidents
- Set up compliance-focused audit configuration
- Detect brute-force login patterns from audit data
- Configure audit output to an index for programmatic querying
Prerequisites
Item
Description
Elasticsearch URL
Cluster endpoint (e.g. https://localhost:9200 or a Cloud deployment URL)
Authentication
Valid credentials (see the elasticsearch-authn skill)
Cluster privileges
manage cluster privilege to update cluster settings
License
Audit logging requires a gold, platinum, enterprise, or trial license
Prompt the user for any missing values.
Enable Audit Logging
Enable audit logging dynamically without a restart:
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.enabled": true
}
}'To disable, set xpack.security.audit.enabled to false. Verify current state:
curl "${ELASTICSEARCH_URL}/_cluster/settings?include_defaults=true&flat_settings=true" \
<auth_flags> | jq '.defaults | with_entries(select(.key | startswith("xpack.security.audit")))'Audit Output
Audit events can be written to two outputs. Both can be active simultaneously.
Output
Setting value
Description
logfile
logfile
Written to <ES_HOME>/logs/<cluster>_audit.json. Default.
index
index
Written to .security-audit-* indices. Queryable via the API.
Configure output via API
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.enabled": true,
"xpack.security.audit.outputs": ["index", "logfile"]
}
}'The index output is required for programmatic querying of audit events. The logfile output is useful for shipping to
external SIEM tools via Filebeat.
Note: On self-managed clusters, xpack.security.audit.outputs may require a static setting in elasticsearch.yml
on older versions (pre-8.x). On 8.x+, prefer the cluster settings API.
Select Events to Record
Control which event types are included or excluded. By default, all events are recorded when audit is enabled.
Include specific events only
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.logfile.events.include": [
"authentication_failed",
"access_denied",
"access_granted",
"anonymous_access_denied",
"tampered_request",
"run_as_denied",
"connection_denied"
]
}
}'Exclude noisy events
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.logfile.events.exclude": [
"access_granted"
]
}
}'Excluding access_granted significantly reduces log volume on busy clusters — use this when only failures matter.
Event types reference
Event
Fires when
authentication_failed
Credentials were rejected
authentication_success
User authenticated successfully
access_granted
An authorized action was performed
access_denied
An action was denied due to insufficient privileges
anonymous_access_denied
An unauthenticated request was rejected
tampered_request
A request was detected as tampered with
connection_granted
A node joined the cluster (transport layer)
connection_denied
A node connection was rejected
run_as_granted
A run-as impersonation was authorized
run_as_denied
A run-as impersonation was denied
security_config_change
A security setting was changed (role, user, API key, etc.)
See references/api-reference.md for the complete event type list with field details.
Filter Policies
Filter policies let you suppress specific audit events by user, realm, role, or index without disabling the event type
globally. Multiple policies can be active — an event is logged only if no policy filters it out.
Ignore system and internal users
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.logfile.events.ignore_filters": {
"system_users": {
"users": ["_xpack_security", "_xpack", "elastic/fleet-server"],
"realms": ["_service_account"]
}
}
}
}'Ignore health-check traffic on specific indices
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.logfile.events.ignore_filters": {
"health_checks": {
"users": ["monitoring-user"],
"indices": [".monitoring-*"]
}
}
}
}'Filter policy fields
Field
Type
Description
users
array[string]
Usernames to exclude (supports wildcards)
realms
array[string]
Realm names to exclude
roles
array[string]
Role names to exclude
indices
array[string]
Index names or patterns to exclude (supports *)
actions
array[string]
Action names to exclude (e.g. indices:data/read/*)
An event is filtered out if it matches all specified fields within a single policy.
Remove a filter policy
Set the policy to null:
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.logfile.events.ignore_filters.health_checks": null
}
}'Query Audit Events
When the index output is enabled, audit events are stored in .security-audit-* indices and can be queried.
Search for failed authentication attempts
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "term": { "event.action": "authentication_failed" } },
{ "range": { "@timestamp": { "gte": "now-24h" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "desc" } }],
"size": 50
}'Search for access denied events on a specific index
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "term": { "event.action": "access_denied" } },
{ "term": { "indices": "logs-*" } },
{ "range": { "@timestamp": { "gte": "now-7d" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "desc" } }],
"size": 20
}'Search for security configuration changes
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "term": { "event.action": "security_config_change" } },
{ "range": { "@timestamp": { "gte": "now-7d" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "desc" } }],
"size": 50
}'This captures role creation/deletion, user changes, API key operations, and role mapping updates.
Count events by type and detect brute-force patterns
Use terms aggregations on event.action (with size: 0) to count events by type over a time window. To detect
brute-force attempts, aggregate authentication_failed events by source.ip with min_doc_count: 5. See
references/api-reference.md for full aggregation query examples.
Correlate with Kibana Audit Logs
Kibana has its own audit log covering application-layer events that Elasticsearch does not see (saved object CRUD,
Kibana logins, space operations). When a user performs an action in Kibana, Kibana makes requests to Elasticsearch on
the user's behalf. Both systems record the same trace.id (passed via the X-Opaque-Id header), which serves as the
primary correlation key.
Prerequisite: Kibana audit must be enabled separately in kibana.yml. See the kibana-audit skill for setup
instructions, event types, and Kibana-specific filter policies.
Find ES audit events triggered by a Kibana action
Given a trace.id from a Kibana audit event, search the ES audit index to see the underlying Elasticsearch operations:
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "term": { "trace.id": "'"${TRACE_ID}"'" } },
{ "range": { "@timestamp": { "gte": "now-24h" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "asc" } }]
}'Correlate by user and time window
When trace.id is unavailable (e.g. direct API calls), fall back to user + time-window correlation:
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "term": { "user.name": "'"${USERNAME}"'" } },
{ "range": { "@timestamp": { "gte": "now-5m" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "asc" } }]
}'Secondary correlation fields: user.name, source.ip, and @timestamp.
Unified querying
Ship Kibana audit logs to Elasticsearch via Filebeat (see kibana-audit for the Filebeat config) so that both
.security-audit-* (ES) and kibana-audit-* (Kibana) indices can be searched together in a single multi-index query
filtered by trace.id.
Examples
Enable audit logging for compliance
Request: "Enable audit logging and record all failed access and authentication events."
curl -X PUT "${ELASTICSEARCH_URL}/_cluster/settings" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"persistent": {
"xpack.security.audit.enabled": true,
"xpack.security.audit.logfile.events.include": [
"authentication_failed",
"access_denied",
"anonymous_access_denied",
"run_as_denied",
"connection_denied",
"tampered_request",
"security_config_change"
]
}
}'This captures all denial and security change events while excluding high-volume success events.
Investigate a suspected unauthorized access attempt
Request: "Someone may have tried to access the secrets-* index. Check the audit logs."
curl -X POST "${ELASTICSEARCH_URL}/.security-audit-*/_search" \
<auth_flags> \
-H "Content-Type: application/json" \
-d '{
"query": {
"bool": {
"filter": [
{ "terms": { "event.action": ["access_denied", "authentication_failed"] } },
{ "wildcard": { "indices": "secrets-*" } },
{ "range": { "@timestamp": { "gte": "now-48h" } } }
]
}
},
"sort": [{ "@timestamp": { "order": "desc" } }],
"size": 100
}'Review user.name, source.ip, and event.action in the results to identify the actor and pattern.
Reduce audit noise on a busy cluster
Request: "Audit logs are too large. Filter out monitoring traffic and successful reads."
Exclude access_granted from event types, then add a filter policy for monitoring users and indices. See
[Filter Policies](#filter-policies) for the full syntax.
Guidelines
Prefer index output for programmatic access
Enable the index output to make audit events queryable. The logfile output is better for shipping to external SIEM
tools via Filebeat but cannot be queried through the Elasticsearch API.
Start restrictive, then widen
Begin with failure events only (authentication_failed, access_denied, security_config_change). Add success events
only when needed — they generate high volume.
Use filter policies instead of disabling events
Suppress specific users or indices with filter policies rather than excluding entire event types.
Monitor audit index size
Set up an ILM policy to roll over and delete old .security-audit-* indices. A 30-90 day retention is typical.
Enable Kibana audit for full coverage
For application-layer events (saved object access, Kibana logins, space operations), enable Kibana audit logging as
well. See the kibana-audit skill for setup. Use trace.id to correlate — see
[Correlate with Kibana Audit Logs](#correlate-with-kibana-audit-logs) above.
Avoid superuser credentials
Use a dedicated admin user or API key with manage privileges. Reserve elastic for emergency recovery only.
Deployment Compatibility
Capability
Self-managed
ECH
Serverless
ES audit via cluster settings
Yes
Yes
Not available
ES logfile output
Yes
Via Cloud UI
Not available
ES index output
Yes
Yes
Not available
Filter policies via cluster settings
Yes
Yes
Not available
Query .security-audit-*
Yes
Yes
Not available
ECH notes: ES audit is configured via the cluster settings API. Logfile output is accessible through the Cloud
console deployment logs. Index output works the same as self-managed.
Serverless notes:
- Audit logging is not user-configurable on Serverless. Security events are managed by Elastic as part of the platform.
- If a user asks about auditing on Serverless, direct them to the Elastic Cloud console or their account team.