SKILL.md
Grafana Beyla
Docs: https://grafana.com/docs/beyla/latest/
Beyla is a Grafana eBPF auto-instrumentation tool that captures HTTP/gRPC traffic and generates traces
and metrics without modifying application code.
Requirements
- Linux kernel: 5.8+ with BTF (BPF Type Format) enabled
- Privileges: root or
CAP_SYS_ADMIN; in Kubernetes must run in host PID namespace
- Architectures: x86_64, ARM64
Check BTF support:
ls /sys/kernel/btf/vmlinux # must exist
Supported Languages / Runtimes
Language
HTTP
gRPC
DB queries
Go
✅
✅
✅
Java (JVM)
✅
✅
✅
Python
✅
✅
-
Ruby
✅
-
-
Node.js
✅
-
-
.NET
✅
✅
-
Rust
✅
✅
-
C/C++
✅
-
-
PHP
✅
-
-
Installation
# Docker
docker run --privileged --pid=host \
-v /sys/kernel/debug:/sys/kernel/debug:ro \
-e BEYLA_OPEN_PORT=8080 \
-e OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 \
grafana/beyla
# Kubernetes (Helm)
helm repo add grafana https://grafana.github.io/helm-charts
helm install beyla grafana/beyla \
--set discovery.services[0].open_port=8080 \
--set otelTraces.endpoint=http://tempo:4318
Configuration File
# beyla-config.yml
log_level: INFO
discovery:
services:
- name: my-app
open_port: 8080
# or by process name:
# exe_path: /usr/bin/myapp
# or by K8s pod metadata (auto-detected in K8s)
ebpf:
wakeup_len: 100 # batch size for events
track_request_headers: false # enable to capture HTTP headers (high cardinality risk)
high_request_volume: false # optimize for high-traffic services
# Distributed tracing output (OTLP)
otel_traces_export:
endpoint: http://tempo:4318 # HTTP OTLP endpoint
# Or gRPC:
# endpoint: tempo:4317
# protocol: grpc
# Metrics output (Prometheus)
prometheus_export:
port: 9090
path: /metrics
# Or metrics via OTLP
otel_metrics_export:
endpoint: http://prometheus-otlp:9090
Kubernetes Deployment
DaemonSet (recommended for cluster-wide)
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: beyla
namespace: monitoring
spec:
selector:
matchLabels:
app: beyla
template:
metadata:
labels:
app: beyla
spec:
hostPID: true # required for eBPF
serviceAccountName: beyla
containers:
- name: beyla
image: grafana/beyla:latest
securityContext:
privileged: true # or use specific capabilities
# Alternative (non-privileged):
# capabilities:
# add: [SYS_ADMIN, SYS_PTRACE, NET_ADMIN]
env:
- name: BEYLA_OPEN_PORT
value: "8080"
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "http://alloy:4318"
volumeMounts:
- name: sys-kernel-debug
mountPath: /sys/kernel/debug
readOnly: true
volumes:
- name: sys-kernel-debug
hostPath:
path: /sys/kernel/debug
Network Policies and RBAC
apiVersion: v1
kind: ServiceAccount
metadata:
name: beyla
namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: beyla
rules:
- apiGroups: [""]
resources: [nodes, pods, services, endpoints, namespaces]
verbs: [get, list, watch]
Environment Variables
Variable
Description
BEYLA_OPEN_PORT
Port(s) to instrument (e.g., 8080, 8080-8090)
BEYLA_EXECUTABLE_NAME
Process name pattern to instrument
OTEL_EXPORTER_OTLP_ENDPOINT
OTLP endpoint for traces and metrics
OTEL_EXPORTER_OTLP_PROTOCOL
grpc or http/protobuf (default)
OTEL_SERVICE_NAME
Override service name in spans
BEYLA_LOG_LEVEL
DEBUG, INFO, WARN, ERROR
BEYLA_PROMETHEUS_PORT
Port for Prometheus metrics scrape
BEYLA_PROMETHEUS_PATH
Path for Prometheus metrics (default /metrics)
Grafana Cloud Integration
# Using Alloy as the OTLP receiver
otel_traces_export:
endpoint: http://alloy:4318 # Alloy forwards to Grafana Cloud Tempo
otel_metrics_export:
endpoint: http://alloy:4318 # Alloy forwards to Grafana Cloud Prometheus
Via Alloy config:
otelcol.receiver.otlp "beyla" {
http { endpoint = "0.0.0.0:4318" }
output {
traces = [otelcol.exporter.otlp.grafana_cloud.input]
metrics = [otelcol.exporter.prometheus.local.input]
}
}
Routes Decorator (Cardinality Control)
Critical for production — prevents HTTP path cardinality explosion:
routes:
patterns:
- /user/{id}
- /api/v1/resources/{resource_id}
ignored_patterns:
- /health
- /metrics
ignore_mode: traces # or: metrics, both
unmatched: heuristic # or: path, wildcard, low-cardinality
unmatched strategies: heuristic (replaces numeric IDs, best default), low-cardinality (threshold-based collapsing), wildcard (/**), path (actual path — risk of explosion).
Trace Sampling
otel_traces_export:
sampler:
name: "parentbased_traceidratio" # parent-aware fraction sampling
arg: "0.1" # 10% sampling — arg is a quoted string
Samplers: always_on, always_off, traceidratio, parentbased_always_on (default), parentbased_traceidratio.
Generated Metrics
Metric
Type
Description
http.server.request.duration
Histogram
Inbound HTTP request duration
http.client.request.duration
Histogram
Outbound HTTP request duration
rpc.server.duration
Histogram
gRPC server call duration
rpc.client.duration
Histogram
gRPC client call duration
db.client.operation.duration
Histogram
DB query duration
Labels: http.method, http.route, http.response.status_code, service.name, service.namespace
Kubernetes Auto-Discovery
In Kubernetes, Beyla auto-discovers pods and enriches telemetry with K8s metadata:
discovery:
services:
- k8s_namespace: "production" # instrument all pods in namespace
- k8s_pod_name: "frontend.*" # by pod name regex
- k8s_deployment_name: "api" # by deployment name
- open_port: 8080 # or by port (any pod)
Auto-enriched span attributes: k8s.namespace.name, k8s.pod.name, k8s.node.name, k8s.deployment.name