SKILL.md
mTLS Configuration
Comprehensive guide to implementing mutual TLS for zero-trust service mesh communication.
When to Use This Skill
- Implementing zero-trust networking
- Securing service-to-service communication
- Certificate rotation and management
- Debugging TLS handshake issues
- Compliance requirements (PCI-DSS, HIPAA)
- Multi-cluster secure communication
Core Concepts
1. mTLS Flow
┌─────────┐ ┌─────────┐
│ Service │ │ Service │
│ A │ │ B │
└────┬────┘ └────┬────┘
│ │
┌────┴────┐ TLS Handshake ┌────┴────┐
│ Proxy │◄───────────────────────────►│ Proxy │
│(Sidecar)│ 1. ClientHello │(Sidecar)│
│ │ 2. ServerHello + Cert │ │
│ │ 3. Client Cert │ │
│ │ 4. Verify Both Certs │ │
│ │ 5. Encrypted Channel │ │
└─────────┘ └─────────┘2. Certificate Hierarchy
Root CA (Self-signed, long-lived)
│
├── Intermediate CA (Cluster-level)
│ │
│ ├── Workload Cert (Service A)
│ └── Workload Cert (Service B)
│
└── Intermediate CA (Multi-cluster)
│
└── Cross-cluster certsTemplates
Template 1: Istio mTLS (Strict Mode)
# Enable strict mTLS mesh-wide
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system
spec:
mtls:
mode: STRICT
---
# Namespace-level override (permissive for migration)
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: legacy-namespace
spec:
mtls:
mode: PERMISSIVE
---
# Workload-specific policy
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: payment-service
namespace: production
spec:
selector:
matchLabels:
app: payment-service
mtls:
mode: STRICT
portLevelMtls:
8080:
mode: STRICT
9090:
mode: DISABLE # Metrics port, no mTLSTemplate 2: Istio Destination Rule for mTLS
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: default
namespace: istio-system
spec:
host: "*.local"
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
---
# TLS to external service
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: external-api
spec:
host: api.external.com
trafficPolicy:
tls:
mode: SIMPLE
caCertificates: /etc/certs/external-ca.pem
---
# Mutual TLS to external service
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: partner-api
spec:
host: api.partner.com
trafficPolicy:
tls:
mode: MUTUAL
clientCertificate: /etc/certs/client.pem
privateKey: /etc/certs/client-key.pem
caCertificates: /etc/certs/partner-ca.pemTemplate 3: Cert-Manager with Istio
# Install cert-manager issuer for Istio
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: istio-ca
spec:
ca:
secretName: istio-ca-secret
---
# Create Istio CA secret
apiVersion: v1
kind: Secret
metadata:
name: istio-ca-secret
namespace: cert-manager
type: kubernetes.io/tls
data:
tls.crt: <base64-encoded-ca-cert>
tls.key: <base64-encoded-ca-key>
---
# Certificate for workload
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my-service-cert
namespace: my-namespace
spec:
secretName: my-service-tls
duration: 24h
renewBefore: 8h
issuerRef:
name: istio-ca
kind: ClusterIssuer
commonName: my-service.my-namespace.svc.cluster.local
dnsNames:
- my-service
- my-service.my-namespace
- my-service.my-namespace.svc
- my-service.my-namespace.svc.cluster.local
usages:
- server auth
- client authTemplate 4: SPIFFE/SPIRE Integration
# SPIRE Server configuration
apiVersion: v1
kind: ConfigMap
metadata:
name: spire-server
namespace: spire
data:
server.conf: |
server {
bind_address = "0.0.0.0"
bind_port = "8081"
trust_domain = "example.org"
data_dir = "/run/spire/data"
log_level = "INFO"
ca_ttl = "168h"
default_x509_svid_ttl = "1h"
}
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
connection_string = "/run/spire/data/datastore.sqlite3"
}
}
NodeAttestor "k8s_psat" {
plugin_data {
clusters = {
"demo-cluster" = {
service_account_allow_list = ["spire:spire-agent"]
}
}
}
}
KeyManager "memory" {
plugin_data {}
}
UpstreamAuthority "disk" {
plugin_data {
key_file_path = "/run/spire/secrets/bootstrap.key"
cert_file_path = "/run/spire/secrets/bootstrap.crt"
}
}
}
---
# SPIRE Agent DaemonSet (abbreviated)
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: spire-agent
namespace: spire
spec:
selector:
matchLabels:
app: spire-agent
template:
spec:
containers:
- name: spire-agent
image: ghcr.io/spiffe/spire-agent:1.8.0
volumeMounts:
- name: spire-agent-socket
mountPath: /run/spire/sockets
volumes:
- name: spire-agent-socket
hostPath:
path: /run/spire/sockets
type: DirectoryOrCreateTemplate 5: Linkerd mTLS (Automatic)
# Linkerd enables mTLS automatically
# Verify with:
# linkerd viz edges deployment -n my-namespace
# For external services without mTLS
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata:
name: external-api
namespace: my-namespace
spec:
podSelector:
matchLabels:
app: my-app
port: external-api
proxyProtocol: HTTP/1 # or TLS for passthrough
---
# Skip TLS for specific port
apiVersion: v1
kind: Service
metadata:
name: my-service
annotations:
config.linkerd.io/skip-outbound-ports: "3306" # MySQLCertificate Rotation
# Istio - Check certificate expiry
istioctl proxy-config secret deploy/my-app -o json | \
jq '.dynamicActiveSecrets[0].secret.tlsCertificate.certificateChain.inlineBytes' | \
tr -d '"' | base64 -d | openssl x509 -text -noout
# Force certificate rotation
kubectl rollout restart deployment/my-app
# Check Linkerd identity
linkerd identity -n my-namespaceDebugging mTLS Issues
# Istio - Check if mTLS is enabled
istioctl authn tls-check my-service.my-namespace.svc.cluster.local
# Verify peer authentication
kubectl get peerauthentication --all-namespaces
# Check destination rules
kubectl get destinationrule --all-namespaces
# Debug TLS handshake
istioctl proxy-config log deploy/my-app --level debug
kubectl logs deploy/my-app -c istio-proxy | grep -i tls
# Linkerd - Check mTLS status
linkerd viz edges deployment -n my-namespace
linkerd viz tap deploy/my-app --to deploy/my-backendBest Practices
Do's
- Start with PERMISSIVE - Migrate gradually to STRICT
- Monitor certificate expiry - Set up alerts
- Use short-lived certs - 24h or less for workloads
- Rotate CA periodically - Plan for CA rotation
- Log TLS errors - For debugging and audit
Don'ts
- Don't disable mTLS - For convenience in production
- Don't ignore cert expiry - Automate rotation
- Don't use self-signed certs - Use proper CA hierarchy
- Don't skip verification - Verify the full chain