Docker Compose Orchestration

A comprehensive skill for orchestrating multi-container applications using Docker Compose. This skill enables rapid development, deployment, and management of containerized applications with service definitions, networking strategies, volume management, health checks, and production-ready configurations.

When to Use This Skill

Use this skill when:

• Building multi-container applications (microservices, full-stack apps)

• Setting up development environments with databases, caching, and services

• Orchestrating frontend, backend, and database services together

• Managing service dependencies and startup order

• Configuring networks and inter-service communication

• Implementing persistent storage with volumes

• Deploying applications to development, staging, or production

• Creating reproducible development environments

• Managing application lifecycle (start, stop, rebuild, scale)

• Monitoring application health and implementing health checks

• Migrating from single containers to multi-service architectures

• Testing distributed systems locally

Core Concepts

Docker Compose Philosophy

Docker Compose simplifies multi-container application management through:

• Declarative Configuration: Define entire application stacks in YAML

• Service Abstraction: Each component is a service with its own configuration

• Automatic Networking: Services can communicate by name automatically

• Volume Management: Persistent data and shared storage across containers

• Environment Isolation: Each project gets its own network namespace

• Reproducibility: Same configuration works across all environments

Key Docker Compose Entities

• Services: Individual containers and their configurations

• Networks: Communication channels between services

• Volumes: Persistent storage and data sharing

• Configs: Non-sensitive configuration files

• Secrets: Sensitive data (passwords, API keys)

• Projects: Collection of services under a single namespace

Compose File Structure

version: "3.8"  # Compose file format version

services:       # Define containers

  service-name:

    # Service configuration

networks:       # Define custom networks

  network-name:

    # Network configuration

volumes:        # Define named volumes

  volume-name:

    # Volume configuration

configs:        # Application configs (optional)

  config-name:

    # Config source

secrets:        # Sensitive data (optional)

  secret-name:

    # Secret source

Service Definition Patterns

Basic Service Definition

services:

  web:

    image: nginx:alpine           # Use existing image

    container_name: my-web        # Custom container name

    restart: unless-stopped       # Restart policy

    ports:

      - "80:80"                   # Host:Container port mapping

    environment:

      - ENV_VAR=value             # Environment variables

    volumes:

      - ./html:/usr/share/nginx/html  # Volume mount

    networks:

      - frontend                  # Connect to network

Build-Based Service

services:

  app:

    build:

      context: ./app              # Build context directory

      dockerfile: Dockerfile      # Custom Dockerfile

      args:                       # Build arguments

        NODE_ENV: development

      target: development         # Multi-stage build target

    image: myapp:latest           # Tag resulting image

    ports:

      - "3000:3000"

Service with Dependencies

services:

  web:

    image: nginx

    depends_on:

      db:

        condition: service_healthy  # Wait for health check

      redis:

        condition: service_started  # Wait for start only

  db:

    image: postgres:15

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U postgres"]

      interval: 10s

      timeout: 5s

      retries: 5

      start_period: 30s

  redis:

    image: redis:alpine

Service with Advanced Configuration

services:

  backend:

    build: ./backend

    command: npm run dev          # Override default command

    working_dir: /app             # Set working directory

    user: "1000:1000"             # Run as specific user

    hostname: api-server          # Custom hostname

    domainname: example.com       # Domain name

    env_file:

      - .env                      # Load env from file

      - .env.local

    environment:

      DATABASE_URL: "postgresql://db:5432/myapp"

      REDIS_URL: "redis://cache:6379"

    volumes:

      - ./backend:/app            # Source code mount

      - /app/node_modules         # Preserve node_modules

      - app-data:/data            # Named volume

    ports:

      - "3000:3000"               # Application port

      - "9229:9229"               # Debug port

    expose:

      - "8080"                    # Expose to other services only

    networks:

      - backend

      - frontend

    labels:

      - "com.example.description=Backend API"

      - "com.example.version=1.0"

    logging:

      driver: json-file

      options:

        max-size: "10m"

        max-file: "3"

    deploy:

      resources:

        limits:

          cpus: '2'

          memory: 1G

        reservations:

          cpus: '0.5'

          memory: 512M

Multi-Container Application Patterns

Pattern 1: Full-Stack Web Application

Scenario: React frontend + Node.js backend + PostgreSQL database

version: "3.8"

services:

  # Frontend React Application

  frontend:

    build:

      context: ./frontend

      dockerfile: Dockerfile

      target: development

    ports:

      - "3000:3000"

    volumes:

      - ./frontend/src:/app/src

      - /app/node_modules

    environment:

      - REACT_APP_API_URL=http://localhost:4000/api

      - CHOKIDAR_USEPOLLING=true  # For hot reload

    networks:

      - frontend

    depends_on:

      - backend

  # Backend Node.js API

  backend:

    build:

      context: ./backend

      dockerfile: Dockerfile

    ports:

      - "4000:4000"

      - "9229:9229"  # Debugger

    volumes:

      - ./backend:/app

      - /app/node_modules

    environment:

      - NODE_ENV=development

      - DATABASE_URL=postgresql://postgres:password@db:5432/myapp

      - REDIS_URL=redis://cache:6379

      - JWT_SECRET=dev-secret

    env_file:

      - ./backend/.env.local

    networks:

      - frontend

      - backend

    depends_on:

      db:

        condition: service_healthy

      cache:

        condition: service_started

    command: npm run dev

  # PostgreSQL Database

  db:

    image: postgres:15-alpine

    container_name: postgres-db

    restart: unless-stopped

    ports:

      - "5432:5432"

    environment:

      - POSTGRES_USER=postgres

      - POSTGRES_PASSWORD=password

      - POSTGRES_DB=myapp

    volumes:

      - postgres-data:/var/lib/postgresql/data

      - ./database/init.sql:/docker-entrypoint-initdb.d/init.sql

    networks:

      - backend

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U postgres"]

      interval: 10s

      timeout: 5s

      retries: 5

  # Redis Cache

  cache:

    image: redis:7-alpine

    container_name: redis-cache

    restart: unless-stopped

    ports:

      - "6379:6379"

    volumes:

      - redis-data:/data

    networks:

      - backend

    command: redis-server --appendonly yes

    healthcheck:

      test: ["CMD", "redis-cli", "ping"]

      interval: 10s

      timeout: 3s

      retries: 5

networks:

  frontend:

    driver: bridge

  backend:

    driver: bridge

volumes:

  postgres-data:

    driver: local

  redis-data:

    driver: local

Pattern 2: Microservices Architecture

Scenario: Multiple services with reverse proxy and service discovery

version: "3.8"

services:

  # NGINX Reverse Proxy

  proxy:

    image: nginx:alpine

    container_name: reverse-proxy

    ports:

      - "80:80"

      - "443:443"

    volumes:

      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro

      - ./nginx/conf.d:/etc/nginx/conf.d:ro

      - ./ssl:/etc/nginx/ssl:ro

    networks:

      - public

    depends_on:

      - auth-service

      - user-service

      - order-service

    restart: unless-stopped

  # Authentication Service

  auth-service:

    build: ./services/auth

    container_name: auth-service

    expose:

      - "8001"

    environment:

      - SERVICE_NAME=auth

      - DATABASE_URL=postgresql://db:5432/auth_db

      - JWT_SECRET=${JWT_SECRET}

    networks:

      - public

      - internal

    depends_on:

      db:

        condition: service_healthy

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:8001/health"]

      interval: 30s

      timeout: 10s

      retries: 3

  # User Service

  user-service:

    build: ./services/user

    container_name: user-service

    expose:

      - "8002"

    environment:

      - SERVICE_NAME=user

      - DATABASE_URL=postgresql://db:5432/user_db

      - AUTH_SERVICE_URL=http://auth-service:8001

    networks:

      - public

      - internal

    depends_on:

      - auth-service

      - db

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:8002/health"]

      interval: 30s

      timeout: 10s

      retries: 3

  # Order Service

  order-service:

    build: ./services/order

    container_name: order-service

    expose:

      - "8003"

    environment:

      - SERVICE_NAME=order

      - DATABASE_URL=postgresql://db:5432/order_db

      - USER_SERVICE_URL=http://user-service:8002

      - RABBITMQ_URL=amqp://rabbitmq:5672

    networks:

      - public

      - internal

    depends_on:

      - user-service

      - db

      - rabbitmq

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:8003/health"]

      interval: 30s

      timeout: 10s

      retries: 3

  # Shared PostgreSQL Database

  db:

    image: postgres:15-alpine

    container_name: postgres-db

    environment:

      - POSTGRES_USER=postgres

      - POSTGRES_PASSWORD=${DB_PASSWORD}

    volumes:

      - postgres-data:/var/lib/postgresql/data

      - ./database/init-multi-db.sql:/docker-entrypoint-initdb.d/init.sql

    networks:

      - internal

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U postgres"]

      interval: 10s

      timeout: 5s

      retries: 5

  # RabbitMQ Message Broker

  rabbitmq:

    image: rabbitmq:3-management-alpine

    container_name: rabbitmq

    ports:

      - "5672:5672"   # AMQP

      - "15672:15672" # Management UI

    environment:

      - RABBITMQ_DEFAULT_USER=admin

      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASSWORD}

    volumes:

      - rabbitmq-data:/var/lib/rabbitmq

    networks:

      - internal

    healthcheck:

      test: ["CMD", "rabbitmq-diagnostics", "ping"]

      interval: 30s

      timeout: 10s

      retries: 5

networks:

  public:

    driver: bridge

  internal:

    driver: bridge

    internal: true  # No external access

volumes:

  postgres-data:

  rabbitmq-data:

Pattern 3: Development Environment with Hot Reload

Scenario: Development setup with live code reloading and debugging

version: "3.8"

services:

  # Development Frontend

  frontend-dev:

    build:

      context: ./frontend

      dockerfile: Dockerfile.dev

    ports:

      - "3000:3000"

      - "9222:9222"  # Chrome DevTools

    volumes:

      - ./frontend:/app

      - /app/node_modules

      - /app/.next  # Next.js build cache

    environment:

      - NODE_ENV=development

      - WATCHPACK_POLLING=true

      - NEXT_PUBLIC_API_URL=http://localhost:4000

    networks:

      - dev-network

    stdin_open: true

    tty: true

    command: npm run dev

  # Development Backend

  backend-dev:

    build:

      context: ./backend

      dockerfile: Dockerfile.dev

    ports:

      - "4000:4000"

      - "9229:9229"  # Node.js debugger

    volumes:

      - ./backend:/app

      - /app/node_modules

    environment:

      - NODE_ENV=development

      - DEBUG=app:*

      - DATABASE_URL=postgresql://postgres:dev@db:5432/dev_db

    networks:

      - dev-network

    depends_on:

      - db

      - mailhog

    command: npm run dev:debug

  # PostgreSQL with pgAdmin

  db:

    image: postgres:15-alpine

    environment:

      - POSTGRES_PASSWORD=dev

      - POSTGRES_DB=dev_db

    ports:

      - "5432:5432"

    volumes:

      - dev-db-data:/var/lib/postgresql/data

    networks:

      - dev-network

  pgadmin:

    image: dpage/pgadmin4:latest

    environment:

      - PGADMIN_DEFAULT_EMAIL=admin@dev.local

      - PGADMIN_DEFAULT_PASSWORD=admin

    ports:

      - "5050:80"

    networks:

      - dev-network

    depends_on:

      - db

  # MailHog for Email Testing

  mailhog:

    image: mailhog/mailhog:latest

    ports:

      - "1025:1025"  # SMTP

      - "8025:8025"  # Web UI

    networks:

      - dev-network

networks:

  dev-network:

    driver: bridge

volumes:

  dev-db-data:

Networking Strategies

Default Bridge Network

services:

  web:

    image: nginx

    # Automatically connected to default network

  app:

    image: myapp

    # Can communicate with 'web' via service name

Custom Bridge Networks

version: "3.8"

services:

  frontend:

    image: react-app

    networks:

      - public

  backend:

    image: api-server

    networks:

      - public    # Accessible from frontend

      - private   # Accessible from database

  database:

    image: postgres

    networks:

      - private   # Isolated from frontend

networks:

  public:

    driver: bridge

  private:

    driver: bridge

    internal: true  # No internet access

Network Aliases

services:

  api:

    image: api-server

    networks:

      backend:

        aliases:

          - api-server

          - api.internal

          - api-v1.internal

networks:

  backend:

    driver: bridge

Host Network Mode

services:

  app:

    image: myapp

    network_mode: "host"  # Use host network stack

    # No port mapping needed, uses host ports directly

Custom Network Configuration

networks:

  custom-network:

    driver: bridge

    driver_opts:

      com.docker.network.bridge.name: br-custom

    ipam:

      driver: default

      config:

        - subnet: 172.28.0.0/16

          gateway: 172.28.0.1

    labels:

      - "com.example.description=Custom network"

Volume Management

Named Volumes

version: "3.8"

services:

  db:

    image: postgres:15

    volumes:

      - postgres-data:/var/lib/postgresql/data  # Named volume

  backup:

    image: postgres:15

    volumes:

      - postgres-data:/backup:ro  # Read-only mount

    command: pg_dump -U postgres > /backup/dump.sql

volumes:

  postgres-data:

    driver: local

    driver_opts:

      type: none

      o: bind

      device: /path/on/host

Bind Mounts

services:

  web:

    image: nginx

    volumes:

      # Relative path bind mount

      - ./html:/usr/share/nginx/html

      # Absolute path bind mount

      - /var/log/nginx:/var/log/nginx

      # Read-only bind mount

      - ./config/nginx.conf:/etc/nginx/nginx.conf:ro

tmpfs Mounts (In-Memory)

services:

  app:

    image: myapp

    tmpfs:

      - /tmp

      - /run

    # Or with options:

    volumes:

      - type: tmpfs

        target: /app/cache

        tmpfs:

          size: 1000000000  # 1GB

Volume Sharing Between Services

services:

  app:

    image: myapp

    volumes:

      - shared-data:/data

  worker:

    image: worker

    volumes:

      - shared-data:/data

  backup:

    image: backup-tool

    volumes:

      - shared-data:/backup:ro

volumes:

  shared-data:

Advanced Volume Configuration

volumes:

  data:

    driver: local

    driver_opts:

      type: "nfs"

      o: "addr=10.40.0.199,nolock,soft,rw"

      device: ":/docker/example"

  cache:

    driver: local

    driver_opts:

      type: tmpfs

      device: tmpfs

      o: "size=100m,uid=1000"

  external-volume:

    external: true  # Volume created outside Compose

    name: my-existing-volume

Health Checks

HTTP Health Check

services:

  web:

    image: nginx

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost/health"]

      interval: 30s

      timeout: 10s

      retries: 3

      start_period: 40s

Database Health Check

services:

  postgres:

    image: postgres:15

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U postgres"]

      interval: 10s

      timeout: 5s

      retries: 5

      start_period: 30s

  mysql:

    image: mysql:8

    healthcheck:

      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]

      interval: 10s

      timeout: 5s

      retries: 3

  mongodb:

    image: mongo:6

    healthcheck:

      test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]

      interval: 10s

      timeout: 5s

      retries: 5

Application Health Check

services:

  app:

    build: ./app

    healthcheck:

      test: ["CMD", "node", "healthcheck.js"]

      interval: 30s

      timeout: 10s

      retries: 3

      start_period: 60s

  api:

    build: ./api

    healthcheck:

      test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1"]

      interval: 30s

      timeout: 10s

      retries: 3

Complex Health Checks

services:

  redis:

    image: redis:alpine

    healthcheck:

      test: |

        sh -c '

        redis-cli ping | grep PONG &&

        redis-cli --raw incr ping | grep 1

        '

      interval: 10s

      timeout: 3s

      retries: 5

Development vs Production Configurations

Base Configuration (compose.yaml)

version: "3.8"

services:

  web:

    image: myapp:latest

    environment:

      - NODE_ENV=production

    networks:

      - app-network

  db:

    image: postgres:15-alpine

    networks:

      - app-network

networks:

  app-network:

    driver: bridge

Development Override (compose.override.yaml)

# Automatically merged with compose.yaml in development

version: "3.8"

services:

  web:

    build:

      context: .

      target: development

    volumes:

      - ./src:/app/src  # Live code reload

      - /app/node_modules

    ports:

      - "3000:3000"     # Expose for local access

      - "9229:9229"     # Debugger port

    environment:

      - NODE_ENV=development

      - DEBUG=*

    command: npm run dev

  db:

    ports:

      - "5432:5432"     # Expose for local tools

    environment:

      - POSTGRES_PASSWORD=dev

    volumes:

      - ./init-dev.sql:/docker-entrypoint-initdb.d/init.sql

Production Configuration (compose.prod.yaml)

version: "3.8"

services:

  web:

    image: myapp:${VERSION:-latest}

    restart: always

    environment:

      - NODE_ENV=production

    deploy:

      replicas: 3

      resources:

        limits:

          cpus: '2'

          memory: 2G

        reservations:

          cpus: '1'

          memory: 1G

      update_config:

        parallelism: 1

        delay: 10s

        failure_action: rollback

      rollback_config:

        parallelism: 1

        delay: 5s

    logging:

      driver: json-file

      options:

        max-size: "10m"

        max-file: "5"

  db:

    image: postgres:15-alpine

    restart: always

    environment:

      - POSTGRES_PASSWORD_FILE=/run/secrets/db_password

    secrets:

      - db_password

    volumes:

      - postgres-data:/var/lib/postgresql/data

    deploy:

      resources:

        limits:

          cpus: '2'

          memory: 4G

  # Production additions

  nginx:

    image: nginx:alpine

    ports:

      - "80:80"

      - "443:443"

    volumes:

      - ./nginx/prod.conf:/etc/nginx/nginx.conf:ro

      - ssl-certs:/etc/nginx/ssl:ro

    restart: always

    depends_on:

      - web

secrets:

  db_password:

    external: true

volumes:

  postgres-data:

    driver: local

  ssl-certs:

    external: true

Staging Configuration (compose.staging.yaml)

version: "3.8"

services:

  web:

    image: myapp:staging-${VERSION:-latest}

    restart: unless-stopped

    environment:

      - NODE_ENV=staging

    deploy:

      replicas: 2

      resources:

        limits:

          cpus: '1'

          memory: 1G

  db:

    environment:

      - POSTGRES_PASSWORD=${DB_PASSWORD}

    volumes:

      - staging-db-data:/var/lib/postgresql/data

volumes:

  staging-db-data:

Essential Docker Compose Commands

Project Management

# Start services

docker compose up                    # Foreground

docker compose up -d                 # Detached (background)

docker compose up --build            # Rebuild images

docker compose up --force-recreate   # Recreate containers

docker compose up --scale web=3      # Scale service to 3 instances

# Stop services

docker compose stop                  # Stop containers

docker compose down                  # Stop and remove containers/networks

docker compose down -v               # Also remove volumes

docker compose down --rmi all        # Also remove images

# Restart services

docker compose restart               # Restart all services

docker compose restart web           # Restart specific service

Service Management

# Build services

docker compose build                 # Build all services

docker compose build web             # Build specific service

docker compose build --no-cache      # Build without cache

docker compose build --pull          # Pull latest base images

# View services

docker compose ps                    # List containers

docker compose ps -a                 # Include stopped containers

docker compose top                   # Display running processes

docker compose images                # List images

# Logs

docker compose logs                  # View all logs

docker compose logs -f               # Follow logs

docker compose logs web              # Service-specific logs

docker compose logs --tail=100 web   # Last 100 lines

Execution and Debugging

# Execute commands

docker compose exec web sh           # Interactive shell

docker compose exec web npm test     # Run command

docker compose exec -u root web sh   # Run as root

# Run one-off commands

docker compose run web npm install   # Run command in new container

docker compose run --rm web test     # Remove container after

docker compose run --no-deps web sh  # Don't start dependencies

Configuration Management

# Multiple compose files

docker compose -f compose.yaml -f compose.prod.yaml up

# Environment-specific deployment

docker compose --env-file .env.prod up

docker compose -p myproject up       # Custom project name

# Configuration validation

docker compose config                # Validate and view config

docker compose config --quiet        # Only validation

docker compose config --services     # List services

docker compose config --volumes      # List volumes

15+ Compose Examples

Example 1: NGINX + PHP + MySQL (LAMP Stack)

version: "3.8"

services:

  nginx:

    image: nginx:alpine

    ports:

      - "80:80"

    volumes:

      - ./public:/var/www/html

      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro

    networks:

      - lamp

    depends_on:

      - php

  php:

    build:

      context: ./php

      dockerfile: Dockerfile

    volumes:

      - ./public:/var/www/html

    networks:

      - lamp

    depends_on:

      - mysql

  mysql:

    image: mysql:8.0

    environment:

      MYSQL_ROOT_PASSWORD: secret

      MYSQL_DATABASE: myapp

      MYSQL_USER: user

      MYSQL_PASSWORD: password

    volumes:

      - mysql-data:/var/lib/mysql

    networks:

      - lamp

networks:

  lamp:

volumes:

  mysql-data:

Example 2: Django + PostgreSQL + Redis + Celery

version: "3.8"

services:

  web:

    build: .

    command: python manage.py runserver 0.0.0.0:8000

    volumes:

      - .:/code

    ports:

      - "8000:8000"

    environment:

      - DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db

      - REDIS_URL=redis://redis:6379/0

    depends_on:

      - db

      - redis

  db:

    image: postgres:15-alpine

    environment:

      POSTGRES_DB: django_db

      POSTGRES_USER: postgres

      POSTGRES_PASSWORD: postgres

    volumes:

      - postgres-data:/var/lib/postgresql/data

  redis:

    image: redis:alpine

    volumes:

      - redis-data:/data

  celery:

    build: .

    command: celery -A myproject worker -l info

    volumes:

      - .:/code

    environment:

      - DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db

      - REDIS_URL=redis://redis:6379/0

    depends_on:

      - db

      - redis

  celery-beat:

    build: .

    command: celery -A myproject beat -l info

    volumes:

      - .:/code

    environment:

      - DATABASE_URL=postgresql://postgres:postgres@db:5432/django_db

      - REDIS_URL=redis://redis:6379/0

    depends_on:

      - db

      - redis

volumes:

  postgres-data:

  redis-data:

Example 3: React + Node.js + MongoDB + NGINX

version: "3.8"

services:

  frontend:

    build:

      context: ./frontend

      args:

        REACT_APP_API_URL: http://localhost/api

    volumes:

      - ./frontend:/app

      - /app/node_modules

    environment:

      - CHOKIDAR_USEPOLLING=true

    networks:

      - app-network

  backend:

    build: ./backend

    ports:

      - "5000:5000"

    volumes:

      - ./backend:/app

      - /app/node_modules

    environment:

      - MONGODB_URI=mongodb://mongo:27017/myapp

      - JWT_SECRET=dev-secret

    depends_on:

      - mongo

    networks:

      - app-network

  mongo:

    image: mongo:6

    ports:

      - "27017:27017"

    volumes:

      - mongo-data:/data/db

      - mongo-config:/data/configdb

    environment:

      - MONGO_INITDB_ROOT_USERNAME=admin

      - MONGO_INITDB_ROOT_PASSWORD=secret

    networks:

      - app-network

  nginx:

    image: nginx:alpine

    ports:

      - "80:80"

    volumes:

      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro

    depends_on:

      - frontend

      - backend

    networks:

      - app-network

networks:

  app-network:

    driver: bridge

volumes:

  mongo-data:

  mongo-config:

Example 4: Spring Boot + MySQL + Adminer

version: "3.8"

services:

  app:

    build:

      context: .

      dockerfile: Dockerfile

    ports:

      - "8080:8080"

    environment:

      - SPRING_DATASOURCE_URL=jdbc:mysql://db:3306/springdb?useSSL=false

      - SPRING_DATASOURCE_USERNAME=root

      - SPRING_DATASOURCE_PASSWORD=secret

      - SPRING_JPA_HIBERNATE_DDL_AUTO=update

    depends_on:

      db:

        condition: service_healthy

    networks:

      - spring-network

  db:

    image: mysql:8.0

    environment:

      MYSQL_ROOT_PASSWORD: secret

      MYSQL_DATABASE: springdb

    volumes:

      - mysql-data:/var/lib/mysql

    networks:

      - spring-network

    healthcheck:

      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]

      interval: 10s

      timeout: 5s

      retries: 5

  adminer:

    image: adminer:latest

    ports:

      - "8081:8080"

    environment:

      ADMINER_DEFAULT_SERVER: db

    networks:

      - spring-network

networks:

  spring-network:

volumes:

  mysql-data:

Example 5: WordPress + MySQL + phpMyAdmin

version: "3.8"

services:

  wordpress:

    image: wordpress:latest

    ports:

      - "8000:80"

    environment:

      WORDPRESS_DB_HOST: db:3306

      WORDPRESS_DB_USER: wordpress

      WORDPRESS_DB_PASSWORD: wordpress

      WORDPRESS_DB_NAME: wordpress

    volumes:

      - wordpress-data:/var/www/html

    depends_on:

      - db

    networks:

      - wordpress-network

  db:

    image: mysql:8.0

    environment:

      MYSQL_DATABASE: wordpress

      MYSQL_USER: wordpress

      MYSQL_PASSWORD: wordpress

      MYSQL_ROOT_PASSWORD: rootpassword

    volumes:

      - db-data:/var/lib/mysql

    networks:

      - wordpress-network

  phpmyadmin:

    image: phpmyadmin/phpmyadmin:latest

    ports:

      - "8080:80"

    environment:

      PMA_HOST: db

      PMA_USER: root

      PMA_PASSWORD: rootpassword

    depends_on:

      - db

    networks:

      - wordpress-network

networks:

  wordpress-network:

volumes:

  wordpress-data:

  db-data:

Example 6: Elasticsearch + Kibana + Logstash (ELK Stack)

version: "3.8"

services:

  elasticsearch:

    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0

    container_name: elasticsearch

    environment:

      - discovery.type=single-node

      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

      - xpack.security.enabled=false

    ports:

      - "9200:9200"

      - "9300:9300"

    volumes:

      - elasticsearch-data:/usr/share/elasticsearch/data

    networks:

      - elk

  logstash:

    image: docker.elastic.co/logstash/logstash:8.10.0

    container_name: logstash

    volumes:

      - ./logstash/pipeline:/usr/share/logstash/pipeline:ro

      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro

    ports:

      - "5000:5000"

      - "9600:9600"

    environment:

      LS_JAVA_OPTS: "-Xmx256m -Xms256m"

    networks:

      - elk

    depends_on:

      - elasticsearch

  kibana:

    image: docker.elastic.co/kibana/kibana:8.10.0

    container_name: kibana

    ports:

      - "5601:5601"

    environment:

      ELASTICSEARCH_URL: http://elasticsearch:9200

      ELASTICSEARCH_HOSTS: http://elasticsearch:9200

    networks:

      - elk

    depends_on:

      - elasticsearch

networks:

  elk:

    driver: bridge

volumes:

  elasticsearch-data:

Example 7: GitLab + GitLab Runner

version: "3.8"

services:

  gitlab:

    image: gitlab/gitlab-ce:latest

    container_name: gitlab

    restart: unless-stopped

    hostname: gitlab.local

    environment:

      GITLAB_OMNIBUS_CONFIG: |

        external_url 'http://gitlab.local'

        gitlab_rails['gitlab_shell_ssh_port'] = 2222

    ports:

      - "80:80"

      - "443:443"

      - "2222:22"

    volumes:

      - gitlab-config:/etc/gitlab

      - gitlab-logs:/var/log/gitlab

      - gitlab-data:/var/opt/gitlab

    networks:

      - gitlab-network

  gitlab-runner:

    image: gitlab/gitlab-runner:latest

    container_name: gitlab-runner

    restart: unless-stopped

    volumes:

      - gitlab-runner-config:/etc/gitlab-runner

      - /var/run/docker.sock:/var/run/docker.sock

    networks:

      - gitlab-network

    depends_on:

      - gitlab

networks:

  gitlab-network:

volumes:

  gitlab-config:

  gitlab-logs:

  gitlab-data:

  gitlab-runner-config:

Example 8: Jenkins + Docker-in-Docker

version: "3.8"

services:

  jenkins:

    image: jenkins/jenkins:lts

    container_name: jenkins

    user: root

    ports:

      - "8080:8080"

      - "50000:50000"

    volumes:

      - jenkins-data:/var/jenkins_home

      - /var/run/docker.sock:/var/run/docker.sock

      - /usr/bin/docker:/usr/bin/docker

    environment:

      - JAVA_OPTS=-Djenkins.install.runSetupWizard=false

    networks:

      - jenkins-network

  jenkins-agent:

    image: jenkins/inbound-agent:latest

    container_name: jenkins-agent

    environment:

      - JENKINS_URL=http://jenkins:8080

      - JENKINS_AGENT_NAME=agent1

      - JENKINS_SECRET=${AGENT_SECRET}

      - JENKINS_AGENT_WORKDIR=/home/jenkins/agent

    volumes:

      - /var/run/docker.sock:/var/run/docker.sock

    networks:

      - jenkins-network

    depends_on:

      - jenkins

networks:

  jenkins-network:

volumes:

  jenkins-data:

Example 9: Prometheus + Grafana + Node Exporter

version: "3.8"

services:

  prometheus:

    image: prom/prometheus:latest

    container_name: prometheus

    ports:

      - "9090:9090"

    volumes:

      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro

      - prometheus-data:/prometheus

    command:

      - '--config.file=/etc/prometheus/prometheus.yml'

      - '--storage.tsdb.path=/prometheus'

    networks:

      - monitoring

  grafana:

    image: grafana/grafana:latest

    container_name: grafana

    ports:

      - "3000:3000"

    environment:

      - GF_SECURITY_ADMIN_USER=admin

      - GF_SECURITY_ADMIN_PASSWORD=admin

      - GF_INSTALL_PLUGINS=grafana-piechart-panel

    volumes:

      - grafana-data:/var/lib/grafana

      - ./grafana/provisioning:/etc/grafana/provisioning:ro

    networks:

      - monitoring

    depends_on:

      - prometheus

  node-exporter:

    image: prom/node-exporter:latest

    container_name: node-exporter

    ports:

      - "9100:9100"

    volumes:

      - /proc:/host/proc:ro

      - /sys:/host/sys:ro

      - /:/rootfs:ro

    command:

      - '--path.procfs=/host/proc'

      - '--path.sysfs=/host/sys'

      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'

    networks:

      - monitoring

networks:

  monitoring:

volumes:

  prometheus-data:

  grafana-data:

Example 10: RabbitMQ + Multiple Consumers

version: "3.8"

services:

  rabbitmq:

    image: rabbitmq:3-management-alpine

    container_name: rabbitmq

    ports:

      - "5672:5672"   # AMQP

      - "15672:15672" # Management UI

    environment:

      RABBITMQ_DEFAULT_USER: admin

      RABBITMQ_DEFAULT_PASS: secret

    volumes:

      - rabbitmq-data:/var/lib/rabbitmq

      - ./rabbitmq/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro

    networks:

      - messaging

    healthcheck:

      test: ["CMD", "rabbitmq-diagnostics", "ping"]

      interval: 30s

      timeout: 10s

      retries: 5

  producer:

    build: ./services/producer

    environment:

      RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672

    depends_on:

      rabbitmq:

        condition: service_healthy

    networks:

      - messaging

  consumer-1:

    build: ./services/consumer

    environment:

      RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672

      WORKER_ID: 1

    depends_on:

      rabbitmq:

        condition: service_healthy

    networks:

      - messaging

    deploy:

      replicas: 3

  consumer-2:

    build: ./services/consumer

    environment:

      RABBITMQ_URL: amqp://admin:secret@rabbitmq:5672

      WORKER_ID: 2

    depends_on:

      rabbitmq:

        condition: service_healthy

    networks:

      - messaging

networks:

  messaging:

volumes:

  rabbitmq-data:

Example 11: Traefik Reverse Proxy

version: "3.8"

services:

  traefik:

    image: traefik:v2.10

    container_name: traefik

    command:

      - --api.insecure=true

      - --providers.docker=true

      - --providers.docker.exposedbydefault=false

      - --entrypoints.web.address=:80

      - --entrypoints.websecure.address=:443

    ports:

      - "80:80"

      - "443:443"

      - "8080:8080"  # Traefik dashboard

    volumes:

      - /var/run/docker.sock:/var/run/docker.sock:ro

      - ./traefik/traefik.yml:/etc/traefik/traefik.yml:ro

      - ./traefik/dynamic:/etc/traefik/dynamic:ro

    networks:

      - traefik-network

  whoami:

    image: traefik/whoami

    labels:

      - "traefik.enable=true"

      - "traefik.http.routers.whoami.rule=Host(`whoami.local`)"

      - "traefik.http.routers.whoami.entrypoints=web"

    networks:

      - traefik-network

  app:

    image: nginx:alpine

    labels:

      - "traefik.enable=true"

      - "traefik.http.routers.app.rule=Host(`app.local`)"

      - "traefik.http.routers.app.entrypoints=web"

      - "traefik.http.services.app.loadbalancer.server.port=80"

    networks:

      - traefik-network

networks:

  traefik-network:

    driver: bridge

Example 12: MinIO + PostgreSQL Backup

version: "3.8"

services:

  minio:

    image: minio/minio:latest

    container_name: minio

    command: server /data --console-address ":9001"

    ports:

      - "9000:9000"

      - "9001:9001"

    environment:

      MINIO_ROOT_USER: minioadmin

      MINIO_ROOT_PASSWORD: minioadmin

    volumes:

      - minio-data:/data

    networks:

      - storage

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]

      interval: 30s

      timeout: 20s

      retries: 3

  postgres:

    image: postgres:15-alpine

    environment:

      POSTGRES_DB: myapp

      POSTGRES_USER: postgres

      POSTGRES_PASSWORD: secret

    volumes:

      - postgres-data:/var/lib/postgresql/data

    networks:

      - storage

  backup:

    image: postgres:15-alpine

    environment:

      POSTGRES_HOST: postgres

      POSTGRES_DB: myapp

      POSTGRES_USER: postgres

      POSTGRES_PASSWORD: secret

      MINIO_ENDPOINT: minio:9000

      MINIO_ACCESS_KEY: minioadmin

      MINIO_SECRET_KEY: minioadmin

    volumes:

      - ./scripts/backup.sh:/backup.sh:ro

    entrypoint: ["/bin/sh", "/backup.sh"]

    depends_on:

      - postgres

      - minio

    networks:

      - storage

networks:

  storage:

volumes:

  minio-data:

  postgres-data:

Example 13: Apache Kafka + Zookeeper

version: "3.8"

services:

  zookeeper:

    image: confluentinc/cp-zookeeper:latest

    container_name: zookeeper

    environment:

      ZOOKEEPER_CLIENT_PORT: 2181

      ZOOKEEPER_TICK_TIME: 2000

    ports:

      - "2181:2181"

    volumes:

      - zookeeper-data:/var/lib/zookeeper/data

      - zookeeper-logs:/var/lib/zookeeper/log

    networks:

      - kafka-network

  kafka:

    image: confluentinc/cp-kafka:latest

    container_name: kafka

    depends_on:

      - zookeeper

    ports:

      - "9092:9092"

      - "29092:29092"

    environment:

      KAFKA_BROKER_ID: 1

      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092

      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT

      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT

      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

    volumes:

      - kafka-data:/var/lib/kafka/data

    networks:

      - kafka-network

  kafka-ui:

    image: provectuslabs/kafka-ui:latest

    container_name: kafka-ui

    depends_on:

      - kafka

    ports:

      - "8080:8080"

    environment:

      KAFKA_CLUSTERS_0_NAME: local

      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092

      KAFKA_CLUSTERS_0_ZOOKEEPER: zookeeper:2181

    networks:

      - kafka-network

networks:

  kafka-network:

volumes:

  zookeeper-data:

  zookeeper-logs:

  kafka-data:

Example 14: Keycloak + PostgreSQL (Identity & Access Management)

version: "3.8"

services:

  postgres:

    image: postgres:15-alpine

    container_name: keycloak-db

    environment:

      POSTGRES_DB: keycloak

      POSTGRES_USER: keycloak

      POSTGRES_PASSWORD: password

    volumes:

      - postgres-data:/var/lib/postgresql/data

    networks:

      - keycloak-network

  keycloak:

    image: quay.io/keycloak/keycloak:latest

    container_name: keycloak

    environment:

      KC_DB: postgres

      KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak

      KC_DB_USERNAME: keycloak

      KC_DB_PASSWORD: password

      KEYCLOAK_ADMIN: admin

      KEYCLOAK_ADMIN_PASSWORD: admin

    command: start-dev

    ports:

      - "8080:8080"

    depends_on:

      - postgres

    networks:

      - keycloak-network

networks:

  keycloak-network:

volumes:

  postgres-data:

Example 15: Portainer (Docker Management UI)

version: "3.8"

services:

  portainer:

    image: portainer/portainer-ce:latest

    container_name: portainer

    restart: unless-stopped

    ports:

      - "9000:9000"

      - "8000:8000"

    volumes:

      - /var/run/docker.sock:/var/run/docker.sock

      - portainer-data:/data

    networks:

      - portainer-network

networks:

  portainer-network:

volumes:

  portainer-data:

Example 16: SonarQube + PostgreSQL (Code Quality)

version: "3.8"

services:

  sonarqube:

    image: sonarqube:community

    container_name: sonarqube

    depends_on:

      - db

    environment:

      SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonar

      SONAR_JDBC_USERNAME: sonar

      SONAR_JDBC_PASSWORD: sonar

    volumes:

      - sonarqube-conf:/opt/sonarqube/conf

      - sonarqube-data:/opt/sonarqube/data

      - sonarqube-logs:/opt/sonarqube/logs

      - sonarqube-extensions:/opt/sonarqube/extensions

    ports:

      - "9000:9000"

    networks:

      - sonarqube-network

  db:

    image: postgres:15-alpine

    container_name: sonarqube-db

    environment:

      POSTGRES_USER: sonar

      POSTGRES_PASSWORD: sonar

      POSTGRES_DB: sonar

    volumes:

      - postgresql-data:/var/lib/postgresql/data

    networks:

      - sonarqube-network

networks:

  sonarqube-network:

volumes:

  sonarqube-conf:

  sonarqube-data:

  sonarqube-logs:

  sonarqube-extensions:

  postgresql-data:

Best Practices

Service Configuration

• Use Specific Image Tags: Avoid latest in production

• Health Checks: Always define health checks for critical services

• Resource Limits: Set CPU and memory limits in production

• Restart Policies: Use appropriate restart policies

• Environment Variables: Use .env files for sensitive data

• Named Volumes: Use named volumes for data persistence

• Network Isolation: Separate frontend/backend networks

• Logging Configuration: Set up proper log rotation

Development Workflow

• Hot Reload: Mount source code as volumes for live updates

• Debug Ports: Expose debugger ports in development

• Override Files: Use compose.override.yaml for local config

• Build Caching: Structure Dockerfiles for efficient caching

• Separate Concerns: One process per container

• Service Naming: Use descriptive, consistent service names

Security

• Secrets Management: Use Docker secrets or external secret managers

• Non-Root Users: Run containers as non-root users

• Read-Only Filesystems: Mount volumes as read-only when possible

• Network Segmentation: Use multiple networks for isolation

• Environment Isolation: Never commit sensitive .env files

• Image Scanning: Scan images for vulnerabilities

• Minimal Base Images: Use Alpine or distroless images

Production Deployment

• Image Versioning: Tag images with semantic versions

• Rolling Updates: Configure gradual rollout strategies

• Monitoring: Integrate with monitoring solutions

• Backup Strategy: Implement automated backups

• High Availability: Deploy replicas of critical services

• Load Balancing: Use reverse proxies for load distribution

• Configuration Management: Externalize configuration

• Disaster Recovery: Test backup and restore procedures

Troubleshooting

Common Issues

Services can't communicate

• Check network configuration

• Verify service names are correct

• Ensure services are on same network

• Check firewall rules

Volumes not persisting

• Verify named volumes are defined

• Check volume mount paths

• Ensure proper permissions

• Review Docker volume driver

Services failing health checks

• Increase start_period

• Verify health check command

• Check service logs

• Ensure dependencies are ready

Port conflicts

• Check for existing services on ports

• Use different host ports

• Review port mapping syntax

Build failures

• Clear build cache: docker compose build --no-cache

• Check Dockerfile syntax

• Verify build context

• Review build arguments

Debugging Commands

# View detailed container information

docker compose ps -a

docker compose logs -f service-name

docker inspect container-name

# Execute commands in running containers

docker compose exec service-name sh

docker compose exec service-name env

# Check network connectivity

docker compose exec service-name ping other-service

docker compose exec service-name netstat -tulpn

# Review configuration

docker compose config

docker compose config --services

docker compose config --volumes

# Clean up resources

docker compose down -v

docker system prune -a --volumes

Advanced Usage

Multi-Stage Builds for Optimization

services:

  app:

    build:

      context: .

      dockerfile: Dockerfile

      target: production

    # Dockerfile uses multi-stage builds

# Development stage

FROM node:18-alpine AS development

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . .

CMD ["npm", "run", "dev"]

# Build stage

FROM node:18-alpine AS builder

WORKDIR /app

COPY package*.json ./

RUN npm ci --only=production

COPY . .

RUN npm run build

# Production stage

FROM node:18-alpine AS production

WORKDIR /app

COPY --from=builder /app/dist ./dist

COPY --from=builder /app/node_modules ./node_modules

COPY package*.json ./

EXPOSE 3000

CMD ["node", "dist/index.js"]

Environment-Specific Deployments

# Development

docker compose up

# Staging

docker compose -f compose.yaml -f compose.staging.yaml up

# Production

docker compose -f compose.yaml -f compose.prod.yaml up -d

# With environment file

docker compose --env-file .env.prod -f compose.yaml -f compose.prod.yaml up -d

Scaling Services

# Scale specific service

docker compose up -d --scale worker=5

# Scale multiple services

docker compose up -d --scale worker=5 --scale consumer=3

Conditional Service Activation with Profiles

services:

  web:

    image: nginx

    # Always starts

  debug:

    image: debug-tools

    profiles:

      - debug  # Only starts with --profile debug

  test:

    build: .

    profiles:

      - test   # Only starts with --profile test

# Start with debug profile

docker compose --profile debug up

# Start with multiple profiles

docker compose --profile debug --profile test up

Quick Reference

Essential Commands

# Start and manage

docker compose up -d                    # Start detached

docker compose down                     # Stop and remove

docker compose restart                  # Restart all

docker compose stop                     # Stop without removing

# Build and pull

docker compose build                    # Build all images

docker compose pull                     # Pull all images

docker compose build --no-cache        # Clean build

# View and monitor

docker compose ps                       # List containers

docker compose logs -f                  # Follow logs

docker compose top                      # Running processes

docker compose events                   # Real-time events

# Execute and debug

docker compose exec service sh          # Interactive shell

docker compose run --rm service cmd     # One-off command

File Structure

project/

├── compose.yaml              # Base configuration

├── compose.override.yaml     # Local overrides (auto-loaded)

├── compose.prod.yaml         # Production config

├── compose.staging.yaml      # Staging config

├── .env                      # Default environment

├── .env.prod                 # Production environment

├── services/

│   ├── frontend/

│   │   ├── Dockerfile

│   │   └── src/

│   ├── backend/

│   │   ├── Dockerfile

│   │   └── src/

│   └── worker/

│       ├── Dockerfile

│       └── src/

└── docker/

    ├── nginx/

    │   └── nginx.conf

    └── scripts/

        └── init.sql

Resources

• Docker Compose Documentation: https://docs.docker.com/compose/

• Compose File Specification: https://docs.docker.com/compose/compose-file/

• Docker Hub: https://hub.docker.com/

• Awesome Compose Examples: https://github.com/docker/awesome-compose

• Docker Compose GitHub: https://github.com/docker/compose

• Best Practices Guide: https://docs.docker.com/develop/dev-best-practices/

Skill Version: 1.0.0

Last Updated: October 2025

Skill Category: DevOps, Container Orchestration, Application Deployment

Compatible With: Docker Compose v3.8+, Docker Engine 20.10+