What is Docker Compose? #

Docker Compose is the tool that completes Docker for managing multi-container applications. When your application grows from one service to several interdependent services, running containers one by one with docker run becomes impractical. Docker Compose answers that need with one declarative file that defines the entire application stack.

This article covers the basics of Docker Compose — what it is, how it works, when to use it, and how it differs from the manual docker run approach. Once you understand this, you’ll be ready to dive into the docker-compose.yml file structure and advanced configuration.

The Problem Docker Compose Solves #

In simple applications, one container may be enough. But almost every modern production application consists of many services: web apps, databases, caches, message brokers, reverse proxies, monitoring agents, and more. Managing all of that with docker run on the command line is painful.

Imagine this setup with docker run:

docker network create app-network
docker volume create pg-data
docker run -d --name postgres --network app-network -v pg-data:/var/lib/postgresql/data -e POSTGRES_PASSWORD=secret postgres:16
docker run -d --name redis --network app-network redis:7
docker run -d --name api --network app-network -e DATABASE_URL=postgres://postgres:pass@postgres:5432/myapp -e REDIS_URL=redis://redis:6379 -p 8080:8080 myapi:latest
docker run -d --name web --network app-network -p 3000:3000 myweb:latest

Four command lines, each long, each requiring a specific order, with many parameters to remember. If one fails, you must roll back manually. If a new developer wants to run this stack, they have to jot down all those commands.

With Docker Compose, all of that becomes one file:

# docker-compose.yml
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - pg-data:/var/lib/postgresql/data

  redis:
    image: redis:7

  api:
    image: myapi:latest
    environment:
      DATABASE_URL: postgres://postgres:pass@postgres:5432/myapp
      REDIS_URL: redis://redis:6379
    ports:
      - "8080:8080"
    depends_on:
      - postgres
      - redis

  web:
    image: myweb:latest
    ports:
      - "3000:3000"
    depends_on:
      - api

volumes:
  pg-data:

And to run everything:

docker compose up

One command. The complete application stack stands up in seconds. A new developer just does git clone then docker compose up — done.

Defining Docker Compose #

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with one command, you create and start all the services from that configuration.

Main characteristics:

• A single file as the source of truth — all configuration in one place.
• Declarative — you declare what you want; Compose figures out how.
• Idempotent — running docker compose up several times produces the same end state.
• Lifecycle management — start, stop, restart, scale, logs, all available via a consistent command line.
• Environment-specific — you can have several files (default, development, production) merged as needed.

Docker Compose is an official project from Docker Inc. and is available in all modern Docker installations. Since Docker Compose V2, the tool integrates directly with the Docker CLI (docker compose) — no longer as a separate binary.

A Brief History #

Docker Compose was first introduced in 2014 as a separate project (docker-compose with a dash). Initially it was a Python script communicating with the Docker daemon via the Docker Remote API.

Key milestones:

• 2014 — Compose v1 released as a separate project.
• 2017 — The Compose Specification moved to a Go codebase.
• 2020 — Compose V2 (docker compose with a space) announced, rewritten in Go and integrated with the Docker CLI.
• 2023 — Compose V2 becomes the default; V1 is deprecated.

In Compose V2, docker compose is part of the Docker CLI itself (not a separate binary). This makes the experience more consistent and removes the YAML parsing overhead that existed in V1.

Migration note: If you have old scripts using docker-compose (with a dash), modern Docker installations usually provide it as a wrapper. But for new code, always use docker compose (with a space). The V1 command is deprecated.

How Docker Compose Works #

Docker Compose works in three steps: read the file, create resources, run.

flowchart LR
    A[docker-compose.yml] -->|compose parse| B[In-memory spec]
    B -->|compose convert| C[Container, Network, Volume]
    C -->|compose up| D[Running Stack]
    D -->|compose down| E[Removed Stack]
    A2[.env file] --> A
    A3[override.yml] --> A

1. Parse — docker compose reads the docker-compose.yml file (and additional files like .env or docker-compose.override.yml), then turns it into an internal representation.

2. Convert — the internal representation is translated into concrete Docker resources: containers, networks, volumes, and more.

3. Run — the resources are started. For each service, Compose creates a container with the appropriate configuration. Services with depends_on wait for their dependencies to be ready first.

The docker-compose.yml file doesn’t replace the Dockerfile. The two complement each other:

• Dockerfile defines the image (what runs inside the container).
• docker-compose.yml defines the stack (which containers run, networks, volumes, environment).

flowchart TB
    subgraph Compose["docker-compose.yml"]
        S1[Service 1: which image? port? env?]
        S2[Service 2: which image? port? env?]
        S3[Service 3: which image? port? env?]
    end
    subgraph Dockerfile["Dockerfile"]
        I[Image: OS + runtime + app code]
    end
    S1 --> I

In practice, Compose services can use images from a registry (Postgres, Redis, nginx) or images built locally from a Dockerfile (build: ./path/to/Dockerfile).

Differences from docker run #

docker run and Compose aren’t opponents — they complement each other. docker run is the command for running a single container. Compose is a way to orchestrate many containers at once.

When to use docker run:

• Short experiments (testing one image).
• Production deployments already orchestrated by Kubernetes or another platform.
• One-off scripts.

When to use Docker Compose:

• Multi-container applications.
• Local development environments.
• Small-to-medium production deployments.
• CI/CD pipelines needing to spin environments up/down.
• Demos and prototyping.

Docker Compose Use Cases #

Local Development #

The most common. Developers run the complete stack (app, database, cache, message broker) with one command. Identical to production (or as close as possible), so bugs that appear locally also appear in production.

# Developer clones the repo, then:
docker compose up

The whole stack runs on their laptop. No need to install Postgres, Redis, or other services manually.

Automated Testing #

CI/CD pipelines run tests that need an isolated environment. Docker Compose enables fast setup and automatic cleanup.

# docker-compose.test.yml
services:
  app:
    build: .
    environment:
      DATABASE_URL: postgres://test:pass@db:5432/test
    depends_on:
      - db

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: test

CI script: docker compose -f docker-compose.test.yml up --abort-on-container-exit. The app container runs the tests, then exits automatically. CI reads the exit code.

Demos and Prototyping #

For demos to stakeholders or short experiments, Compose enables fast, reproducible setups. Delete the folder, run docker compose up, and the demo environment stands again.

Small Production Deployments #

For low-to-medium traffic, Compose can be used directly in production. With a reverse proxy (Traefik, Caddy) in front and good images, it’s enough for many use cases.

But for high traffic, multi-host, or high availability, you need a more powerful orchestrator (Kubernetes, Docker Swarm, or managed platforms like ECS/GKE/AKS).

CI/CD Ephemeral Environments #

Some CI/CD platforms (GitLab CI, GitHub Actions with self-hosted runners) support Docker Compose for creating ephemeral environments per branch or per merge request. Containers stand up while the pipeline runs and are removed when it finishes.

Lifecycle Commands #

Docker Compose provides several commands for managing the stack lifecycle.

For specific tasks:

# View logs with follow
docker compose logs -f --tail 100

# Execute in a running container
docker compose exec db psql -U postgres

# Scale a specific service
docker compose up -d --scale worker=3

# Validate docker-compose.yml without running
docker compose config

The docker-compose.yml File Structure #

The docker-compose.yml file is the core of Docker Compose. It uses YAML with a hierarchical structure.

# Schema version (optional in Compose V2, dropped for compatibility)
version: "3.8"

# Service list
services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    depends_on:
      - api

  api:
    build: ./api
    environment:
      DATABASE_URL: postgres://user:pass@db:5432/myapp
    depends_on:
      - db

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - db-data:/var/lib/postgresql/data

# Volumes used
volumes:
  db-data:

# Networks used (if custom needed)
networks:
  backend:

Each service has configuration: image (or build), ports, environment, volumes, dependencies, and more. Top-level keys also include volumes and networks for declaring the resources services use.

The next articles in this section will cover each part in detail: how to define services, use variables, configure networks, healthchecks, and best practices for writing docker-compose.yml.

Docker Compose Limitations #

Powerful as it is, Docker Compose has limitations worth understanding:

• Single host only — Compose runs all services on one host. For multi-host, you need an orchestrator (Kubernetes, Swarm).
• No built-in autoscaling — the --scale flag scales the container count, but there’s no metric-based autoscaling (CPU, traffic).
• No self-healing — if a container crashes, Compose doesn’t automatically restart it (unless restart: always is set).
• No rolling updates — docker compose up recreates containers all at once; no zero-downtime deploys.
• No cross-host service discovery — all services must be reachable on the same network.

For more advanced needs, consider:

• Docker Swarm — Docker’s built-in orchestrator, simple but rarely used in industry.
• Kubernetes — the industry standard, most powerful, most complex.
• AWS ECS / Fargate — AWS’s managed container service.
• Google Cloud Run — serverless containers, auto-scaling to 0.

When Docker Compose Isn’t Needed #

Docker Compose isn’t a universal answer. There are situations where it’s overkill or not the right fit.

• Single-container applications — if your app is truly one service without dependencies, docker run is enough, or even no Docker at all.
• Multi-host production — Compose runs on one host. For multi-host, you need Kubernetes, Swarm, or a managed platform.
• Metric-based autoscaling — Compose has no mechanism to auto-scale based on CPU or traffic. You need Kubernetes HPA or an equivalent platform.
• Rolling updates — Compose recreates containers all at once. For zero-downtime deploys, you need an orchestrator with rolling update support.

When is Compose right? An application stack running on one host, with several interdependent services. That’s Compose’s sweet spot. For larger scale and complexity, move up to Kubernetes or a managed platform.

Migrating from docker run to Compose #

If you already have a docker run-based setup and want to migrate to Compose, the process is easy. Take each docker run command and turn it into a service in YAML.

Migration example:

# docker run command
docker run -d \
  --name postgres \
  -e POSTGRES_USER=app \
  -e POSTGRES_PASSWORD=secret \
  -e POSTGRES_DB=myapp \
  -v pg-data:/var/lib/postgresql/data \
  --network app-net \
  --restart unless-stopped \
  postgres:16-alpine

Becomes a service in Compose:

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - pg-data:/var/lib/postgresql/data
    networks:
      - app-net
    restart: unless-stopped

Mapping docker run flags → Compose fields:

After migrating, compare behavior — run docker compose up and make sure all services work as before. Check exposed ports, mounted volumes, environment variables, and network connectivity between services.

Interactions with the Rest of Docker #

Docker Compose isn’t a standalone tool — it’s part of the Docker ecosystem. Some important interactions:

• docker context — Compose respects the active Docker context. Use docker context use production before docker compose up to deploy to a remote host.
• docker buildx — Build caching and multi-platform builds via buildx can be used by Compose through the builder field (experimental).
• docker scan — Security scanning per image is still done per service. There’s no built-in docker compose scan (per V2 workflow).
• docker system — docker compose doesn’t remove built images. Use docker image prune or docker system prune for cleanup.

Best Practices in Brief #

• One docker-compose.yml file per project — no need for many files except for different environments.
• Use .env for secrets and configuration — don’t hardcode them in YAML.
• Naming conventions — use service names that describe their function, e.g. web, api, db, redis, not container1.
• Explicit image tags — postgres:16-alpine, not postgres:latest. Avoid surprises on update.
• Healthchecks on critical services — so dependency-aware startup works correctly.
• Volumes for persistent data — always, for databases, uploads, and important logs.
• .dockerignore in build contexts — exclude node_modules, .git, etc. so the build context stays small.

Summary #

• Docker Compose is a tool for defining and running multi-container applications via one YAML file (docker-compose.yml).
• The problem it solves — managing many containers with docker run is impractical. Compose brings reproducibility, built-in documentation, and lifecycle management.
• How it works — Compose parses the YAML file → converts it to Docker resources → runs containers with correct dependencies. Since V2, Compose is integrated with the Docker CLI (docker compose).
• The docker-compose.yml file ≠ Dockerfile — Dockerfiles define images, Compose defines stacks. They complement each other.
• Most common use cases — local development, automated testing, demos, and small-to-medium production deployments.
• Lifecycle commands — up, down, ps, logs, exec, build, pull, restart, stop, start, and top.
• Limitations — single host only, no built-in autoscaling/self-healing, no rolling updates. For multi-host and high availability, you need Kubernetes or a managed platform.
• Compose isn’t an orchestrator replacement — for serious production, Compose is enough for dev/test/small prod, but Kubernetes or ECS for larger scale.

← Previous: Best Practice   Next: docker-compose.yml →