Engine & Daemon #

Behind the simplicity of docker run and docker build is an internal architecture more layered than it looks. The two most commonly used terms — Docker Engine and Docker Daemon — are often treated as the same thing, when they actually cover different scopes. Understanding this difference isn’t just academic knowledge; it determines whether you can diagnose problems correctly when a container won’t start, secure access to Docker on a production server, and choose a deployment strategy that fits your environment.

This article dissects Docker Engine and Docker Daemon in depth. We’ll start with an architecture map, then move into the daemon’s specific role, the Engine’s anatomy, its relationship with containerd and runc, and finally the security aspects that are often overlooked. After reading this article, you’ll know not just what runs, but why it runs and how to control it.

Client-Server Architecture: The Communication Map #

Docker is fundamentally a client-server application. This understanding is a prerequisite for everything that follows, so let’s start here.

flowchart LR
    subgraph CLIENTS["Clients (can be on different machines)"]
        CLI["docker CLI<br/>(binary)"]
        SDK1[Python SDK]
        SDK2[Go SDK]
        SDK3[Node SDK]
        COMP[Docker Compose]
    end

    subgraph SERVER["Server (Docker host)"]
        DAEMON["dockerd<br/>(Docker Daemon)"]
        subgraph RUNTIME["Container Runtime"]
            CONTAINERD["containerd<br/>(high-level)"]
            RUNC["runc<br/>(low-level, OCI)"]
        end
    end

    CLI -->|"REST API<br/>unix:///var/run/docker.sock<br/>or tcp://host:2375"| DAEMON
    SDK1 --> DAEMON
    SDK2 --> DAEMON
    SDK3 --> DAEMON
    COMP --> DAEMON
    DAEMON --> CONTAINERD --> RUNC
    RUNC -->|"syscall"| KERNEL[Linux Kernel]

The client is what you touch. The docker CLI is the most common client — a binary that parses commands, builds JSON requests, and sends them over HTTP to the daemon. Docker Compose, the Docker SDKs for Python/Go/Node, and third-party tools (ctop, dive, lazydocker) are also clients — all of them talk to the daemon through the same REST API.

The server is the daemon itself — the dockerd process running continuously on the host. The daemon listens on a socket (default: /var/run/docker.sock on Linux), receives requests, and executes them. The daemon never touches hardware directly — all container execution is delegated to the runtime.

This separation has important consequences. First, the client and daemon can be on different machines. You can run docker on your laptop to control a daemon on a remote server, as long as the TCP socket is enabled and secured. Second, the client has no logic — all decisions happen in the daemon. That’s why two different clients (CLI and Compose) sending commands at the same time won’t clash: the daemon serializes all requests.

Interesting detail: Docker’s REST API is fully open and documented. Everything the CLI can do can be done with curl to the same socket. That’s also why debugging tools like curl --unix-socket /var/run/docker.sock http://localhost/containers/json work — it’s an ordinary HTTP request, not magic.

Docker Daemon (dockerd) — The Brain Behind the Scenes #

The Docker Daemon is the background process that runs continuously on the host and is responsible for almost everything container-related. Its process name is dockerd.

Main Responsibilities #

The daemon isn’t just “a program that runs containers”. It manages all of Docker’s state on the host:

Every time you run docker ps, it’s the daemon querying its internal state. Every docker run has the daemon orchestrating image pulls, container creation, network setup, and startup. Without the daemon, Docker can’t do anything — a CLI without a daemon just errors with “Cannot connect to the Docker daemon”.

Where and How the Daemon Runs on Different OSes #

How the daemon runs differs across operating systems, and this is often a source of confusion.

On Linux, the daemon runs as a systemd service. This is the “purest” setup because the daemon genuinely runs on the host, talks directly to the Linux kernel, and has no extra virtualization layer.

# Check daemon status
systemctl status docker

# See the daemon process
ps aux | grep dockerd

# See the daemon configuration
cat /etc/docker/daemon.json

On macOS and Windows, the daemon doesn’t run natively on the host because the host isn’t Linux. Instead, Docker Desktop runs a lightweight Linux VM (based on Apple’s Virtualization framework on Mac, or Hyper-V/WSL2 on Windows), then runs dockerd inside that VM. The CLI on the host talks to the daemon through a forwarded socket.

# On macOS, the Docker socket appears at
/var/run/docker.sock
# but it's actually forwarded from the VM

This explains several behaviors that often confuse people:

• I/O performance on Mac/Windows is slower than Linux because of the VM layer. The solution: use the virtiofs/WSL2 backend, or named volumes (which live in the VM, not the host filesystem).
• Path binding on Mac/Windows needs Mac/Windows paths, not Linux paths. Docker Desktop translates them.
• Resource limits (--memory, --cpus) on Mac/Windows limit the VM’s resources, not the host’s — because the containers genuinely run in the VM.

A way to remember: every Docker container always runs on a Linux kernel. It doesn’t matter whether your host is Mac, Windows, or Linux — the kernel executing container processes is always Linux. That’s why Linux images run anywhere, and why Windows images can only run on Windows hosts.

Docker Engine — The Complete Platform #

Docker Engine is a broader term than dockerd. It’s the complete platform for building and running containers, and the daemon is just one component inside it.

Docker Engine Anatomy #

flowchart TB
    subgraph ENGINE["Docker Engine"]
        subgraph INTERFACE["Interface Layer"]
            CLI["docker CLI<br/>(user commands)"]
            API["REST API<br/>(HTTP endpoints)"]
        end
        subgraph CORE["Core Layer"]
            DAEMON["dockerd<br/>(state management)"]
        end
        subgraph RUNTIME["Runtime Layer"]
            CONTAINERD["containerd<br/>(image & container lifecycle)"]
            RUNC["runc<br/>(OCI runtime)"]
        end
    end

    CLI --> API
    API --> DAEMON
    DAEMON --> CONTAINERD
    CONTAINERD --> RUNC
    RUNC -->|"syscall"| KERNEL[Linux Kernel]

The four components that make up Docker Engine:

1. docker CLI — the binary you invoke in your terminal.
2. REST API — the communication protocol between clients and the daemon.
3. dockerd — the daemon managing state and delegating execution.
4. Container Runtime — containerd + runc, which actually create and run containers.

Docker Engine vs Docker Daemon — What’s the Difference? #

A way to remember it: Docker Daemon is part of Docker Engine, not the other way around. Docker Engine is the product concept; Docker Daemon is one of its implementations.

containerd and runc — The Execution Layer #

This is the part that confuses Docker beginners the most: if the daemon manages everything, then what do containerd and runc do? Why not just have the daemon run containers directly?

A Brief History: Why They Were Split #

Before Docker 1.11 (released April 2016), all runtime logic — from image extraction to namespace creation — lived inside dockerd. This made the daemon big, monolithic, and fragile: a runtime bug could crash the daemon, and the daemon couldn’t be restarted gracefully because that would kill every running container.

The solution: refactor into three layers.

flowchart LR
    A["Docker CLI"] -->|"REST API"| B["dockerd<br/>(daemon)"]
    B -->|"gRPC"| C["containerd<br/>(supervisor)"]
    C -->|"fork+exec"| D["runc<br/>(OCI runtime)"]
    D -->|"syscall"| E[Linux Kernel]

    style B fill:#e1f5ff
    style C fill:#fff4e1
    style D fill:#ffe1e1

containerd is the supervisor for containers. It manages:

• Image pulls and extraction
• Storage management (snapshots, layers)
• Container lifecycle (low-level, not just metadata)
• Network attachment (working with CNI plugins)
• Volume attachment

containerd runs as a separate process from dockerd. They communicate over gRPC on an internal Unix socket. If dockerd is restarted, containerd keeps running and already-started containers are unaffected.

runc is the low-level runtime that follows the OCI (Open Container Initiative) standard. It actually creates the container process by calling the clone() system call with namespace flags, setting up cgroups, then execve()-ing the container’s main binary. runc does one thing extremely well: turning a container specification (an OCI bundle) into a running process.

Interesting detail: runc is the reference implementation of the OCI Runtime Specification. Other projects compliant with the same standard include crun (written in C, lighter than the Go-written runc) and youki (written in Rust). K8s often uses crun in certain RuntimeClasses for startup optimization.

The Complete Flow: From CLI to a Live Container #

Let’s see what happens when you run docker run -d nginx:

sequenceDiagram
    participant CLI as docker CLI
    participant D as dockerd
    participant CD as containerd
    participant R as runc
    participant K as Kernel

    CLI->>D: POST /containers/create<br/>(name, image, config)
    D->>D: validate, generate ID
    D->>D: set up network, volumes
    D->>CD: CreateContainer (gRPC)
    CD->>CD: image pull (if needed)
    CD->>CD: snapshot writable layer
    CD->>CD: set up cgroup, namespace spec
    CD->>R: fork+exec runc
    R->>K: clone(CLONE_NEWNS|NEWPID|NEWNET|...)
    K-->>R: new namespace
    R->>K: execve(/nginx)
    K-->>R: nginx process started
    R-->>CD: container started
    CD-->>D: success
    D-->>CLI: container ID + status

Notice that dockerd and containerd communicate via gRPC (not HTTP), and runc is invoked via fork+exec (a new process). This is real process isolation — if runc crashes, only the container is affected; the daemon and containerd stay healthy.

The REST API — A Language All Clients Understand #

Docker’s REST API is the foundation that makes the Docker ecosystem extensible. Every endpoint and every request field is openly documented at docs.docker.com.

The Most Frequently Used Endpoints #

Accessing the API Directly #

Because this API is plain HTTP, you can call it with curl (or any HTTP library) directly against the daemon’s Unix socket.

# List all running containers
curl --unix-socket /var/run/docker.sock \
  http://localhost/containers/json

# Watch real-time events
curl --unix-socket /var/run/docker.sock \
  http://localhost/events?stream=true

Note that the hostname in the URL (http://localhost/) is actually ignored when you use --unix-socket — what matters is the Unix path. That’s a curl feature, not Docker’s.

Accessing the Daemon from Another Machine #

To control the daemon from another machine, you need to enable TCP socket listening on the server side and add authentication.

// /etc/docker/daemon.json on the server
{
  "hosts": ["tcp://0.0.0.0:2376"],
  "tlsverify": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlscert": "/etc/docker/server-cert.pem",
  "tlskey": "/etc/docker/server-key.pem"
}

On the client side:

docker -H tcp://server.example.com:2376 ps
# or
docker context create remote --docker "host=tcp://server.example.com:2376,ca=...,cert=...,key=..."
docker context use remote
docker ps

Never expose Docker’s port (2375/2376) without TLS to the public internet. This port grants root-equivalent access to the host. Many bots on the internet actively scan port 2375 to exploit unsecured Docker hosts. If you must expose it, use mutual TLS, a firewall, and a VPN.

Docker Daemon Security — a Critical Attack Surface #

Access to the Docker daemon equals root access to the host. That’s not hyperbole — with access to dockerd, an attacker can:

• Run containers with --privileged mode, granting nearly unlimited access to the host.
• Mount the host filesystem (-v /:/host) into a container, then read/modify any file.
• Install new software, create new users, or even boot into another system.
• Bypass most traditional security controls.

Mandatory Security Principles #

MUST do:
  ✓ Restrict access to /var/run/docker.sock — only users
    in the `docker` group may access it.
  ✓ If exposing via TCP, use mutual TLS (mTLS).
  ✓ Audit the daemon configuration regularly
    (/etc/docker/daemon.json).
  ✓ Enable user namespace remapping to constrain
    container UIDs to a non-root range on the host.
  ✓ Consider rootless mode for environments that need
    minimum privileges.

MUST NOT do:
  ✗ Expose ports 2375/2376 without TLS to a public network.
  ✗ Run containers with --privileged unless
    genuinely necessary.
  ✗ Mount / or /etc from the host into a container.
  ✗ Add just anyone to the `docker` group —
    that's equivalent to handing out a root shell.

Auditing the Daemon Configuration #

The /etc/docker/daemon.json file is the central configuration for the daemon. Some settings worth paying attention to:

{
  "log-level": "info",
  "live-restore": true,
  "userland-proxy": false,
  "no-new-privileges": true,
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  },
  "storage-driver": "overlay2",
  "features": {
    "containerd-snapshotter": true
  }
}

no-new-privileges: true is an important security setting: it prevents containers from gaining new privileges through setuid binaries. live-restore: true lets containers keep running while the daemon restarts — very useful for maintenance. containerd-snapshotter: true enables the containerd-based snapshotter, which is faster and more storage-efficient.

Rootless Mode — Docker Without Root #

One of Docker’s security breakthroughs in recent years is rootless mode. In this mode, the daemon and containers run as an ordinary user without root privileges, using kernel user namespaces.

flowchart TB
    subgraph TRAD["Traditional (root daemon)"]
        ROOT1["dockerd (root)"] --> K1[Kernel]
        K1 --> HOST1[Host filesystem]
    end
    subgraph ROOTLESS["Rootless Mode"]
        USER["user (UID 1000)"]
        DAEMON_R["dockerd (rootless)"]
        USER --> DAEMON_R
        DAEMON_R -->|"unshare(NEWUSER)"| K2[Kernel]
        K2 --> HOST2[Host filesystem]
    end

Rootless Advantages #

• Reduces the attack blast radius. If a container escapes, the attacker only gets ordinary user access, not root.
• Can run on shared hosts without worrying that one compromised container will take over the whole machine.
• Safer CI/CD setups because runners don’t need root access.

Rootless Limitations #

• Ports < 1024 can’t be bound directly (needs extra setup).
• Some features like --privileged, certain network drivers, and some volume mounts don’t work.
• cgroup v1 is limited; better to use cgroup v2 with a modern kernel.
• Performance is slightly lower due to user namespace mapping overhead.

Rootless mode suits development, CI/CD runners, and shared environments where security matters more than a full feature set. For dedicated production servers, the traditional mode with proper hardening is still more common.

When to consider rootless: if your team runs Docker on laptops also used for other things, or on shared CI runners, rootless provides a significant extra security layer at a low setup cost. Docker supports rootless automatically via the dockerd-rootless-setuptool.sh sub-command included in the docker-ce-rootless-extras package.

Decision Tree — Docker Operating Mode for Your Needs #

Not every Docker setup needs the same configuration. Use this decision tree to pick the configuration that fits your needs.

flowchart TD
    A{Docker<br/>environment?}
    A -- Dev laptop --> B{Need<br/>ports < 1024?}
    B -- Yes --> C[Rootless<br/>+ port forwarding]
    B -- No --> D[Rootless default]
    A -- Production server --> E{Dedicated<br/>or shared host?}
    E -- Dedicated --> F[Root daemon<br/>+ hardening]
    E -- Shared --> G[Rootless<br/>or unshare per user]
    A -- CI/CD runner --> H[Rootless<br/>or container-in-container]

    F --> FI{Remote<br/>access?}
    FI -- Yes --> J[TLS mTLS<br/>+ firewall]
    FI -- No --> K[Unix socket<br/>+ docker group]

Understanding Engine and Daemon isn’t the end of everything — it’s the foundation for all the configurations above. Once you know why each setting exists, you can make the right decisions for your case.

Summary #

• Docker Engine = the complete platform (CLI + API + daemon + runtime). Docker Daemon = just the dockerd process. Confusing these often aims security configuration at the wrong target.
• Docker’s client-server architecture explains almost every behavior you see: why the CLI can be on another machine, why third-party tools can inspect containers, and why remote management works.
• The modern container runtime consists of containerd (supervisor) + runc (low-level OCI). The Docker Daemon delegates execution to them; it no longer runs containers itself. This split brings stability and industry standards.
• Docker’s REST API is plain HTTP on a Unix socket. All clients (CLI, Compose, SDKs) talk to the same endpoints. This is what makes the Docker ecosystem easy to extend.
• Access to the Docker Daemon = root access to the host — this is a security principle that can’t be compromised. Always protect the socket with permissions, and if exposing via TCP, use mutual TLS.
• Rootless mode is the right choice for development, CI runners, and shared environments. It provides minimum privileges at the cost of limited features.
• Daemon configuration via /etc/docker/daemon.json is the central way to enable security features like no-new-privileges, live-restore, and user namespace remapping.

← Previous: Overview   Next: Image →