An AI application may work on one laptop but fail elsewhere because of different operating-system libraries, Python versions, package builds, environment settings, or startup commands. Docker packages the application and its runtime dependencies into an image that can be tested and deployed consistently.

What Is Docker?

Docker is a platform for building, distributing, and running applications in containers. A container is an isolated process created from a versioned image. It uses the host kernel but has its own filesystem, process space, networking, and resource controls.

A container is not a complete virtual machine. It is generally lighter because it does not package and run a separate guest operating-system kernel.

Why Docker Matters for AI and MLOps

• Captures the runtime, system packages, application code, and startup command.
• Reduces differences between development, continuous integration, staging, and production.
• Creates an immutable artifact that can be scanned, signed, registered, and promoted.
• Simplifies deployment to container services, Kubernetes, batch platforms, and edge systems.
• Supports isolated tests for APIs, workers, training jobs, and evaluation services.
• Makes rollback easier when each release uses a unique image digest.
• Provides a clear unit for resource limits, health checks, logging, and scaling.

Core Concepts

Image vs Container

An image is the packaged blueprint; a container is an execution created from that blueprint. Many containers can run from the same image. Changes written inside a container's writable layer are normally ephemeral, so durable business data should live in an external database, object store, or managed volume.

A Minimal Python API

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class PredictionRequest(BaseModel):
    text: str

@app.get("/health/live")
def live():
    return {"status": "alive"}

@app.post("/predict")
def predict(payload: PredictionRequest):
    # Replace with validated preprocessing and model inference.
    return {"label": "needs_review", "confidence": 0.82}

A Production-Minded Dockerfile

FROM python:3.11-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN groupadd --system app && useradd --system --gid app app

WORKDIR /app

COPY requirements.txt ./
RUN pip install --no-cache-dir --require-hashes -r requirements.txt

COPY --chown=app:app src/ ./src/

USER app
EXPOSE 8000

CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]

This example uses a smaller base, installs dependencies before source code for better build caching, requires locked hashes, runs as a non-root user, and uses the exec form of CMD so signals reach the application process correctly.

Build and Run

docker build -t ai-api:dev .
docker run --rm --name ai-api -p 8000:8000 ai-api:dev

# In another terminal
curl http://localhost:8000/health/live
curl -X POST http://localhost:8000/predict \
  -H 'Content-Type: application/json' \
  -d '{"text":"Customer cannot access the dashboard"}'

Useful Commands

The Build Context and .dockerignore

Docker sends the build context to the builder. Exclude unnecessary, sensitive, and large files so they cannot accidentally enter layers and so builds remain fast.

.git
.venv
__pycache__
*.pyc
.env
.env.*
secrets/
data/
models/
notebooks/
logs/
tests/
README.md
.DS_Store

Adapt this file to the application. Keep tests in the build context when the Docker build itself runs tests, and include models only when the chosen packaging strategy intentionally embeds them.

Layer Caching

Dockerfile instructions create layers. Put stable dependency files before frequently changing source code so unchanged dependencies reuse the cache. Avoid invalidating expensive layers by copying the entire project too early.

WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/

Pin Dependencies

A container build is reproducible only when base images, system packages, language dependencies, and model artifacts are controlled. Use lock files or pinned versions and integrity hashes where supported. Tags such as latest are mutable; production systems should record or deploy by immutable digest.

Multi-Stage Builds

Multi-stage builds use one stage for compilers and build tools and another for the runtime. This can reduce image size and attack surface.

FROM python:3.11-slim AS builder
WORKDIR /build
RUN python -m venv /opt/venv
ENV PATH=/opt/venv/bin:$PATH
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.11-slim AS runtime
ENV PATH=/opt/venv/bin:$PATH \
    PYTHONUNBUFFERED=1
RUN useradd --system --create-home app
WORKDIR /app
COPY --from=builder /opt/venv /opt/venv
COPY --chown=app:app src/ ./src/
USER app
CMD ["python", "-m", "src.main"]

Model Packaging Strategies

Whichever strategy is selected, record the model version and checksum and ensure the serving code, preprocessing, schema, and model are compatible.

Configuration and Secrets

• Keep environment-specific configuration outside the image.
• Inject secrets at runtime from a secret manager or orchestrator integration.
• Never use Dockerfile ENV or build arguments for long-lived secrets; image history and build systems may expose them.
• Validate required configuration at startup and fail with a clear, non-secret error.
• Separate development, staging, and production endpoints and credentials.
• Do not print resolved secrets in startup logs or diagnostics.

Volumes and Persistent Data

Containers should generally be replaceable. Store durable data outside the container writable layer. Volumes are useful for local development and some stateful workloads, while managed databases and object stores are usually better for production ML metadata, datasets, model artifacts, and business records.

docker volume create ai-cache
docker run --rm \
  -v ai-cache:/app/cache \
  ai-batch:dev

Networking

Inside a container, localhost refers to that container, not the host or another container. Use published ports for host access and a Docker network with service names for container-to-container communication.

docker network create ai-network
docker run -d --name feature-db --network ai-network postgres:16
docker run --rm --network ai-network \
  -e DATABASE_HOST=feature-db \
  ai-api:dev

Docker Compose for Local Development

services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      DATABASE_HOST: db
    depends_on:
      db:
        condition: service_healthy
  db:
    image: postgres:16
    environment:
      POSTGRES_DB: app
      POSTGRES_USER: app
      POSTGRES_PASSWORD: local-development-only
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      timeout: 3s
      retries: 10

Compose is useful for repeatable local environments and integration tests. Do not treat example development credentials or configuration as production deployment guidance.

Health Checks

Keep liveness checks simple. A temporary database outage should not necessarily cause every application container to restart. Readiness can reflect whether the instance has loaded its model and is ready to serve.

Graceful Shutdown

Container platforms send termination signals during deployment and scaling. The application should stop accepting new work, complete or safely checkpoint current requests, release resources, and exit before the grace period ends. Use exec-form ENTRYPOINT or CMD so signals reach the application.

Logging and Metrics

• Write structured application logs to standard output and error.
• Include request or trace ID, model version, endpoint, status, latency, and error category.
• Do not log secrets, full authorization headers, unnecessary prompts, or sensitive features.
• Expose application metrics such as prediction latency, errors, queue depth, and model-load status.
• Let the container platform collect, retain, and route logs rather than writing unbounded local files.
• Include image digest and release version in deployment metadata.

CPU, Memory, and GPU Resources

AI workloads can consume large and uneven resources. Measure startup memory, steady-state memory, request peaks, model concurrency, CPU threads, and GPU memory. Set realistic requests and limits in the runtime platform and load test under representative traffic.

• Avoid loading one model copy per worker without accounting for memory multiplication.
• Limit parallel inference to prevent memory exhaustion and latency collapse.
• Match accelerator drivers, runtime, libraries, and model format.
• Monitor throttling, out-of-memory termination, saturation, and cold-start time.
• Separate CPU preprocessing from GPU inference when architecture and scale justify it.

Container Security

• Choose maintained minimal base images from trusted sources.
• Pin base images and dependencies and rebuild regularly for security updates.
• Run as a non-root user and drop unnecessary Linux capabilities.
• Use a read-only root filesystem and writable mounts only where needed.
• Do not mount the Docker socket into untrusted containers.
• Scan images and dependencies for vulnerabilities and secrets.
• Generate and retain a software bill of materials when required.
• Sign or attest images and verify provenance before deployment.
• Restrict network access and runtime permissions using least privilege.

Supply-Chain Security

A container includes code from base images, operating-system packages, language libraries, build tools, and downloaded artifacts. Build in controlled CI, review Dockerfile and dependency changes, verify checksums, avoid untrusted installation scripts, and promote only scanned immutable images from an approved registry.

Testing Container Images

• Build the image in continuous integration from a clean context.
• Run unit tests and static checks before packaging.
• Start the container and test health, prediction, invalid-input, and shutdown behavior.
• Verify the process runs without root and without unexpected writable paths.
• Scan vulnerabilities, secrets, licenses, and image contents.
• Test model and feature compatibility and record evaluation evidence.
• Load test latency, throughput, concurrency, memory, and cold starts.
• Deploy the same immutable image to staging before production.

CI/CD Workflow

Reviewed Git commit
-> Unit, integration, and AI evaluation tests
-> Build image in controlled CI
-> Generate metadata and software bill of materials
-> Scan image and dependencies
-> Push immutable digest to registry
-> Deploy same digest to staging
-> Smoke, compatibility, and load checks
-> Approve and progressively deploy to production
-> Monitor and roll back by digest when required

Image Tagging

Human-readable tags are useful but mutable. Use tags such as application version and Git commit for discovery, and record the immutable digest for deployment and audit.

docker build \
  -t registry.example.com/ai-api:1.4.0 \
  -t registry.example.com/ai-api:git-a1b2c3d \
  .
docker push registry.example.com/ai-api:1.4.0
docker push registry.example.com/ai-api:git-a1b2c3d

Docker in the MLOps Lifecycle

Versioned source and environment
-> Reproducible tests and training
-> Approved model artifact
-> Build serving image
-> Scan, sign, and push registry
-> Staged deployment by digest
-> Monitor service, model, data, and cost
-> Promote, roll back, rebuild, or retire

Common Problems

Common Mistakes

Practical Exercise

• Create a small FastAPI prediction service with request validation.
• Write pinned dependencies and a non-root Dockerfile.
• Add a .dockerignore file and inspect the image layers and size.
• Build and run the image with runtime configuration.
• Test liveness, readiness, valid prediction, invalid input, logs, and graceful shutdown.
• Scan the image and record its Git commit, model version, tag, and digest.
• Run the API and a database locally with Compose.
• Document build, run, test, security, and troubleshooting instructions.

Best Practices

• Use trusted minimal bases and pinned dependencies.
• Make builds cache-efficient, deterministic, and free of unnecessary context.
• Keep environment configuration and secrets outside the image.
• Run as non-root with minimal runtime capabilities and writable paths.
• Choose and document how model artifacts are versioned and loaded.
• Implement health checks, graceful shutdown, structured logs, metrics, and resource controls.
• Scan, attest, register, and promote immutable images through environments.
• Test architecture, performance, compatibility, security, and rollback before production.