DevFW-CICD/website-and-documentation
23
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added physical-envs/docker
Some checks failed
Hugo Site Tests / test (push) Failing after 1s
Details
ci / build (push) Successful in 55s
Details

Browse source
This commit is contained in:
Manuel Ganter 2025-12-16 11:41:16 +01:00
parent bbbb40d178
commit d39ffeb08a
No known key found for this signature in database
1 changed files with 801 additions and 60 deletions
Download patch file Download diff file Expand all files Collapse all files

861
content/en/docs/components/physical-envs/docker.md
View file

@ -6,123 +6,864 @@ description: >
Container runtime for running containerized applications
---
{{% alert title="Draft" color="warning" %}}
**Editorial Status**: This page is currently being developed.
* **Jira Ticket**: [TICKET-XXX](https://your-jira/browse/TICKET-XXX)
* **Assignee**: [Name or Team]
* **Status**: Draft
* **Last Updated**: YYYY-MM-DD
* **TODO**:
* [ ] Add detailed component description
* [ ] Include usage examples and code samples
* [ ] Add architecture diagrams
* [ ] Review and finalize content
{{% /alert %}}
## Overview
[Detailed description of the component - what it is, what it does, and why it exists]
Docker is a container platform that packages applications and dependencies into standardized units called containers. In the Edge Developer Platform, Docker serves three primary functions: powering local development environments through Docker Desktop, building container images in CI/CD pipelines, and providing Docker-in-Docker (DinD) execution environments for Forgejo Actions runners.
Docker provides a consistent runtime environment across development, testing, and production, ensuring applications behave identically regardless of the underlying infrastructure.
## Key Features
* [Feature 1]
* [Feature 2]
* [Feature 3]
* **Container Runtime**: Execute isolated application containers with process, network, and filesystem isolation
* **Image Building**: Create container images using Dockerfile specifications and layer caching
* **Docker-in-Docker**: Nested Docker execution for CI/CD runners and containerized build environments
* **Multi-stage Builds**: Optimize image size through efficient build processes
* **Volume Management**: Persistent data storage with bind mounts and named volumes
* **Network Isolation**: Software-defined networking with bridge, host, and overlay networks
* **Resource Control**: CPU, memory, and I/O limits through Linux cgroups
## Purpose in EDP
[Explain the role this component plays in the Edge Developer Platform and how it contributes to the overall platform capabilities]
Docker plays several critical roles in the Edge Developer Platform:
**Local Development**: Docker Desktop provides developers with containerized development environments, enabling consistent tooling and dependencies across different machines. Developers can run entire application stacks locally using docker-compose configurations.
**Image Building**: CI/CD pipelines use Docker to build container images from source code. The `docker build` command transforms Dockerfiles into layered images that can be deployed to Kubernetes or other container orchestration platforms.
**CI/CD Execution**: Forgejo Actions runners use Docker-in-Docker to execute workflow steps in isolated containers. Each job runs in a fresh container environment, ensuring clean state and reproducible builds.
## Repository
**Code**: [Link to source code repository]
**Code**: Docker is an open-source project
**Documentation**: [Link to component-specific documentation]
**Documentation**:
* [Docker Official Documentation](https://docs.docker.com/)
* [Containerd Documentation](https://containerd.io/docs/)
* [OCI Runtime Specification](https://github.com/opencontainers/runtime-spec)
* [OverlayFS Documentation](https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html)
## Getting Started
### Prerequisites
* [Prerequisite 1]
* [Prerequisite 2]
* Linux kernel 3.10+ with namespace and cgroup support
* 64-bit processor architecture (x86_64, ARM64)
* At least 4GB RAM for Docker Desktop
* 20GB available disk space for images and containers
### Quick Start
[Step-by-step guide to get started with this component]
**Install Docker on Linux**:
1. [Step 1]
2. [Step 2]
3. [Step 3]
```bash
# Install Docker using official script
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Add user to docker group (optional, allows running docker without sudo)
sudo usermod -aG docker $USER
# Start Docker daemon
sudo systemctl start docker
sudo systemctl enable docker
```
**Install Docker Desktop** (macOS/Windows):
1. Download Docker Desktop from [docker.com](https://www.docker.com/products/docker-desktop)
2. Run the installer
3. Launch Docker Desktop from Applications
4. Verify installation in system tray/menu bar
### Verification
[How to verify the component is working correctly]
Verify Docker installation:
```bash
# Check Docker version
docker --version
# Verify daemon is running
docker info
# Run test container
docker run hello-world
# Check running containers
docker ps
# View all containers (including stopped)
docker ps -a
# List downloaded images
docker images
```
## Usage Examples
### [Use Case 1]
### Running Containers
[Example with code/commands showing common use case]
**Basic Container Execution**:
```bash
# Example commands
# Run container in foreground
docker run ubuntu:22.04 echo "Hello from container"
# Run container in background (detached)
docker run -d --name nginx-server nginx:latest
# Run interactive container with shell
docker run -it ubuntu:22.04 /bin/bash
# Run container with port mapping
docker run -d -p 8080:80 nginx:latest
# Run container with volume mount
docker run -d -v /host/data:/container/data ubuntu:22.04
# Run container with environment variables
docker run -d -e POSTGRES_PASSWORD=secret postgres:15
# Run container with resource limits
docker run -d --memory=512m --cpus=1.5 nginx:latest
```
### [Use Case 2]
**Container Management**:
[Another common scenario]
```bash
# Stop container
docker stop nginx-server
## Integration Points
# Start stopped container
docker start nginx-server
* **[Component A]**: [How it integrates]
* **[Component B]**: [How it integrates]
* **[Component C]**: [How it integrates]
# Restart container
docker restart nginx-server
# View container logs
docker logs nginx-server
docker logs -f nginx-server # Follow logs
# Execute command in running container
docker exec -it nginx-server /bin/bash
# Copy files between host and container
docker cp myfile.txt nginx-server:/tmp/
docker cp nginx-server:/tmp/myfile.txt ./
# Inspect container details
docker inspect nginx-server
# View container resource usage
docker stats nginx-server
# Remove container
docker rm nginx-server
docker rm -f nginx-server # Force remove running container
```
### Building Images
**Simple Dockerfile**:
```dockerfile
# Dockerfile
FROM ubuntu:22.04
# Install dependencies
RUN apt-get update && apt-get install -y \
curl \
vim \
&& rm -rf /var/lib/apt/lists/*
# Set working directory
WORKDIR /app
# Copy application files
COPY . /app
# Set environment variable
ENV APP_ENV=production
# Expose port
EXPOSE 8080
# Define entrypoint
CMD ["./start.sh"]
```
**Build and Tag Image**:
```bash
# Build image from Dockerfile
docker build -t myapp:1.0 .
# Build with build arguments
docker build --build-arg VERSION=1.0 -t myapp:1.0 .
# Build without cache
docker build --no-cache -t myapp:1.0 .
# Tag image for registry
docker tag myapp:1.0 registry.example.com/myapp:1.0
# Push to registry
docker push registry.example.com/myapp:1.0
# Pull from registry
docker pull registry.example.com/myapp:1.0
```
**Multi-stage Build**:
```dockerfile
# Dockerfile with multi-stage build
FROM golang:1.21 AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /app/server
# Final stage
FROM alpine:3.19
RUN apk --no-cache add ca-certificates
COPY --from=builder /app/server /usr/local/bin/server
USER nobody
EXPOSE 8080
CMD ["server"]
```
Multi-stage builds reduce final image size by excluding build tools and intermediate artifacts.
### Docker-in-Docker for CI/CD
**DinD Container for Building Images**:
```bash
# Start DinD daemon
docker run -d \
--name dind \
--privileged \
-e DOCKER_TLS_CERTDIR=/certs \
-v docker-certs:/certs \
docker:28.0.4-dind
# Run build container connected to DinD
docker run --rm \
-e DOCKER_HOST=tcp://dind:2376 \
-e DOCKER_TLS_VERIFY=1 \
-e DOCKER_CERT_PATH=/certs/client \
-v docker-certs:/certs:ro \
-v $(pwd):/workspace \
-w /workspace \
--link dind:dind \
docker:28.0.4-cli \
docker build -t myapp:latest .
```
**Kubernetes DinD Sidecar (Forgejo Runner Pattern)**:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: forgejo-runner
spec:
containers:
- name: runner
image: code.forgejo.org/forgejo/runner:6.4.0
env:
- name: DOCKER_HOST
value: tcp://localhost:2376
- name: DOCKER_TLS_VERIFY
value: "1"
- name: DOCKER_CERT_PATH
value: /certs/client
volumeMounts:
- name: docker-certs
mountPath: /certs
readOnly: true
- name: runner-data
mountPath: /data
- name: dind
image: docker:28.0.4-dind
securityContext:
privileged: true
env:
- name: DOCKER_TLS_CERTDIR
value: /certs
volumeMounts:
- name: docker-certs
mountPath: /certs
- name: docker-storage
mountPath: /var/lib/docker
volumes:
- name: docker-certs
emptyDir: {}
- name: runner-data
emptyDir: {}
- name: docker-storage
emptyDir: {}
```
This configuration runs a Forgejo Actions runner with a DinD sidecar, enabling containerized builds within Kubernetes pods.
### Local Development with Docker Compose
**Docker Compose Configuration**:
```yaml
# docker-compose.yml
version: '3.8'
services:
app:
build: .
ports:
- "8080:8080"
environment:
- DATABASE_URL=postgresql://user:pass@db:5432/appdb
depends_on:
- db
- redis
volumes:
- ./src:/app/src
db:
image: postgres:15
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=appdb
volumes:
- postgres-data:/var/lib/postgresql/data
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
postgres-data:
```
**Compose Commands**:
```bash
# Start all services
docker-compose up -d
# View logs
docker-compose logs -f
# Stop services
docker-compose down
# Rebuild and restart
docker-compose up -d --build
# Run command in service
docker-compose exec app /bin/bash
# Scale service
docker-compose up -d --scale app=3
```
### Image Management
**Image Operations**:
```bash
# List images
docker images
# Remove image
docker rmi myapp:1.0
# Remove unused images
docker image prune
# Remove all unused images
docker image prune -a
# Inspect image layers
docker history myapp:1.0
```
**Registry Operations**:
```bash
# Login to registry
docker login registry.example.com
# Login with credentials
echo $PASSWORD | docker login -u $USERNAME --password-stdin registry.example.com
# Push image
docker push registry.example.com/myapp:1.0
# Pull image
docker pull registry.example.com/myapp:1.0
# Search registry
docker search nginx
```
## Architecture
[Optional: Add architectural diagrams and descriptions]
### Docker Architecture Overview
### Component Architecture (C4)
Docker follows a client-server architecture with several key components:
[Add C4 Container or Component diagrams showing the internal structure]
**Docker Client**: Command-line tool (docker CLI) that sends API requests to the Docker daemon. Developers interact with Docker through client commands like `docker run`, `docker build`, and `docker ps`.
### Sequence Diagrams
**Docker Daemon (dockerd)**: Background service that manages containers, images, networks, and volumes. The daemon listens for Docker API requests and coordinates with lower-level runtime components.
[Add sequence diagrams showing key interaction flows with other components]
**Containerd**: High-level container runtime that manages container lifecycle operations. Containerd handles image transfer and storage, container execution supervision, and low-level storage and network attachments.
### Deployment Architecture
**runc**: Low-level OCI (Open Container Initiative) runtime that creates and runs containers. Runc interfaces directly with Linux kernel features to spawn and manage container processes.
[Add infrastructure and deployment diagrams showing how the component is deployed]
### Container Runtime Layers
Docker's layered architecture separates concerns across multiple components:
```
┌─────────────────────────────────┐
│ Docker Client (CLI) │
│ docker run, build, push │
└────────────┬────────────────────┘
│ Docker API
┌────────────▼────────────────────┐
│ Docker Daemon (dockerd) │
│ API, Image management, Auth │
└────────────┬────────────────────┘
│ Container API
┌────────────▼────────────────────┐
│ Containerd │
│ Lifecycle, Image pull/push │
└────────────┬────────────────────┘
│ OCI Runtime API
┌────────────▼────────────────────┐
│ runc │
│ Namespace, cgroup creation │
└────────────┬────────────────────┘
│ System Calls
┌────────────▼────────────────────┐
│ Linux Kernel │
│ namespaces, cgroups, seccomp │
└─────────────────────────────────┘
```
**Docker Daemon Layer**: Handles high-level operations like image building, authentication, and API endpoint management. The daemon translates user commands into lower-level runtime operations.
**Containerd Layer**: Manages container lifecycle independent of Docker-specific features. Containerd can be used directly by Kubernetes and other orchestrators, providing a standard container runtime interface.
**runc Layer**: Implements the OCI runtime specification, creating container processes with Linux kernel isolation features. Runc configures namespaces, cgroups, and security policies before executing container entrypoints.
### Linux Kernel Features
Docker leverages several Linux kernel capabilities for container isolation:
**Namespaces**: Provide process isolation by creating separate views of system resources:
- **PID namespace**: Isolates process IDs so containers see only their own processes
- **Network namespace**: Provides separate network stacks with unique IP addresses and routing tables
- **Mount namespace**: Isolates filesystem mount points so containers have independent filesystem views
- **UTS namespace**: Separates hostname and domain name
- **IPC namespace**: Isolates inter-process communication resources like shared memory
- **User namespace**: Maps container user IDs to different host user IDs for privilege separation
**Control Groups (cgroups)**: Limit and account for resource usage:
- CPU allocation and throttling
- Memory limits and swap control
- Block I/O bandwidth limits
- Network bandwidth control (via tc integration)
**Capabilities**: Fine-grained privilege control that breaks down root privileges into discrete capabilities. Containers run with reduced capability sets, dropping dangerous privileges like CAP_SYS_ADMIN.
**Seccomp**: Filters system calls that containerized processes can make, reducing the kernel attack surface. Docker applies a default seccomp profile blocking ~44 dangerous syscalls.
**AppArmor/SELinux**: Mandatory access control systems that enforce security policies on container processes, restricting file access and operations.
### Image Storage and OverlayFS
Docker uses storage drivers to manage image layers and container filesystems. The preferred storage driver is overlay2, which uses OverlayFS:
**Image Layers**: Docker images consist of read-only layers stacked on top of each other. Each Dockerfile instruction creates a new layer. Layers are shared between images, reducing storage requirements.
**Copy-on-Write (CoW)**: When a container modifies a file from an image layer, OverlayFS copies the file to the container's writable layer. The original image layer remains unchanged, enabling efficient image reuse.
**OverlayFS Structure**:
```
Container Writable Layer (upperdir)
│
├─ Modified files
│
Image Layers (lowerdir)
│
├─ Layer 3 (READ-ONLY)
├─ Layer 2 (READ-ONLY)
└─ Layer 1 (READ-ONLY)
│
Merged View (merged)
│
└─ Union of all layers
```
**LowerDir**: Read-only image layers containing base filesystem and application files
**UpperDir**: Writable container layer where all changes are stored
**MergedDir**: Union mount that presents a unified view of all layers to the container
**WorkDir**: Internal working directory used by OverlayFS for atomic operations
When a container reads a file, OverlayFS serves it from the topmost layer where it exists. Writes always go to the upperdir, leaving image layers immutable.
### Networking Architecture
Docker provides several networking modes:
**Bridge Network**: Default network mode that creates a virtual bridge (docker0) on the host. Containers connect to the bridge and receive private IP addresses. Network Address Translation (NAT) enables outbound connectivity.
**Host Network**: Container shares the host's network namespace, using the host's IP address directly. Offers maximum network performance but reduces isolation.
**Overlay Network**: Multi-host networking for container communication across different Docker hosts. Used by Docker Swarm and can be integrated with Kubernetes.
**None**: Disables networking for maximum isolation.
### Docker-in-Docker (DinD) Architecture
Docker-in-Docker runs a nested Docker daemon inside a container. This is used in CI/CD runners to provide isolated build environments:
**Privileged Container**: DinD requires privileged mode to mount filesystems and create namespaces within the container. The `--privileged` flag grants extended capabilities.
**Separate Daemon**: A complete Docker daemon runs inside the container, managing its own containers, images, and networks independently of the host daemon.
**Certificate Management**: DinD uses mutual TLS authentication between the inner client and daemon. Certificates are shared through volumes mounted at `/certs`.
**Storage Driver**: The inner Docker daemon typically uses vfs or overlay2 storage driver. VFS provides maximum compatibility but larger storage overhead.
**Use in Forgejo Runners**:
```yaml
containers:
- name: runner
image: code.forgejo.org/forgejo/runner:6.4.0
env:
- name: DOCKER_HOST
value: tcp://localhost:2376
- name: DOCKER_TLS_VERIFY
value: "1"
- name: dind
image: docker:28.0.4-dind
securityContext:
privileged: true
volumeMounts:
- name: docker-certs
mountPath: /certs
```
The runner container connects to the DinD container via TCP, allowing workflow steps to execute docker commands for building and testing.
## Configuration
[Key configuration options and how to set them]
### Docker Daemon Configuration
The Docker daemon reads configuration from `/etc/docker/daemon.json`:
```json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"default-address-pools": [
{
"base": "172.17.0.0/16",
"size": 24
}
],
"dns": ["8.8.8.8", "8.8.4.4"],
"insecure-registries": [],
"registry-mirrors": [],
"features": {
"buildkit": true
},
"max-concurrent-downloads": 10,
"max-concurrent-uploads": 5
}
```
**Key Configuration Options**:
* `log-driver`: Logging mechanism (json-file, syslog, journald, etc.)
* `storage-driver`: Filesystem driver (overlay2, devicemapper, btrfs, zfs)
* `insecure-registries`: Registries that don't require HTTPS
* `registry-mirrors`: Mirror registries for faster pulls
* `buildkit`: Enable BuildKit for improved build performance
**Apply Configuration Changes**:
```bash
# Restart Docker daemon
sudo systemctl restart docker
# Verify configuration
docker info
```
### Docker-in-Docker Configuration
**Environment Variables**:
* `DOCKER_TLS_CERTDIR`: Directory for TLS certificates (typically `/certs`)
* `DOCKER_HOST`: Docker daemon address (e.g., `tcp://localhost:2376`)
* `DOCKER_TLS_VERIFY`: Enable TLS verification (`1` or `0`)
* `DOCKER_CERT_PATH`: Path to client certificates
**DinD Security Considerations**:
DinD requires privileged mode, which grants extended capabilities. Use DinD only in trusted environments:
- CI/CD runners in isolated namespaces
- Development environments
- Build systems with network isolation
Avoid using DinD for untrusted workloads or multi-tenant environments.
## Integration Points
* **Kubernetes**: Can use Docker (via dockershim, deprecated) or containerd directly as container runtime
* **Forgejo Actions**: Uses Docker-in-Docker for isolated build execution in CI/CD pipelines
* **Container Registries**: Pushes and pulls images to/from OCI-compliant registries
* **Development Environments**: Docker Desktop provides local container runtime for development
* **Image Scanning Tools**: Integrates with security scanners like Trivy and Clair
* **Monitoring Systems**: Exports metrics via Prometheus exporters and logging drivers
## Troubleshooting
### [Common Issue 1]
### Docker Daemon Won't Start
**Problem**: [Description]
**Problem**: Docker daemon fails to start or crashes immediately
**Solution**: [How to fix]
**Solution**:
1. Check daemon logs:
```bash
sudo journalctl -u docker.service
sudo cat /var/log/docker.log
```
### [Common Issue 2]
2. Verify kernel support:
```bash
docker info | grep -i kernel
grep CONFIG_NAMESPACES /boot/config-$(uname -r)
```
**Problem**: [Description]
3. Test daemon in debug mode:
```bash
sudo dockerd --debug
```
**Solution**: [How to fix]
4. Check for port conflicts:
```bash
sudo netstat -tulpn | grep docker
```
## Status
### Container Cannot Connect to Network
**Maturity**: [Production / Beta / Experimental]
**Problem**: Container has no network connectivity or DNS resolution fails
**Solution**:
1. Check container network mode:
```bash
docker inspect container-name | grep -i network
```
2. Verify DNS configuration:
```bash
docker exec container-name cat /etc/resolv.conf
docker exec container-name ping 8.8.8.8
docker exec container-name ping google.com
```
3. Check firewall rules:
```bash
sudo iptables -L -n | grep DOCKER
```
4. Restart Docker network:
```bash
sudo systemctl restart docker
```
5. Recreate default bridge:
```bash
docker network rm bridge
sudo systemctl restart docker
```
### Out of Disk Space
**Problem**: Docker runs out of disk space for images or containers
**Solution**:
1. Check disk usage:
```bash
docker system df
docker system df -v # Verbose output
```
2. Remove unused containers:
```bash
docker container prune
```
3. Remove unused images:
```bash
docker image prune -a
```
4. Remove unused volumes:
```bash
docker volume prune
```
5. Complete cleanup:
```bash
docker system prune -a --volumes
```
6. Configure log rotation in `/etc/docker/daemon.json`:
```json
{
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
```
### Docker Build Fails
**Problem**: Image build fails with errors or hangs
**Solution**:
1. Build with verbose output:
```bash
docker build --progress=plain --no-cache -t myapp .
```
2. Check Dockerfile syntax:
```bash
docker build --check -t myapp .
```
3. Verify base image exists:
```bash
docker pull ubuntu:22.04
```
4. Increase build memory (Docker Desktop):
- Open Docker Desktop settings
- Increase memory allocation to 4GB+
5. Check build context size:
```bash
docker build --progress=plain -t myapp . 2>&1 | grep "transferring context"
```
6. Use `.dockerignore` to exclude large files:
```
# .dockerignore
node_modules
.git
*.log
```
### Docker-in-Docker Container Cannot Build Images
**Problem**: DinD container fails to build images or start daemon
**Solution**:
1. Verify privileged mode:
```bash
docker inspect dind | grep -i privileged
```
2. Check DinD daemon logs:
```bash
docker logs dind
```
3. Verify certificate volumes:
```bash
docker exec dind ls -la /certs
```
4. Test Docker client connection:
```bash
docker run --rm \
-e DOCKER_HOST=tcp://dind:2376 \
-e DOCKER_TLS_VERIFY=1 \
-e DOCKER_CERT_PATH=/certs/client \
-v docker-certs:/certs:ro \
--link dind:dind \
docker:28.0.4-cli \
docker info
```
5. Check storage driver:
```bash
docker exec dind docker info | grep "Storage Driver"
```
### Permission Denied When Running Docker
**Problem**: User cannot run Docker commands without sudo
**Solution**:
1. Add user to docker group:
```bash
sudo usermod -aG docker $USER
```
2. Log out and back in to apply group changes
3. Verify group membership:
```bash
groups $USER
```
4. Test Docker access:
```bash
docker ps
```
5. If issue persists, check socket permissions:
```bash
sudo chmod 666 /var/run/docker.sock
```
Note: Adding users to the docker group grants root-equivalent privileges. Only add trusted users.
## Additional Resources
* [Link to external documentation]
* [Link to community resources]
* [Link to related components]
## Documentation Notes
[Instructions for team members filling in this documentation - remove this section once complete]
* [Docker Documentation](https://docs.docker.com/)
* [Docker Hub](https://hub.docker.com/)
* [Dockerfile Best Practices](https://docs.docker.com/develop/dev-best-practices/)
* [Containerd Documentation](https://containerd.io/)
* [OCI Specifications](https://opencontainers.org/)
* [Docker Security Best Practices](https://docs.docker.com/engine/security/)
* [Kernel Namespace Documentation](https://man7.org/linux/man-pages/man7/namespaces.7.html)
* [Control Groups (cgroups) Documentation](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html)