StageX Base Images

Zero-trust, reproducible container base images for the IPCEI-CIS platform.

Epic: IPCEICIS-6853 | Story: IPCEICIS-8808

---

What is StageX?

StageX is a reproducible, multi-signed OCI image ecosystem that provides minimal container base images built entirely from source. Unlike traditional distro-based images (Alpine, Ubuntu), StageX eliminates package managers, uses a deterministic build pipeline, and enforces multi-party PGP signing on every artifact.

Source: codeberg.org/stagex/stagex (mirrored at github.com/stagex-mirror/stagex)

Why StageX for IPCEI-CIS?

1. Reproducible builds — identical inputs always produce identical outputs (bit-for-bit). SOURCE_DATE_EPOCH=1 eliminates timestamp variance.
2. Zero-trust supply chain — every package is PGP-signed by multiple independent parties (quorum signing). No single compromised key can inject malicious code.
3. Minimal attack surface — no package manager, no shell by default in production images, only explicitly declared dependencies.
4. European sovereignty — hosted on Codeberg (Germany), aligns with IPCEI-CIS data sovereignty requirements.
5. SBOM-friendly — deterministic dependency graph makes software bill of materials trivial to generate.

---

Architecture Overview

The Stage0 Bootstrap Chain

StageX achieves full reproducibility through a bootstrapping chain that eliminates trust in any pre-existing binary:

Stage0 (hex0 → M0 → M1 → M2 → mescc → tcc → gcc → musl → LLVM/Clang)
  └── StageX packages (built with LLVM/Clang + musl libc)
       ├── core-filesystem    (FHS directory layout on scratch)
       ├── core-musl          (C standard library)
       ├── core-busybox       (shell + core utilities)
       ├── core-ca-certificates
       ├── core-openssl
       ├── core-nodejs
       ├── core-go
       ├── pallet-go          (complete Go dev environment)
       ├── pallet-nodejs      (complete Node.js dev environment)
       └── ... (~513 packages)

The bootstrap starts from a minimal hex assembler (hex0, ~500 bytes of auditable code) and builds up through increasingly complex compilers until reaching a full LLVM/Clang toolchain. This means no pre-compiled binary is trusted — everything is built from source, from first principles.

How It Works — The COPY --from Pattern

StageX has no package manager. Dependencies are composed by copying filesystem layers from signed OCI images:

FROM stagex/core-filesystem AS base     # FHS layout (scratch + /usr, /etc, etc.)
COPY --from=stagex/core-musl . /        # C library
COPY --from=stagex/core-busybox . /     # Shell + utilities
COPY --from=stagex/core-ca-certificates . /  # TLS root certificates

Each COPY --from=stagex/core-* pulls a single, signed package and overlays it onto the filesystem. This is explicit, auditable, and reproducible — you see exactly what goes into your image.

Signing Model — PGP Quorum

StageX uses PGP-based multi-party signing, not cosign/sigstore:

• Every package is signed by multiple independent maintainers
• A configurable quorum (e.g., 3-of-5) must be met before a package is considered valid
• Keys are distributed via standard PGP infrastructure
• This prevents single-point-of-compromise attacks on the supply chain

---

Comparison: StageX vs Alternatives

Property	StageX	Alpine	Ubuntu	Chainguard	Distroless
Base size	~2-5 MB	~7 MB	~78 MB	~10 MB	~2-20 MB
C library	musl	musl	glibc	glibc	glibc
Toolchain	LLVM/Clang	GCC	GCC	varies	N/A
Package manager	None (COPY)	apk	apt	apk	None
Reproducible	Yes (bit-for-bit)	Partial	No	Partial	Partial
Signing	PGP quorum	apk signing	apt signing	cosign/sigstore	cosign/sigstore
Bootstrap chain	Full (Stage0)	No	No	No	No
Shell by default	Optional	Yes	Yes	Optional	No
SBOM	Deterministic	Generated	Generated	Generated	Generated
Sovereign hosting	Codeberg (EU)	Alpine Infra	Canonical (UK)	Chainguard (US)	Google (US)
Package count	~513	~30,000+	~60,000+	~1,000+	Limited
License	MIT	MIT	Mixed	Proprietary (images)	Apache 2.0

When to use StageX

• Use StageX when you need reproducible builds, supply chain integrity, minimal attack surface, and European data sovereignty.
• Use Alpine when you need a wide package ecosystem and don't require bit-for-bit reproducibility.
• Use Chainguard/Distroless when you need glibc compatibility and are comfortable with US-hosted signing infrastructure.

---

Prerequisites

Docker v25+ with containerd Image Store

StageX OCI images require Docker v25 or later with the containerd image store enabled.

Check your Docker version:

docker version --format '{{.Server.Version}}'
# Must be 25.x or higher

Enable containerd image store — add to ~/.docker/daemon.json:

{
  "features": {
    "containerd-snapshotter": true
  }
}

Then restart Docker:

sudo systemctl restart docker
# or on macOS: restart Docker Desktop

Verify:

docker info | grep -i containerd
# Should show containerd-related output

GNU Make

The Makefile automates all build operations. Any recent version of GNU Make works.

---

Project Structure

stagex/
├── README.md              ← This document
├── Makefile               ← Build automation
├── .dockerignore
├── base/
│   └── Containerfile      ← Minimal base image (busybox + musl + TLS)
├── base-go/
│   └── Containerfile      ← Go runtime base (musl + ca-certs only)
├── base-nodejs/
│   └── Containerfile      ← Node.js runtime base
└── examples/
    └── go-hello/
        ├── Containerfile  ← Example multi-stage Go build
        ├── main.go
        └── go.mod

---

Building the Images

Quick Start

cd stagex/

# Check prerequisites
make check-docker

# Build all base images
make all

# Build and run the Go example
make run-example-go

Individual Targets

make base          # Build minimal base image
make base-go       # Build Go runtime base
make base-nodejs   # Build Node.js runtime base
make example-go    # Build Go hello-world example

Custom Registry / Tag

make all REGISTRY=my-registry.example.com/team TAG=v1.0.0

Verify Reproducibility

make verify-reproducible
# Builds the base image twice and compares digests
# ✅ REPRODUCIBLE: sha256:abc123... (if working correctly)

---

Using the Base Images

Go Application (Recommended Pattern)

# syntax=docker/dockerfile:1
ARG SOURCE_DATE_EPOCH=1

# Build with full Go toolchain
FROM stagex/pallet-go AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN ["go", "mod", "download"]
COPY . .
RUN ["go", "build", "-trimpath", "-ldflags=-w -s -buildid=", "-o", "/app", "."]

# Run on minimal base — no shell, no toolchain
FROM stagex/core-filesystem AS package
COPY --from=stagex/core-musl . /
COPY --from=stagex/core-ca-certificates . /
COPY --from=build /app /usr/bin/app
ENTRYPOINT ["/usr/bin/app"]

Key flags for reproducible Go builds:

• -trimpath — removes local filesystem paths from binary
• -ldflags=-w -s -buildid= — strips debug info and build ID
• CGO_ENABLED=0 — recommended for fully static binaries (avoids musl/glibc issues)

Node.js Application

# syntax=docker/dockerfile:1
ARG SOURCE_DATE_EPOCH=1

FROM stagex/pallet-nodejs AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN ["npm", "ci", "--production"]
COPY . .

FROM stagex/core-filesystem AS package
COPY --from=stagex/core-musl . /
COPY --from=stagex/core-nodejs . /
COPY --from=stagex/core-ca-certificates . /
COPY --from=stagex/core-openssl . /
COPY --from=stagex/core-zlib . /
COPY --from=build /app /app
WORKDIR /app
ENTRYPOINT ["/usr/bin/node", "index.js"]

---

Security Properties

Property	Detail
Reproducibility	SOURCE_DATE_EPOCH=1 ensures timestamps are deterministic. Build twice → same digest.
Supply chain	PGP quorum signing — multiple independent parties must sign each package.
Minimal surface	Production images have no shell, no package manager, no toolchain.
C library	musl libc — smaller, simpler, fewer CVEs than glibc.
Toolchain	LLVM/Clang — modern, well-audited compiler infrastructure.
Bootstrap	Full Stage0 chain — no pre-compiled binary trust required.
No root by default	Images contain no user database; add core-shadow if needed.

---

Available Packages

StageX provides ~513 packages covering core system libraries, languages, databases, and tools.

Browse the full catalog: stagex.tools/packages

Commonly used packages:

Package	Description
core-filesystem	FHS directory layout (scratch base)
core-musl	musl C library
core-busybox	Shell and core POSIX utilities
core-ca-certificates	Mozilla TLS root certificate bundle
core-openssl	OpenSSL TLS library
core-nodejs	Node.js runtime
core-go	Go compiler and runtime
core-zlib	Compression library
core-git	Git version control
core-curl	HTTP client
pallet-go	Complete Go development environment
pallet-nodejs	Complete Node.js development environment

---

Known Considerations

musl vs glibc

StageX uses musl libc, not glibc. This affects:

• Go applications — use CGO_ENABLED=0 for static builds, or ensure CGO dependencies are musl-compatible. Pure Go code works without issues.
• Node.js native addons — native modules compiled against glibc will not work. Most pure-JS npm packages are fine. For native addons, they must be compiled within the StageX build environment.
• DNS resolution — musl's DNS resolver behaves slightly differently from glibc (e.g., no nsswitch.conf). For most containerized workloads this is transparent.
• Locale support — musl has limited locale support compared to glibc. LC_ALL=C is recommended.

No Shell by Default

Production images (like base-go runtime) intentionally have no shell. This means:

• docker exec -it <container> /bin/sh will fail — this is by design
• Add stagex/core-busybox in development/debug variants if shell access is needed
• All commands in Containerfiles must use exec form: RUN ["cmd", "arg"] not RUN cmd arg

Package Ecosystem Size

With ~513 packages, StageX covers most common needs but is smaller than Alpine (~30k) or Ubuntu (~60k). If a package is missing:

1. Check stagex.tools/packages for the latest catalog
2. Packages can be requested or contributed upstream
3. For rare dependencies, consider a multi-stage build that compiles from source within StageX

---

Reproducibility

All Containerfiles pin base images by @sha256: digest (never :latest tags). This ensures:

• Deterministic builds: Same input → same output, always
• Tamper detection: Any upstream change breaks the build (intentional)
• Auditability: Every dependency is traceable to an exact image

Verifying Reproducibility

# Build twice and compare OCI tar digests
make verify-reproducible

# Verify Go example reproducibility
make verify-reproducible-go

The verification builds the image twice with --no-cache and compares SHA256 hashes of the OCI tar output. Matching hashes prove bit-for-bit reproducibility.

SOURCE_DATE_EPOCH

All builds use SOURCE_DATE_EPOCH (derived from the last git commit timestamp) to ensure filesystem timestamps are deterministic. This is set automatically by the Makefile.

Updating Pinned Digests

When upstream StageX releases new packages:

# Check which digests are outdated
make digest-check

# Update digests manually in Containerfiles, then verify
make all
make verify-reproducible

See DIGESTS.md for the full list of pinned digests.

Multi-Architecture Support

Currently amd64-only (StageX upstream limitation). All Containerfiles include TARGETARCH build arguments for future arm64 support without restructuring.

Architecture	Status
linux/amd64	✅ Fully supported
linux/arm64	⏳ Designed for, pending upstream support

See Siderolabs StageX fork for an existing arm64 build approach.

---

References

• StageX Documentation
• StageX Package Catalog
• StageX Source (Codeberg)
• StageX Mirror (GitHub)
• Stage0 Bootstrap Project
• Epic: IPCEICIS-6853
• Story: IPCEICIS-8808