This commit is contained in:
parent
862fc07328
commit
d0aea88a5b
3 changed files with 15 additions and 11 deletions
|
|
@ -28,7 +28,7 @@ make install-hooks # Install pre-commit and commit-msg hooks
|
|||
|
||||
## Architecture Overview
|
||||
|
||||
This is a Go metrics collector designed for CI/CD environments with shared PID namespaces. It consists of two binaries:
|
||||
A resource optimiser for CI/CD environments with shared PID namespaces. It consists of two binaries — a **collector** and a **receiver** (which includes the **sizer**):
|
||||
|
||||
### Collector (`cmd/collector`)
|
||||
Runs alongside CI workloads, periodically reads `/proc` filesystem, and pushes a summary to the receiver on shutdown (SIGINT/SIGTERM).
|
||||
|
|
@ -40,11 +40,12 @@ Runs alongside CI workloads, periodically reads `/proc` filesystem, and pushes a
|
|||
4. On shutdown, `summary.PushClient` sends the summary to the receiver HTTP endpoint
|
||||
|
||||
### Receiver (`cmd/receiver`)
|
||||
HTTP service that stores metric summaries in SQLite (via GORM) and provides a query API.
|
||||
HTTP service that stores metric summaries in SQLite (via GORM), provides a query API, and includes the **sizer** — which computes right-sized Kubernetes resource requests and limits from historical data.
|
||||
|
||||
**Key Endpoints:**
|
||||
- `POST /api/v1/metrics` - Receive metrics from collectors
|
||||
- `GET /api/v1/metrics/repo/{org}/{repo}/{workflow}/{job}` - Query stored metrics
|
||||
- `GET /api/v1/sizing/repo/{org}/{repo}/{workflow}/{job}` - Compute container sizes from historical data
|
||||
|
||||
### Internal Packages
|
||||
|
||||
|
|
@ -55,7 +56,7 @@ HTTP service that stores metric summaries in SQLite (via GORM) and provides a qu
|
|||
| `internal/proc` | Low-level /proc parsing (stat, status, cgroup) |
|
||||
| `internal/cgroup` | Parses CGROUP_LIMITS and CGROUP_PROCESS_MAP env vars |
|
||||
| `internal/summary` | Accumulates samples, computes stats, pushes to receiver |
|
||||
| `internal/receiver` | HTTP handlers and SQLite store |
|
||||
| `internal/receiver` | HTTP handlers, SQLite store, and sizer logic |
|
||||
| `internal/output` | Metrics output formatting (JSON/text) |
|
||||
|
||||
### Container Metrics
|
||||
|
|
|
|||
17
README.md
17
README.md
|
|
@ -1,10 +1,10 @@
|
|||
# Forgejo Runner Resource Collector
|
||||
# Forgejo Runner Optimiser
|
||||
|
||||
A lightweight metrics collector for CI/CD workloads in shared PID namespace environments. Reads `/proc` to collect CPU and memory metrics, groups them by container/cgroup, and pushes run summaries to a receiver service for storage and querying.
|
||||
A resource optimiser for CI/CD workloads in shared PID namespace environments. The **collector** reads `/proc` to gather CPU and memory metrics grouped by container/cgroup, and pushes run summaries to the **receiver**. The receiver stores metrics and exposes a **sizer** API that computes right-sized Kubernetes resource requests and limits from historical data.
|
||||
|
||||
## Architecture
|
||||
|
||||
The system has two independent binaries:
|
||||
The system has two binaries — a **collector** and a **receiver** (which includes the sizer):
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐ ┌──────────────────────────┐
|
||||
|
|
@ -19,7 +19,9 @@ The system has two independent binaries:
|
|||
│ └───────────┘ └────────┘ └───────────┘ │ │ │ │
|
||||
│ │ │ ▼ │
|
||||
└─────────────────────────────────────────────┘ │ GET /api/v1/metrics/... │
|
||||
└──────────────────────────┘
|
||||
│ GET /api/v1/sizing/... │
|
||||
│ (sizer) │
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
### Collector
|
||||
|
|
@ -56,9 +58,9 @@ Runs as a sidecar alongside CI workloads. On a configurable interval, it reads `
|
|||
|
||||
CPU supports Kubernetes notation (`"2"` = 2 cores, `"500m"` = 0.5 cores). Memory supports `Ki`, `Mi`, `Gi`, `Ti` (binary) or `K`, `M`, `G`, `T` (decimal).
|
||||
|
||||
### Receiver
|
||||
### Receiver (with sizer)
|
||||
|
||||
HTTP service that stores metric summaries in SQLite (via GORM) and exposes a query API.
|
||||
HTTP service that stores metric summaries in SQLite (via GORM), exposes a query API, and provides a **sizer** endpoint that computes right-sized Kubernetes resource requests and limits from historical run data.
|
||||
|
||||
```bash
|
||||
./receiver --addr=:8080 --db=metrics.db --read-token=my-secret-token --hmac-key=my-hmac-key
|
||||
|
|
@ -78,6 +80,7 @@ HTTP service that stores metric summaries in SQLite (via GORM) and exposes a que
|
|||
- `POST /api/v1/metrics` — receive and store a metric summary (requires scoped push token)
|
||||
- `POST /api/v1/token` — generate a scoped push token (requires read token auth)
|
||||
- `GET /api/v1/metrics/repo/{org}/{repo}/{workflow}/{job}` — query stored metrics (requires read token auth)
|
||||
- `GET /api/v1/sizing/repo/{org}/{repo}/{workflow}/{job}` — compute container sizes from historical data (requires read token auth)
|
||||
|
||||
**Authentication:**
|
||||
|
||||
|
|
@ -232,7 +235,7 @@ PUSH_TOKEN=$(curl -s -X POST http://localhost:8080/api/v1/token \
|
|||
| `internal/cgroup` | Parses `CGROUP_PROCESS_MAP` and `CGROUP_LIMITS` env vars |
|
||||
| `internal/collector` | Orchestrates the collection loop and shutdown |
|
||||
| `internal/summary` | Accumulates samples, computes stats, pushes to receiver |
|
||||
| `internal/receiver` | HTTP handlers and SQLite store |
|
||||
| `internal/receiver` | HTTP handlers, SQLite store, and sizer logic |
|
||||
| `internal/output` | Metrics output formatting (JSON/text) |
|
||||
|
||||
## Background
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
// ABOUTME: Computes ideal container sizes from historical run data.
|
||||
// ABOUTME: Provides Kubernetes-style resource recommendations.
|
||||
// ABOUTME: Provides Kubernetes-style resource sizes.
|
||||
package receiver
|
||||
|
||||
import (
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue