docs(forgejo): 💄Updated docs and added diagrams
This commit is contained in:
parent
1eff967f09
commit
ffb9d063a3
GPG key ID:
1F39A8BBCD2EE3D3
4 changed files with 148 additions and 73 deletions
|
|
@ -1,93 +1,163 @@
|
|||
---
|
||||
title: "Forgejo Integration, Extension, and Community Collaboration"
|
||||
linkTitle: Forgejo Software Forge
|
||||
date: "2025-11-17"
|
||||
description: "Summary of the project's work integrating GARM with Forgejo and contributing key features back to the community."
|
||||
tags: ["Forgejo", "GARM", "CI/CD", "OSS", "Community", "Project Report"]
|
||||
title: "Edge Developer Platform: Forgejo Integration & GARM Extension"
|
||||
date: "2025-11-19"
|
||||
description: "Technical report on the implementation of the Edge Developer Platform (EDP) using Forgejo, featuring GARM integration for ephemeral CI/CD runners and upstream open-source contributions."
|
||||
tags: ["Forgejo", "GARM", "CI/CD", "OSS", "Architecture", "Project Report"]
|
||||
categories: ["Workpackage Results"]
|
||||
weight: 10
|
||||
mermaid: true
|
||||
---
|
||||
|
||||
{{% alert title="Draft" color="warning" %}}
|
||||
**Editorial Status**: This page is currently being developed.
|
||||
## 1. Project Identity & Governance
|
||||
|
||||
* **Jira Ticket**: [TICKET-6731](https://jira.telekom-mms.com/browse/IPCEICIS-6731)
|
||||
* **Assignee**: Daniel
|
||||
* **Status**: Draft
|
||||
* **Last Updated**: 2025-11-17
|
||||
* **TODO**:
|
||||
* [ ] Add concrete quick start steps
|
||||
* [ ] Include prerequisites and access information
|
||||
* [ ] Create first application tutorial
|
||||
* **Review/Feedback**:
|
||||
* [ ] Stephan:
|
||||
* in general:
|
||||
* [ ] some parts are worth to go th 'Governance'
|
||||
* [ ] perhaps we should remove the emojis?
|
||||
* [ ] perhaps we should avoid the impression that the text was copy/pated from AI
|
||||
* some details/further ideas:
|
||||
* [ ] where is it, this Forgejo? Why is it called 'edp.buildth.ing'?
|
||||
* [ ] what are the components we use - package managament, actions, ...
|
||||
* [ ] Friendly users? organisations? Public/private stuff?
|
||||
* [ ] App Management discussions (we don't!)?
|
||||
* [ ] what about code snippets how forgejo is deployed? SSO? user base? Federation options?
|
||||
* [ ] storages, Redis, Postgres ... deployment options ... helm charts ...
|
||||
* [ ] Migrations we did, where is the migration code?
|
||||
* [ ] git POSIX filesystem concurrency discussion, S/3 bucket
|
||||
* [ ] what is our general experience?
|
||||
* [ ] repository centric domain data model
|
||||
* [ ] how did we develop? which version did we take first? how did we upgrade?
|
||||
* [ ] which development flows did we use? which pipleines?
|
||||
* [ ] provide codeberg links for the PRs
|
||||
* [ ] provide architecture drawings and repo links for the cache registry thing
|
||||
* [ ] provide a hight level actions arch diagram from the perspective of forgejo - link to the GARM component here
|
||||
{{% /alert %}}
|
||||
### 1.1 Project Scope
|
||||
|
||||
## 🧾 Result short description / cognitions
|
||||
The internal service is officially designated as the **Edge Developer Platform (EDP)**. It is hosted at **[edp.buildth.ing](https://edp.buildth.ing)**. The domain selection followed a democratic team process to establish a unique identity distinct from standard corporate naming conventions.
|
||||
|
||||
Here is the management summary of the work package results:
|
||||
**Access Model:**
|
||||
The platform operates on a hybrid visibility model:
|
||||
|
||||
* **📈 Strategic Selection:** We chose **[Forgejo](https://forgejo.org/)** as the project's self-hosted Git service. This decision was based on several key strategic factors:
|
||||
* **EU-Based & Data Sovereignty:** The project is stewarded by **[Codeberg e.V.](https://docs.codeberg.org/getting-started/what-is-codeberg/)**, a non-profit based in Berlin, Germany. This is a massive win for our "funding agency" stakeholders, as it aligns with **GDPR, compliance, and data sovereignty goals**. It's governed by EU laws, not a US tech entity.
|
||||
* **True Open Source (GPL v3+):** Forgejo is a community-driven fork of Gitea, created to *guarantee* it stays 100% free and open-source (FOSS).
|
||||
* **License Protects Our Contributions:** It uses the **GPL v3+ "copyleft" license**. This is *perfect* for our collaboration goal. It legally ensures that the features we contribute back (like GARM support) can **never be taken and locked into a proprietary, closed-source product by anyone**. It protects our work and keeps the community open.
|
||||
* **Public Access:** The [`DEVFW-CICD`](https://edp.buildth.ing/DevFW-CICD) organization is publicly accessible, fostering transparency.
|
||||
* **Private Access:** Sensitive development occurs in restricted organizations (e.g., [`DEVFW`](https://edp.buildth.ing/DevFW)).
|
||||
* **User Base:** Primary users include the internal development team, with friendly user access granted to the IPCEI team and MMS BT.
|
||||
|
||||
* **⚙️ Core Use Case:** Forgejo is used for all project source code **versioning** and as the backbone for our **CI/CD (Continuous Integration/Continuous Deployment)** pipelines.
|
||||
### 1.2 Strategic Selection & Governance
|
||||
|
||||
* **🛠️ Key Extension (GARM Support):** The main technical achievement was integrating **[GARM (GitHub Actions Runner Manager)](https://github.com/cloudbase/garm)**. This was *not* supported by Forgejo out-of-the-box.
|
||||
The decision to utilize **[Forgejo](https://forgejo.org/)** as the core self-hosted Git service was driven by specific strategic requirements:
|
||||
|
||||
* **✨ Required Enhancements:** To make GARM work, our team developed and implemented several critical features:
|
||||
* Webhook support for workflow events (to tell runners when to start).
|
||||
* Support for ephemeral runners (for secure, clean-slate builds every time).
|
||||
* GitHub API-compatible endpoints (to allow the runners to register themselves correctly).
|
||||
|
||||
* **💖 Community Contribution:** We didn't just keep this for ourselves! We contributed all these features **directly back to the upstream Forgejo community**. This wasn't just a code-dump; we actively collaborated via **issues**, **feature requests**, and **pull requests (PRs) on [codeberg.org](https://codeberg.org/)**.
|
||||
|
||||
* **🚀 Bonus Functionality:** We also implemented **artifact caching**. This configures Forgejo to act as a **pull-through proxy** for remote container registries (like Docker Hub), which seriously speeds up our build times and saves bandwidth.
|
||||
* **EU-Based Stewardship:** Forgejo is stewarded by **[Codeberg e.V.](https://docs.codeberg.org/getting-started/what-is-codeberg/)**, a non-profit organization based in Berlin, Germany. This alignment ensures compliance with GDPR and data sovereignty requirements, placing governance under EU jurisdiction rather than US tech entities.
|
||||
* **License Protection (GPL v3+):** Unlike "Open Core" models, Forgejo uses a copyleft license. This legally protects our custom extensions (such as GARM support) from being appropriated into proprietary software, ensuring the ecosystem remains open.
|
||||
* **Open Source Strategy:** The platform aligns with the "Public Money, Public Code" philosophy, mandating that funded developments are returned to the community.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Key Performance Indicators (KPIs)
|
||||
## 2. Technical Architecture & Deployment
|
||||
|
||||
These metrics assess the success of the work package across technical quality, development efficiency, and open-source engagement.
|
||||
### 2.1 Infrastructure Stack
|
||||
|
||||
### 🚀 CI/CD & Delivery Efficiency
|
||||
The platform is hosted on the **Open Telekom Cloud (OTC)**. The infrastructure adheres to Infrastructure-as-Code (IaC) principles.
|
||||
|
||||
These KPIs measure the effectiveness of the Forgejo + GARM setup.
|
||||
* **Deployment Method:** The official Forgejo Helm Chart is deployed via **ArgoCD**.
|
||||
* **Infrastructure Provisioning:** **Terraform** is used to provision all underlying OTC services, including:
|
||||
* **Container Orchestration**: CCE (Cloud Container Engine): Kubernetes
|
||||
* **Database:** RDS (Distributed Cache Service): PostgreSQL
|
||||
* **Caching:** DCS (Distributed Cache Service): Redis
|
||||
* **Object Storage:** OBS (Object Storage Service, S3-compatible): for user data (avatars, attachments).
|
||||
* **Search:** CSS (Cloud Search Service): Elasticsearch
|
||||
|
||||
### 2.2 The "Self-Replicating" Pipeline
|
||||
|
||||
A key architectural feature is the ability of the platform to maintain itself. A Forgejo Action can trigger the deployment script, which runs Terraform and syncs ArgoCD, effectively allowing "Forgejo to create/update Forgejo."
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph "Open Telekom Cloud (OTC)"
|
||||
subgraph "Control Plane"
|
||||
Dev[DevOps Engineer] -->|Triggers| Pipeline[Deployment Pipeline]
|
||||
Pipeline -->|Executes| TF[Terraform]
|
||||
end
|
||||
|
||||
subgraph "Provisioned Infrastructure"
|
||||
TF -->|Provisions| CCE[(CCE K8s Cluster)]
|
||||
TF -->|Provisions| RDS[(RDS PostgreSQL)]
|
||||
TF -->|Provisions| Redis[(DCS Redis)]
|
||||
TF -->|Provisions| S3[(OBS S3 Bucket)]
|
||||
TF -->|Provisions| CSS[(CSS Elasticsearch)]
|
||||
end
|
||||
|
||||
subgraph "Application Layer (on CCE K8s)"
|
||||
Pipeline -->|Helm Chart| Argo[ArgoCD]
|
||||
Argo -->|Deploys| ForgejoApp[Forgejo]
|
||||
end
|
||||
|
||||
CCE -- Runs --> Argo
|
||||
CCE -- Runs --> ForgejoApp
|
||||
ForgejoApp -->|Connects| RDS
|
||||
ForgejoApp -->|Connects| Redis
|
||||
ForgejoApp -->|Connects| S3
|
||||
ForgejoApp -->|Connects| CSS
|
||||
end
|
||||
```
|
||||
|
||||
### 2.3 Migration History
|
||||
|
||||
The initial environment was a manual setup on the Open Sovereign Cloud (OSC). Once the automation stack (Terraform/ArgoCD) was matured, the platform was migrated to the current OTC environment.
|
||||
|
||||
---
|
||||
|
||||
## 3. Application Extensions & GARM Integration
|
||||
|
||||
### 3.1 Core Functionality
|
||||
|
||||
Beyond standard Git versioning, the platform utilizes:
|
||||
|
||||
* **Releases:** Hosting binaries for software distribution (e.g., Edge Connect CLI).
|
||||
* **CI/CD:** Extensive pipeline usage for build, test, and deployment automation.
|
||||
* **Note on Issues:** While initially used, issue tracking was migrated to JIRA to align with the broader IPCEI program standards.
|
||||
|
||||
### 3.2 GARM (Git-based Actions Runner Manager)
|
||||
|
||||
The primary technical innovation was the integration of **[GARM](https://github.com/cloudbase/garm)** to enable ephemeral, scalable runners. This required extending Forgejo's capabilities to support GitHub-compatible runner registration and webhook events.
|
||||
|
||||
**Workflow Architecture:**
|
||||
|
||||
1. **Event:** A workflow event occurs in Forgejo.
|
||||
2. **Trigger:** A webhook notifies GARM.
|
||||
3. **Provisioning:** GARM spins up a fresh, ephemeral runner.
|
||||
4. **Execution:** The runner registers via the API, executes the job, and is terminated immediately after, ensuring a clean build environment.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant Forgejo
|
||||
participant GARM
|
||||
participant Runner as Ephemeral Runner
|
||||
|
||||
User->>Forgejo: Push Code / Trigger Event
|
||||
Forgejo->>GARM: Webhook Event (Workflow Dispatch)
|
||||
GARM->>Forgejo: Register Runner (via API)
|
||||
GARM->>Runner: Spin up Instance
|
||||
Runner->>Forgejo: Request Job
|
||||
Forgejo->>Runner: Send Job Payload
|
||||
Runner->>Runner: Execute Steps
|
||||
Runner->>Forgejo: Report Status
|
||||
GARM->>Runner: Terminate (Ephemeral)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Development Methodology & Contributions
|
||||
|
||||
### 4.1 Workflow
|
||||
|
||||
* **Branching Strategy:** Trunk-based development was utilized to ensure rapid integration.
|
||||
* **Collaboration:** The team adopted **Mob Programming**. This practice proved essential for knowledge sharing and onboarding junior developers, creating a resilient and high-intensity learning environment.
|
||||
* **Versions:** The platform evolved from Forgejo v7/8 to the current v11.0.3-edp1. An upgrade is pending to leverage the latest upstream GARM features.
|
||||
|
||||
### 4.2 Open Source Contributions
|
||||
|
||||
We actively contributed our extensions back to the upstream Forgejo project on **[Codeberg.org](https://codeberg.org/)**.
|
||||
|
||||
**Key Pull Requests:**
|
||||
|
||||
* **API Compatibility:** Added GitHub-compatible endpoints for runner registration.
|
||||
* [PR #9409: Feat: Add endpoints for GARM](https://codeberg.org/forgejo/forgejo/pulls/9409)
|
||||
* **Webhook Support:** Implemented webhook triggers for workflow events.
|
||||
* [PR #9803: Feat: Add webhook support for workflow events](https://codeberg.org/forgejo/forgejo/pulls/9803)
|
||||
* **Ephemeral Runners:** Added support for runners that terminate after a single job.
|
||||
* [PR #9962: Feat: Support for ephemeral runners](https://codeberg.org/forgejo/forgejo/pulls/9962)
|
||||
|
||||
**Artifact Caching (Pull-Through Proxy):**
|
||||
We implemented a feature allowing Forgejo to act as a pull-through proxy for remote container registries, optimizing bandwidth and build speeds.
|
||||
|
||||
* [Source Code Branch: refactor-remote-registry-client](https://edp.buildth.ing/DevFW/edp-forgejo/src/branch/refactor-remote-registry-client)
|
||||
|
||||
---
|
||||
|
||||
## 5. Key Performance Indicators (KPIs)
|
||||
|
||||
| KPI | Description | Target / Benchmark |
|
||||
| :--- | :--- | :--- |
|
||||
| **Deployment Frequency** | How often changes are successfully pushed to production/staging (e.g., deploys per day/week). | High frequency (daily or better) indicates high agility. |
|
||||
| **Change Failure Rate (CFR)** | Percentage of deployments that require immediate remediation (e.g., a rollback or hotfix). | Below 5% (Industry standard for high performance). |
|
||||
| **Cycle Time** | Time from the first commit to deployment/release. | Measured in days, trending downward, indicating pipeline efficiency. |
|
||||
| **Artifact Cache Hit Rate** | Percentage of build requests successfully served by the local Forgejo proxy cache. | **90%+** demonstrates effective use of the proxy and reduced external dependencies. |
|
||||
|
||||
### 💖 Open Source & Community Engagement
|
||||
|
||||
These KPIs quantify our strategic commitment to the Forgejo community.
|
||||
|
||||
| KPI | Description | Target / Benchmark |
|
||||
| :--- | :--- | :--- |
|
||||
| **Upstream Contribution Ratio** | Percentage of project code (GARM-related features) committed to Forgejo's upstream repositories versus being maintained in a private fork. | **100%** demonstrates full commitment to collaboration. |
|
||||
| **Pull Request (PR) Resolution Time** | Average time taken for a PR submitted to Codeberg to be merged or closed by the Forgejo community. | Trending downward (e.g., < 7 days) indicates healthy, responsive collaboration. |
|
||||
| **Number of Accepted External PRs** | Count of community-submitted features/fixes related to our GARM extension that we review/merge into our implementation. | Increase over time indicates growing external adoption/interest in our work. |
|
||||
| **Deployment Frequency** | Frequency of successful pipeline executions. | High (Daily/On-demand) |
|
||||
| **Artifact Cache Hit Rate** | Percentage of build requests served by the local Forgejo proxy. | > 90% (Reduced external traffic) |
|
||||
| **Upstream Contribution** | Percentage of GARM-related features contributed back to Codeberg. | 100% (No vendor lock-in) |
|
||||
| **PR Resolution Time** | Average time for upstream community review and merge. | < 14 days (Healthy collaboration) |
|
||||
|
|
|
|||
1
go.mod
1
go.mod
|
|
@ -5,5 +5,6 @@ go 1.22.5
|
|||
require (
|
||||
github.com/FortAwesome/Font-Awesome v0.0.0-20241216213156-af620534bfc3 // indirect
|
||||
github.com/google/docsy v0.12.0 // indirect
|
||||
github.com/hugomods/mermaid v0.1.4 // indirect
|
||||
github.com/twbs/bootstrap v5.3.8+incompatible // indirect
|
||||
)
|
||||
|
|
|
|||
2
go.sum
2
go.sum
|
|
@ -2,6 +2,8 @@ github.com/FortAwesome/Font-Awesome v0.0.0-20241216213156-af620534bfc3 h1:/iluJk
|
|||
github.com/FortAwesome/Font-Awesome v0.0.0-20241216213156-af620534bfc3/go.mod h1:IUgezN/MFpCDIlFezw3L8j83oeiIuYoj28Miwr/KUYo=
|
||||
github.com/google/docsy v0.12.0 h1:CddZKL39YyJzawr8GTVaakvcUTCJRAAYdz7W0qfZ2P4=
|
||||
github.com/google/docsy v0.12.0/go.mod h1:1bioDqA493neyFesaTvQ9reV0V2vYy+xUAnlnz7+miM=
|
||||
github.com/hugomods/mermaid v0.1.4 h1:/u0FZRSBMpqTbDh4XPBL7c6p0DUjvIkoo9cC89GASSI=
|
||||
github.com/hugomods/mermaid v0.1.4/go.mod h1:GOID2Ko1vdrX03ZMaFNQL/vdTU1tZmGcVjOgBN8O42Y=
|
||||
github.com/twbs/bootstrap v5.3.6+incompatible/go.mod h1:fZTSrkpSf0/HkL0IIJzvVspTt1r9zuf7XlZau8kpcY0=
|
||||
github.com/twbs/bootstrap v5.3.8+incompatible h1:eK1fsXP7R/FWFt+sSNmmvUH9usPocf240nWVw7Dh02o=
|
||||
github.com/twbs/bootstrap v5.3.8+incompatible/go.mod h1:fZTSrkpSf0/HkL0IIJzvVspTt1r9zuf7XlZau8kpcY0=
|
||||
|
|
|
|||
|
|
@ -156,6 +156,8 @@ enable = false
|
|||
[[module.imports]]
|
||||
path = "github.com/google/docsy"
|
||||
disable = false
|
||||
[[module.imports]]
|
||||
path = "github.com/hugomods/mermaid"
|
||||
|
||||
[params.mermaid]
|
||||
version = "10.9.0"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue