diff --git a/content/en/docs/components/forgejo/_index.md b/content/en/docs/components/forgejo/_index.md index bc41163..0439c8d 100644 --- a/content/en/docs/components/forgejo/_index.md +++ b/content/en/docs/components/forgejo/_index.md @@ -2,27 +2,107 @@ title: "Forgejo" linkTitle: "Forgejo" weight: 20 -description: > - Self-hosted Git service with project management and CI/CD capabilities. +description: Forgejo provides source code management, project management, and CI/CD automation for the EDP. --- -{{% alert title="Draft" color="warning" %}} -**Editorial Status**: This page is currently being developed. +The internal service is officially designated as the Edge Developer Platform (EDP). It is hosted at **[edp.buildth.ing](https://edp.buildth.ing)**. The domain selection followed a democratic team process to establish a unique identity distinct from standard corporate naming conventions. -* **Jira Ticket**: [TICKET-XXX](https://your-jira/browse/TICKET-XXX) -* **Assignee**: [Name or Team] -* **Status**: Draft -* **Last Updated**: YYYY-MM-DD -* **TODO**: - * [ ] Add detailed component description - * [ ] Include usage examples and code samples - * [ ] Add architecture diagrams - * [ ] Review and finalize content -{{% /alert %}} +![alt text](image.png) -Forgejo provides source code management, project management, and CI/CD automation for the EDP. +![alt text](image-1.png) -## Sub-Components +## Technical Architecture & Deployment -* **Project Management**: Issue tracking and project management features -* **Actions**: CI/CD automation (see CI/CD section) +### Infrastructure Stack + +The platform is hosted on the **Open Telekom Cloud (OTC)**. The infrastructure adheres to Infrastructure-as-Code (IaC) principles. + +* **Deployment Method:** The official Forgejo Helm Chart is deployed via **ArgoCD**. +* **Infrastructure Provisioning:** **Terraform** is used to provision all underlying OTC services, including: + * **Container Orchestration**: CCE (Cloud Container Engine): Kubernetes + * **Database:** RDS (Distributed Cache Service): PostgreSQL + * **Caching:** DCS (Distributed Cache Service): Redis + * **Object Storage:** OBS (Object Storage Service, S3-compatible): for user data (avatars, attachments). + * **Search:** CSS (Cloud Search Service): Elasticsearch + +### The "Self-Replicating" Pipeline + +A key architectural feature is the ability of the platform to maintain itself. A Forgejo Action can trigger the deployment script, which runs Terraform and syncs ArgoCD, effectively allowing "Forgejo to create/update Forgejo." + +```mermaid +graph TD + subgraph "Open Telekom Cloud (OTC)" + subgraph "Control Plane" + Dev[DevOps Engineer] -->|Triggers| Pipeline[Deployment Pipeline] + Pipeline -->|Executes| TF[Terraform] + end + + subgraph "Provisioned Infrastructure" + TF -->|Provisions| CCE[(CCE K8s Cluster)] + TF -->|Provisions| RDS[(RDS PostgreSQL)] + TF -->|Provisions| Redis[(DCS Redis)] + TF -->|Provisions| S3[(OBS S3 Bucket)] + TF -->|Provisions| CSS[(CSS Elasticsearch)] + end + + subgraph "Application Layer (on CCE K8s)" + Pipeline -->|Helm Chart| Argo[ArgoCD] + Argo -->|Deploys| ForgejoApp[Forgejo] + end + + CCE -- Runs --> Argo + CCE -- Runs --> ForgejoApp + ForgejoApp -->|Connects| RDS + ForgejoApp -->|Connects| Redis + ForgejoApp -->|Connects| S3 + ForgejoApp -->|Connects| CSS + end +``` + +### Migration History + +The initial environment was a manual setup on the Open Sovereign Cloud (OSC). Once the automation stack (Terraform/ArgoCD) was matured, the platform was migrated to the current OTC environment. + +## Application Extensions + +### Core Functionality + +Beyond standard Git versioning, the platform utilizes: + +* **Releases:** Hosting binaries for software distribution (e.g., Edge Connect CLI). +* **CI/CD:** Extensive pipeline usage for build, test, and deployment automation. +* **Note on Issues:** While initially used, issue tracking was migrated to JIRA to align with the broader IPCEI program standards. + +### GARM (Git-based Actions Runner Manager) + +The primary technical innovation was the integration of [GARM](./actions/runner-orchestration.md) to enable ephemeral, scalable runners. This required extending Forgejo's capabilities to support GitHub-compatible runner registration and webhook events. + +## Development Methodology & Contributions + +### Workflow + +* **Branching Strategy:** Trunk-based development was utilized to ensure rapid integration. +* **Collaboration:** The team adopted **Mob Programming**. This practice proved essential for knowledge sharing and onboarding junior developers, creating a resilient and high-intensity learning environment. +* **Versions:** The platform evolved from Forgejo v7/8 to the current v11.0.3-edp1. An upgrade is pending to leverage the latest upstream GARM features. + +### Open Source Contributions + +We actively contributed our extensions back to the upstream Forgejo project in [a list of Codeberg.org pull requests](../../governance/_index.md#forgejo) + + +### Artifact Caching (Pull-Through Proxy) + +We implemented a feature allowing Forgejo to act as a pull-through proxy for remote container registries, optimizing bandwidth and build speeds. + +* [Source Code Branch: refactor-remote-registry-client](https://edp.buildth.ing/DevFW/edp-forgejo/src/branch/refactor-remote-registry-client) + +## Key Performance Indicators (KPIs) + +These KPIs measure the effectiveness of the Forgejo setup and quantify our strategic commitment to the Forgejo community. + +| KPI | Description | Target / Benchmark | +| :--- | :--- | :--- | +| **Deployment Frequency** | Frequency of successful pipeline executions. | High (Daily/On-demand) | +| **Artifact Cache Hit Rate** | Percentage of build requests served by the local Forgejo proxy. | > 90% (Reduced external traffic) | +| **Upstream Contribution** | Percentage of GARM-related features contributed back to Codeberg. | 100% (No vendor lock-in) | +| **PR Resolution Time** | Average time for upstream community review and merge. | < 14 days (Healthy collaboration) | diff --git a/content/en/docs/components/forgejo/actions/runner-orchestration.md b/content/en/docs/components/forgejo/actions/runner-orchestration.md index 151fff8..00fcb0c 100644 --- a/content/en/docs/components/forgejo/actions/runner-orchestration.md +++ b/content/en/docs/components/forgejo/actions/runner-orchestration.md @@ -80,19 +80,32 @@ description: GARM ## Architecture -[Optional: Add architectural diagrams and descriptions] +The primary technical innovation was the integration of **[GARM](https://github.com/cloudbase/garm)** to enable ephemeral, scalable runners. This required extending Forgejo's capabilities to support GitHub-compatible runner registration and webhook events. -### Component Architecture (C4) +**Workflow Architecture:** -[Add C4 Container or Component diagrams showing the internal structure] +1. **Event:** A workflow event occurs in Forgejo. +2. **Trigger:** A webhook notifies GARM. +3. **Provisioning:** GARM spins up a fresh, ephemeral runner. +4. **Execution:** The runner registers via the API, executes the job, and is terminated immediately after, ensuring a clean build environment. -### Sequence Diagrams - -[Add sequence diagrams showing key interaction flows with other components] - -### Deployment Architecture - -[Add infrastructure and deployment diagrams showing how the component is deployed] +```mermaid +sequenceDiagram + participant User + participant Forgejo + participant GARM + participant Runner as Ephemeral Runner + + User->>Forgejo: Push Code / Trigger Event + Forgejo->>GARM: Webhook Event (Workflow Dispatch) + GARM->>Forgejo: Register Runner (via API) + GARM->>Runner: Spin up Instance + Runner->>Forgejo: Request Job + Forgejo->>Runner: Send Job Payload + Runner->>Runner: Execute Steps + Runner->>Forgejo: Report Status + GARM->>Runner: Terminate (Ephemeral) +``` ## Configuration diff --git a/content/en/docs/components/forgejo/forgejo.md b/content/en/docs/components/forgejo/forgejo.md deleted file mode 100644 index c7f0b78..0000000 --- a/content/en/docs/components/forgejo/forgejo.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -title: "Edge Developer Platform: Forgejo Integration & GARM Extension" -date: "2025-11-19" -description: "Technical report on the implementation of the Edge Developer Platform (EDP) using Forgejo, featuring GARM integration for ephemeral CI/CD runners and upstream open-source contributions." -tags: ["Forgejo", "GARM", "CI/CD", "OSS", "Architecture", "Project Report"] -categories: ["Workpackage Results"] -mermaid: true ---- - -## 1. Project Identity & Governance - -### 1.1 Project Scope - -The internal service is officially designated as the **Edge Developer Platform (EDP)**. It is hosted at **[edp.buildth.ing](https://edp.buildth.ing)**. The domain selection followed a democratic team process to establish a unique identity distinct from standard corporate naming conventions. - -**Access Model:** -The platform operates on a hybrid visibility model: - -* **Public Access:** The [`DEVFW-CICD`](https://edp.buildth.ing/DevFW-CICD) organization is publicly accessible, fostering transparency. -* **Private Access:** Sensitive development occurs in restricted organizations (e.g., [`DEVFW`](https://edp.buildth.ing/DevFW)). -* **User Base:** Primary users include the internal development team, with friendly user access granted to the IPCEI team and MMS BT. - -### 1.2 Strategic Selection & Governance - -The decision to utilize **[Forgejo](https://forgejo.org/)** as the core self-hosted Git service was driven by specific strategic requirements: - -* **EU-Based Stewardship:** Forgejo is stewarded by **[Codeberg e.V.](https://docs.codeberg.org/getting-started/what-is-codeberg/)**, a non-profit organization based in Berlin, Germany. This alignment ensures compliance with GDPR and data sovereignty requirements, placing governance under EU jurisdiction rather than US tech entities. -* **License Protection (GPL v3+):** Unlike "Open Core" models, Forgejo uses a copyleft license. This legally protects our custom extensions (such as GARM support) from being appropriated into proprietary software, ensuring the ecosystem remains open. -* **Open Source Strategy:** The platform aligns with the "Public Money, Public Code" philosophy, mandating that funded developments are returned to the community. - ---- - -## 2. Technical Architecture & Deployment - -### 2.1 Infrastructure Stack - -The platform is hosted on the **Open Telekom Cloud (OTC)**. The infrastructure adheres to Infrastructure-as-Code (IaC) principles. - -* **Deployment Method:** The official Forgejo Helm Chart is deployed via **ArgoCD**. -* **Infrastructure Provisioning:** **Terraform** is used to provision all underlying OTC services, including: - * **Container Orchestration**: CCE (Cloud Container Engine): Kubernetes - * **Database:** RDS (Distributed Cache Service): PostgreSQL - * **Caching:** DCS (Distributed Cache Service): Redis - * **Object Storage:** OBS (Object Storage Service, S3-compatible): for user data (avatars, attachments). - * **Search:** CSS (Cloud Search Service): Elasticsearch - -### 2.2 The "Self-Replicating" Pipeline - -A key architectural feature is the ability of the platform to maintain itself. A Forgejo Action can trigger the deployment script, which runs Terraform and syncs ArgoCD, effectively allowing "Forgejo to create/update Forgejo." - -```mermaid -graph TD - subgraph "Open Telekom Cloud (OTC)" - subgraph "Control Plane" - Dev[DevOps Engineer] -->|Triggers| Pipeline[Deployment Pipeline] - Pipeline -->|Executes| TF[Terraform] - end - - subgraph "Provisioned Infrastructure" - TF -->|Provisions| CCE[(CCE K8s Cluster)] - TF -->|Provisions| RDS[(RDS PostgreSQL)] - TF -->|Provisions| Redis[(DCS Redis)] - TF -->|Provisions| S3[(OBS S3 Bucket)] - TF -->|Provisions| CSS[(CSS Elasticsearch)] - end - - subgraph "Application Layer (on CCE K8s)" - Pipeline -->|Helm Chart| Argo[ArgoCD] - Argo -->|Deploys| ForgejoApp[Forgejo] - end - - CCE -- Runs --> Argo - CCE -- Runs --> ForgejoApp - ForgejoApp -->|Connects| RDS - ForgejoApp -->|Connects| Redis - ForgejoApp -->|Connects| S3 - ForgejoApp -->|Connects| CSS - end -``` - -### 2.3 Migration History - -The initial environment was a manual setup on the Open Sovereign Cloud (OSC). Once the automation stack (Terraform/ArgoCD) was matured, the platform was migrated to the current OTC environment. - ---- - -## 3. Application Extensions & GARM Integration - -### 3.1 Core Functionality - -Beyond standard Git versioning, the platform utilizes: - -* **Releases:** Hosting binaries for software distribution (e.g., Edge Connect CLI). -* **CI/CD:** Extensive pipeline usage for build, test, and deployment automation. -* **Note on Issues:** While initially used, issue tracking was migrated to JIRA to align with the broader IPCEI program standards. - -### 3.2 GARM (Git-based Actions Runner Manager) - -The primary technical innovation was the integration of **[GARM](https://github.com/cloudbase/garm)** to enable ephemeral, scalable runners. This required extending Forgejo's capabilities to support GitHub-compatible runner registration and webhook events. - -**Workflow Architecture:** - -1. **Event:** A workflow event occurs in Forgejo. -2. **Trigger:** A webhook notifies GARM. -3. **Provisioning:** GARM spins up a fresh, ephemeral runner. -4. **Execution:** The runner registers via the API, executes the job, and is terminated immediately after, ensuring a clean build environment. - -```mermaid -sequenceDiagram - participant User - participant Forgejo - participant GARM - participant Runner as Ephemeral Runner - - User->>Forgejo: Push Code / Trigger Event - Forgejo->>GARM: Webhook Event (Workflow Dispatch) - GARM->>Forgejo: Register Runner (via API) - GARM->>Runner: Spin up Instance - Runner->>Forgejo: Request Job - Forgejo->>Runner: Send Job Payload - Runner->>Runner: Execute Steps - Runner->>Forgejo: Report Status - GARM->>Runner: Terminate (Ephemeral) -``` - ---- - -## 4. Development Methodology & Contributions - -### 4.1 Workflow - -* **Branching Strategy:** Trunk-based development was utilized to ensure rapid integration. -* **Collaboration:** The team adopted **Mob Programming**. This practice proved essential for knowledge sharing and onboarding junior developers, creating a resilient and high-intensity learning environment. -* **Versions:** The platform evolved from Forgejo v7/8 to the current v11.0.3-edp1. An upgrade is pending to leverage the latest upstream GARM features. - -### 4.2 Open Source Contributions - -We actively contributed our extensions back to the upstream Forgejo project on **[Codeberg.org](https://codeberg.org/)**. - -**Key Pull Requests:** - -* **API Compatibility:** Added GitHub-compatible endpoints for runner registration. - * [PR #9409: Feat: Add endpoints for GARM](https://codeberg.org/forgejo/forgejo/pulls/9409) -* **Webhook Support:** Implemented webhook triggers for workflow events. - * [PR #9803: Feat: Add webhook support for workflow events](https://codeberg.org/forgejo/forgejo/pulls/9803) -* **Ephemeral Runners:** Added support for runners that terminate after a single job. - * [PR #9962: Feat: Support for ephemeral runners](https://codeberg.org/forgejo/forgejo/pulls/9962) - -**Artifact Caching (Pull-Through Proxy):** -We implemented a feature allowing Forgejo to act as a pull-through proxy for remote container registries, optimizing bandwidth and build speeds. - -* [Source Code Branch: refactor-remote-registry-client](https://edp.buildth.ing/DevFW/edp-forgejo/src/branch/refactor-remote-registry-client) - ---- - -## 5. Key Performance Indicators (KPIs) - -These KPIs measure the effectiveness of the Forgejo setup and quantify our strategic commitment to the Forgejo community. - -| KPI | Description | Target / Benchmark | -| :--- | :--- | :--- | -| **Deployment Frequency** | Frequency of successful pipeline executions. | High (Daily/On-demand) | -| **Artifact Cache Hit Rate** | Percentage of build requests served by the local Forgejo proxy. | > 90% (Reduced external traffic) | -| **Upstream Contribution** | Percentage of GARM-related features contributed back to Codeberg. | 100% (No vendor lock-in) | -| **PR Resolution Time** | Average time for upstream community review and merge. | < 14 days (Healthy collaboration) | diff --git a/content/en/docs/components/forgejo/image-1.png b/content/en/docs/components/forgejo/image-1.png new file mode 100644 index 0000000..d247705 Binary files /dev/null and b/content/en/docs/components/forgejo/image-1.png differ diff --git a/content/en/docs/components/forgejo/image.png b/content/en/docs/components/forgejo/image.png new file mode 100644 index 0000000..944674e Binary files /dev/null and b/content/en/docs/components/forgejo/image.png differ diff --git a/content/en/docs/governance/_index.md b/content/en/docs/governance/_index.md index c5ab30c..94a1333 100644 --- a/content/en/docs/governance/_index.md +++ b/content/en/docs/governance/_index.md @@ -51,6 +51,26 @@ Mapping to IPCEI-CIS deliverables and project milestones. Documentation of technology evaluation and selection process for key components (e.g., VictoriaMetrics, GARM, Terraform, ArgoCD). +#### Forgejo + +The internal service is officially designated as the [Edge Developer Platform (EDP)](../components/forgejo/_index.md). It is hosted at **[edp.buildth.ing](https://edp.buildth.ing)**. The domain selection followed a democratic team process to establish a unique identity distinct from standard corporate naming conventions. + +**Solution selection:** + +The decision to utilize **[Forgejo](https://forgejo.org/)** as the core self-hosted Git service was driven by specific strategic requirements: + +* **EU-Based Stewardship:** Forgejo is stewarded by **[Codeberg e.V.](https://docs.codeberg.org/getting-started/what-is-codeberg/)**, a non-profit organization based in Berlin, Germany. This alignment ensures compliance with GDPR and data sovereignty requirements, placing governance under EU jurisdiction rather than US tech entities. +* **License Protection (GPL v3+):** Unlike "Open Core" models, Forgejo uses a copyleft license. This legally protects our custom extensions (such as GARM support) from being appropriated into proprietary software, ensuring the ecosystem remains open. +* **Open Source Strategy:** The platform aligns with the "Public Money, Public Code" philosophy, mandating that funded developments are returned to the community. + +**Access Model:** + +The platform operates on a hybrid visibility model: + +* **Public Access:** The [`DEVFW-CICD`](https://edp.buildth.ing/DevFW-CICD) organization is publicly accessible, fostering transparency. +* **Private Access:** Sensitive development occurs in restricted organizations (e.g., [`DEVFW`](https://edp.buildth.ing/DevFW)). +* **User Base:** Primary users include the internal development team, with friendly user access granted to the IPCEI team and MMS BT. + ### Security Controls Overview of implemented security controls and compliance measures. @@ -65,6 +85,19 @@ Cross-references to Jira tickets, epics, and project tracking for audit trails. Contributions to the Forgejo community and other open-source projects. +#### Forgejo + +We actively contributed our extensions back to the upstream Forgejo project on **[Codeberg.org](https://codeberg.org/)**. + +**Key Pull Requests:** + +* **API Compatibility:** Added GitHub-compatible endpoints for runner registration. + * [PR #9409: Feat: Add endpoints for GARM](https://codeberg.org/forgejo/forgejo/pulls/9409) +* **Webhook Support:** Implemented webhook triggers for workflow events. + * [PR #9803: Feat: Add webhook support for workflow events](https://codeberg.org/forgejo/forgejo/pulls/9803) +* **Ephemeral Runners:** Added support for runners that terminate after a single job. + * [PR #9962: Feat: Support for ephemeral runners](https://codeberg.org/forgejo/forgejo/pulls/9962) + ### External Stakeholders User experience research and feedback integration.