DevFW-CICD/website-and-documentation
23
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added core stacks docs
Some checks failed
Hugo Site Tests / test (push) Failing after 1s
Details
ci / build (push) Successful in 54s
Details

Browse source
This commit is contained in:
Manuel Ganter 2025-12-16 10:25:14 +01:00
parent c7ec23e9a0
commit 876cfd9ba0
No known key found for this signature in database
1 changed files with 425 additions and 72 deletions
Download patch file Download diff file Expand all files Collapse all files

497
content/en/docs/components/orchestration/stacks/core.md
View file

@ -1,127 +1,480 @@
--- ---
title: "Core" title: "Core"
linkTitle: "Core" linkTitle: "Core"
weight: 30 weight: 10
description: Core description: >
Essential infrastructure components for GitOps, database management, and single sign-on
--- ---
{{% alert title="Draft" color="warning" %}}
**Editorial Status**: This page is currently being developed.
* **Jira Ticket**: [TBD]
* **Assignee**: [Name or Team]
* **Status**: Draft
* **Last Updated**: YYYY-MM-DD
* **TODO**:
* [ ] Add detailed component description
* [ ] Include usage examples and code samples
* [ ] Add architecture diagrams
* [ ] Review and finalize content
{{% /alert %}}
## Overview ## Overview
[Detailed description of the component - what it is, what it does, and why it exists] The Core stack provides foundational infrastructure components required by all other Edge Developer Platform stacks. It establishes the base layer for continuous deployment, database services, and centralized authentication, enabling a secure, scalable platform architecture.
The Core stack deploys ArgoCD for GitOps orchestration, CloudNativePG for PostgreSQL database management, and Dex for OpenID Connect single sign-on capabilities.
## Key Features ## Key Features
* [Feature 1] * **GitOps Continuous Deployment**: ArgoCD manages declarative infrastructure and application deployments
* [Feature 2] * **Database Operator**: CloudNativePG provides enterprise-grade PostgreSQL clusters for platform services
* [Feature 3] * **Single Sign-On**: Dex offers centralized OIDC authentication across platform components
* **Automated Synchronization**: Self-healing deployments with automatic drift correction
## Purpose in EDP * **Role-Based Access Control**: Integrated RBAC for secure platform administration
* **TLS Certificate Management**: Automated certificate provisioning and renewal
[Explain the role this component plays in the Edge Developer Platform and how it contributes to the overall platform capabilities]
## Repository ## Repository
**Code**: [Link to source code repository] **Code**: [Core Stack Templates](https://edp.buildth.ing/DevFW-CICD/stacks/src/branch/main/template/stacks/core)
**Documentation**: [Link to component-specific documentation] **Documentation**:
* [ArgoCD Documentation](https://argo-cd.readthedocs.io/)
* [CloudNativePG Documentation](https://cloudnative-pg.io/)
* [Dex Documentation](https://dexidp.io/docs/)
## Getting Started ## Getting Started
### Prerequisites ### Prerequisites
* [Prerequisite 1] * Kubernetes cluster (1.24+)
* [Prerequisite 2] * kubectl configured with cluster access
* Ingress controller (nginx recommended)
* cert-manager for TLS certificate management
* Domain names configured for platform services
### Quick Start ### Quick Start
[Step-by-step guide to get started with this component] The Core stack is deployed as the foundation of the EDP installation:
1. [Step 1] 1. **Trigger Deploy Pipeline**
2. [Step 2] - Go to [Infra Deploy Pipeline](https://edp.buildth.ing/DevFW/infra-deploy/actions?workflow=deploy.yaml)
3. [Step 3] - Click on Run workflow
- Enter a name in "Select environment directory to deploy". This must be DNS Compatible. (if you enter `test-me` then domains will be `argocd.test-me.t09.de`, `dex.test-me.t09.de`)
- Execute workflow
2. **ArgoCD Bootstrap**
The deployment automatically provisions:
- ArgoCD control plane in `argocd` namespace
- CloudNativePG operator in `cloudnative-pg` namespace
- Dex identity provider in `dex` namespace
- Ingress configurations with TLS certificates
- OIDC authentication integration
### Verification ### Verification
[How to verify the component is working correctly] Verify the Core stack deployment:
## Usage Examples
### [Use Case 1]
[Example with code/commands showing common use case]
```bash ```bash
# Example commands # Check ArgoCD installation
kubectl get application -n argocd
kubectl get pods -n argocd
# Verify CloudNativePG operator
kubectl get pods -n cloudnative-pg
kubectl get crd | grep cnpg.io
# Check Dex deployment
kubectl get pods -n dex
kubectl get ingress -n dex
# Verify ingress configurations
kubectl get ingress -n argocd
``` ```
### [Use Case 2] Access ArgoCD at `https://argocd.{DOMAIN}` and authenticate via Dex SSO. Or use username `admin` and the secret inside of kubernetes `argocd/argocd-initial-admin-secret` as password `kubectl get secret -n argocd argocd-initial-admin-secret -ojson | jq -r .data.password | base64 -d`.
[Another common scenario]
## Integration Points
* **[Component A]**: [How it integrates]
* **[Component B]**: [How it integrates]
* **[Component C]**: [How it integrates]
## Architecture ## Architecture
[Optional: Add architectural diagrams and descriptions] ### Component Architecture
### Component Architecture (C4) The Core stack establishes a three-tier foundation:
[Add C4 Container or Component diagrams showing the internal structure] **ArgoCD Control Plane**:
- Application management and GitOps reconciliation
- Multi-repository tracking with automated sync
- Resource health monitoring and drift detection
- Integrated RBAC with SSO authentication
### Sequence Diagrams **CloudNativePG Operator**:
- PostgreSQL cluster lifecycle management
- Automated backup and recovery
- High availability and failover
- Storage provisioning via CSI drivers
[Add sequence diagrams showing key interaction flows with other components] **Dex Identity Provider**:
- OpenID Connect authentication service
- Multiple connector support (Forgejo/Gitea, LDAP, SAML)
- Static client registration for platform services
- Token issuance and validation
### Deployment Architecture ### Networking
[Add infrastructure and deployment diagrams showing how the component is deployed] **Ingress Architecture**:
- nginx ingress controller for external access
- TLS termination with cert-manager integration
- Domain-based routing for platform services
**Kubernetes Services**:
- Internal service communication via ClusterIP
- DNS-based service discovery
- Network policies for security segmentation
## Configuration ## Configuration
[Key configuration options and how to set them] ### ArgoCD Configuration
Deployed via Helm chart v9.1.5 with custom values in `stacks/core/argocd/values.yaml`:
**OIDC Authentication**:
```yaml
configs:
cm:
url: "https://{DOMAIN_ARGOCD}"
oidc.config: |
name: Forgejo
issuer: https://{DOMAIN_DEX}
clientID: controller-argocd-dex
clientSecret: $dex-controller-argocd-dex:dex-controller-argocd-dex
requestedScopes: ["openid", "profile", "email", "groups"]
```
**RBAC Policy**:
```yaml
policy.csv: |
g, DevFW, role:admin
```
**Server Settings**:
- Insecure mode enabled (TLS handled by ingress)
- Annotation-based resource tracking
- 60-second reconciliation timeout
- Resource exclusions for ProviderConfigUsage and CiliumIdentity
### CloudNativePG Configuration
Deployed via Helm chart v0.26.1 with values in `stacks/core/cloudnative-pg/values.yaml`:
**Operator Settings**:
- Namespace: `cloudnative-pg`
- Automated database cluster provisioning
- Custom resource definitions for Cluster, Database, and Pooler resources
**Storage Configuration**:
- Uses `csi-disk` storage class by default
- PVC provisioning for PostgreSQL data
- Backup storage integration (S3-compatible)
### Dex Configuration
Deployed via Helm chart v0.23.0 with values in `stacks/core/dex/values.yaml`:
**Issuer Configuration**:
```yaml
config:
issuer: https://{DOMAIN_DEX}
storage:
type: memory # Use persistent storage for production
oauth2:
skipApprovalScreen: true
alwaysShowLoginScreen: false
```
**Forgejo Connector**:
```yaml
connectors:
- type: gitea
id: forgejo
name: Forgejo
config:
clientID: $FORGEJO_CLIENT_ID
clientSecret: $FORGEJO_CLIENT_SECRET
redirectURI: https://{DOMAIN_DEX}/callback
baseURL: https://edp.buildth.ing
orgs:
- name: DevFW
```
**Static OAuth2 Clients**:
- ArgoCD: `controller-argocd-dex`
- Grafana: `controller-grafana-dex`
### Environment Variables
Core stack services use the following environment variables:
**Domain Configuration**:
- `DOMAIN_ARGOCD`: ArgoCD web interface URL
- `DOMAIN_DEX`: Dex authentication service URL
- `DOMAIN_GITEA`: Forgejo/Gitea repository URL
- `DOMAIN_GRAFANA`: Grafana observability dashboard URL
**Repository Configuration**:
- `CLIENT_REPO_ID`: Repository identifier for stack configurations
- `CLIENT_REPO_DOMAIN`: Git repository domain
- `CLIENT_REPO_ORG_NAME`: Organization name for stack instances
## Usage Examples
### Managing Applications with ArgoCD
Access and manage applications through ArgoCD:
```bash
# Login to ArgoCD CLI
argocd login argocd.${DOMAIN} --sso
# List all applications
argocd app list
# Get application status
argocd app get coder
# Sync application manually
argocd app sync coder
# View application logs
argocd app logs coder
# Diff application state
argocd app diff coder
```
### Creating a PostgreSQL Database
Deploy a PostgreSQL cluster using CloudNativePG:
```yaml
# database-cluster.yaml
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: app-db
namespace: my-app
spec:
instances: 3
storage:
size: 20Gi
storageClass: csi-disk
postgresql:
parameters:
max_connections: "100"
shared_buffers: "256MB"
bootstrap:
initdb:
database: appdb
owner: appuser
```
Apply the configuration:
```bash
kubectl apply -f database-cluster.yaml
# Check cluster status
kubectl get cluster app-db -n my-app
kubectl get pods -n my-app -l cnpg.io/cluster=app-db
# Get connection credentials
kubectl get secret app-db-app -n my-app -o jsonpath='{.data.password}' | base64 -d
```
### Configuring SSO for Applications
Add OAuth2 applications to Dex for SSO integration:
```yaml
# Add to dex values.yaml
staticClients:
- id: my-app-client
redirectURIs:
- 'https://myapp.{DOMAIN}/callback'
name: 'My Application'
secretEnv: MY_APP_CLIENT_SECRET
```
Configure the application to use Dex:
```bash
# Application OIDC configuration
OIDC_ISSUER=https://dex.${DOMAIN}
OIDC_CLIENT_ID=my-app-client
OIDC_CLIENT_SECRET=${MY_APP_CLIENT_SECRET}
OIDC_REDIRECT_URI=https://myapp.${DOMAIN}/callback
```
### Deploying Applications via ArgoCD
Create an ArgoCD Application manifest:
```yaml
# my-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
namespace: argocd
spec:
project: default
source:
repoURL: 'https://github.com/myorg/my-app'
targetRevision: main
path: k8s
destination:
server: 'https://kubernetes.default.svc'
namespace: my-app
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
```
Push it to [stacks instances](https://edp.buildth.ing/DevFW-CICD/stacks-instances) to be picked up by argo
## Integration Points
* **All Stacks**: Core stack is a prerequisite for all other EDP stacks
* **OTC Stack**: Provides ingress-nginx and cert-manager dependencies
* **Coder Stack**: Uses CloudNativePG for workspace database management
* **Forgejo Stack**: Integrates with Dex for SSO and ArgoCD for deployment
* **Observability Stack**: Uses Dex for Grafana authentication and ArgoCD for deployment
* **Provider Stack**: Deploys Terraform providers via ArgoCD
## Troubleshooting ## Troubleshooting
### [Common Issue 1] ### ArgoCD Not Accessible
**Problem**: [Description] **Problem**: Cannot access ArgoCD web interface
**Solution**: [How to fix] **Solution**:
1. Verify ingress configuration:
```bash
kubectl get ingress -n argocd
kubectl describe ingress -n argocd
```
### [Common Issue 2] 2. Check ArgoCD server status:
```bash
kubectl get pods -n argocd
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-server
```
**Problem**: [Description] 3. Verify TLS certificate:
```bash
kubectl get certificate -n argocd
kubectl describe certificate -n argocd
```
**Solution**: [How to fix] 4. Test DNS resolution:
```bash
nslookup argocd.${DOMAIN}
```
## Status ### Dex Authentication Failing
**Maturity**: [Production / Beta / Experimental] **Problem**: SSO login fails or redirects incorrectly
**Solution**:
1. Check Dex logs:
```bash
kubectl logs -n dex -l app.kubernetes.io/name=dex
```
2. Verify Forgejo connector configuration:
```bash
kubectl get secret -n dex
kubectl get configmap -n dex dex -o yaml
```
3. Test Dex issuer endpoint:
```bash
curl https://dex.${DOMAIN}/.well-known/openid-configuration
```
4. Verify OAuth2 client credentials match in both Dex and consuming application
### CloudNativePG Operator Not Running
**Problem**: PostgreSQL clusters fail to provision
**Solution**:
1. Check operator status:
```bash
kubectl get pods -n cloudnative-pg
kubectl logs -n cloudnative-pg -l app.kubernetes.io/name=cloudnative-pg
```
2. Verify CRDs are installed:
```bash
kubectl get crd | grep cnpg.io
kubectl describe crd clusters.postgresql.cnpg.io
```
3. Check operator logs for errors:
```bash
kubectl logs -n cloudnative-pg -l app.kubernetes.io/name=cloudnative-pg --tail=100
```
### Application Sync Failures
**Problem**: ArgoCD applications remain out of sync or fail to deploy
**Solution**:
1. Check application status:
```bash
argocd app get <app-name>
kubectl describe application <app-name> -n argocd
```
2. Review sync operation logs:
```bash
argocd app logs <app-name>
```
3. Verify repository access:
```bash
argocd repo list
argocd repo get <repo-url>
```
4. Check for resource conflicts or missing dependencies:
```bash
kubectl get events -n <app-namespace> --sort-by='.lastTimestamp'
```
### Database Connection Issues
**Problem**: Applications cannot connect to CloudNativePG databases
**Solution**:
1. Verify cluster is ready:
```bash
kubectl get cluster <cluster-name> -n <namespace>
kubectl describe cluster <cluster-name> -n <namespace>
```
2. Check database credentials secret:
```bash
kubectl get secret <cluster-name>-app -n <namespace>
kubectl get secret <cluster-name>-app -n <namespace> -o yaml
```
3. Test connection from a pod:
```bash
kubectl run -it --rm psql-test --image=postgres:16 --restart=Never -- \
psql "$(kubectl get secret <cluster-name>-app -n <namespace> -o jsonpath='{.data.uri}' | base64 -d)"
```
4. Review PostgreSQL logs:
```bash
kubectl logs -n <namespace> <cluster-name>-1
```
## Additional Resources ## Additional Resources
* [Link to external documentation] * [ArgoCD Documentation](https://argo-cd.readthedocs.io/)
* [Link to community resources] * [ArgoCD Best Practices](https://argo-cd.readthedocs.io/en/stable/user-guide/best_practices/)
* [Link to related components] * [CloudNativePG Documentation](https://cloudnative-pg.io/)
* [CloudNativePG Architecture](https://cloudnative-pg.io/documentation/current/architecture/)
## Documentation Notes * [Dex Documentation](https://dexidp.io/docs/)
* [Dex Connectors](https://dexidp.io/docs/connectors/)
[Instructions for team members filling in this documentation - remove this section once complete] * [OpenID Connect Specification](https://openid.net/connect/)