Patrick.Sy/website-and-documentation
1
0
Fork 0
forked from DevFW-CICD/website-and-documentation
Code Pull requests Activity
website-and-documentation/content/en/docs/components/orchestration/application.md

832 lines
22 KiB
Markdown
Raw Blame History

---
title: "Application Orchestration"
linkTitle: "Application Orchestration"
weight: 30
description: >
Application deployment via CI/CD pipelines and GitOps - Orchestrating application deployments
---
## Overview
Application Orchestration deals with the automation of application deployment and lifecycle management. It encompasses the entire workflow from source code to running application in production.
In the context of IPCEI-CIS, Application Orchestration includes:
- **CI/CD Pipelines**: Automated build, test, and deployment pipelines
- **GitOps Deployment**: Declarative application deployment via ArgoCD
- **Progressive Delivery**: Canary deployments, blue-green deployments
- **Application Configuration**: Environment-specific configuration management
- **Golden Paths**: Standardized deployment templates and workflows
### Target Audience
Application Orchestration is primarily for:
- **Application Developers**: Teams developing and deploying applications
- **DevOps Teams**: Teams responsible for deployment automation
- **Product Teams**: Teams responsible for application lifecycle
## Key Features
### Automated CI/CD Pipelines
Forgejo Actions provides GitHub Actions-compatible CI/CD:
- **Build Automation**: Automatic building of container images
- **Test Automation**: Automated unit, integration, and E2E tests
- **Security Scanning**: Vulnerability scanning of dependencies and images
- **Artifact Publishing**: Publishing to container registries
- **Deployment Triggering**: Automatic deployment after successful build
### GitOps-based Deployment
ArgoCD enables declarative application deployment:
- **Declarative Configuration**: Applications defined as Kubernetes manifests
- **Automated Sync**: Automatic synchronization between Git and cluster
- **Rollback Capability**: Easy rollback to previous versions
- **Multi-Environment**: Consistent deployment across Dev/Test/Prod
- **Health Monitoring**: Continuous monitoring of application health
### Progressive Delivery
Support for advanced deployment strategies:
- **Canary Deployments**: Gradual rollout to subset of users
- **Blue-Green Deployments**: Zero-downtime deployments with instant rollback
- **A/B Testing**: Traffic splitting for feature testing
- **Feature Flags**: Dynamic feature enablement without deployment
### Configuration Management
Flexible configuration for different environments:
- **Environment Variables**: Configuration via environment variables
- **ConfigMaps**: Kubernetes-native configuration
- **Secrets Management**: Secure handling of sensitive data
- **External Secrets**: Integration with external secret stores (Vault, etc.)
## Purpose in EDP
Application Orchestration is the core of developer experience in IPCEI-CIS Edge Developer Platform.
### Developer Self-Service
Developers can deploy applications independently:
- **Self-Service Deployment**: No dependency on operations team
- **Standardized Workflows**: Clear, documented deployment processes
- **Fast Feedback**: Quick feedback through automated pipelines
- **Environment Parity**: Consistent behavior across all environments
### Quality and Security
Automated checks ensure quality and security:
- **Automated Testing**: All changes are automatically tested
- **Security Scans**: Vulnerability scanning of dependencies and images
- **Policy Enforcement**: Automated policy checks (OPA, Kyverno)
- **Compliance**: Auditability of all deployments
### Efficiency and Productivity
Automation increases team efficiency:
- **Faster Time-to-Market**: Faster deployment of new features
- **Reduced Manual Work**: Automation of repetitive tasks
- **Fewer Errors**: Fewer manual mistakes through automation
- **Better Collaboration**: Clear interfaces between Dev and Ops
## Repository
**Forgejo**: [forgejo.org](https://forgejo.org/)
**Forgejo Actions**: [Forgejo Actions Documentation](https://forgejo.org/docs/latest/user/actions/)
**ArgoCD**: [argoproj.github.io/cd](https://argoproj.github.io/cd/)
## Getting Started
### Prerequisites
- **Forgejo Account**: Access to Forgejo instance
- **Kubernetes Cluster**: Target cluster for deployments
- **ArgoCD Access**: Access to ArgoCD instance
- **Git**: For repository management
### Quick Start: Application Deployment
1. **Create Application Repository**
```bash
# Create new repository in Forgejo
git init my-application
cd my-application
# Add application code and Dockerfile
cat > Dockerfile <<EOF
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]
EOF
```
2. **Add CI/CD Pipeline**
Create `.forgejo/workflows/build.yaml`:
```yaml
name: Build and Push
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Registry
uses: docker/login-action@v2
with:
registry: registry.example.com
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v4
with:
context: .
push: ${{ github.event_name == 'push' }}
tags: registry.example.com/my-app:${{ github.sha }}
```
3. **Create Kubernetes Manifests**
Create `k8s/deployment.yaml`:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-application
spec:
replicas: 3
selector:
matchLabels:
app: my-application
template:
metadata:
labels:
app: my-application
spec:
containers:
- name: app
image: registry.example.com/my-app:latest
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: "production"
---
apiVersion: v1
kind: Service
metadata:
name: my-application
spec:
selector:
app: my-application
ports:
- port: 80
targetPort: 3000
```
4. **Configure ArgoCD Application**
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-application
namespace: argocd
spec:
project: default
source:
repoURL: https://forgejo.example.com/myteam/my-application
targetRevision: main
path: k8s
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
```
5. **Deploy**
```bash
# Commit and push
git add .
git commit -m "Add application and deployment configuration"
git push origin main
# ArgoCD will automatically deploy the application
argocd app sync my-application --watch
```
## Usage Examples
### Use Case 1: Multi-Environment Deployment
Deploy application to multiple environments:
**Repository Structure:**
```text
my-application/
├── .forgejo/
│ └── workflows/
│ └── build.yaml
├── base/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomization.yaml
├── overlays/
│ ├── dev/
│ │ ├── kustomization.yaml
│ │ └── patches.yaml
│ ├── staging/
│ │ ├── kustomization.yaml
│ │ └── patches.yaml
│ └── production/
│ ├── kustomization.yaml
│ └── patches.yaml
```
**Kustomize Base** (`base/kustomization.yaml`):
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
commonLabels:
app: my-application
```
**Environment Overlay** (`overlays/production/kustomization.yaml`):
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../base
namespace: production
replicas:
- name: my-application
count: 5
images:
- name: my-app
newTag: v1.2.3
patches:
- patches.yaml
```
**ArgoCD Applications for each environment:**
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-application-prod
namespace: argocd
spec:
project: default
source:
repoURL: https://forgejo.example.com/myteam/my-application
targetRevision: main
path: overlays/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
```
### Use Case 2: Canary Deployment
Progressive rollout with canary strategy:
**Argo Rollouts Canary:**
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: my-application
spec:
replicas: 10
strategy:
canary:
steps:
- setWeight: 10
- pause: {duration: 5m}
- setWeight: 30
- pause: {duration: 5m}
- setWeight: 60
- pause: {duration: 5m}
- setWeight: 100
selector:
matchLabels:
app: my-application
template:
metadata:
labels:
app: my-application
spec:
containers:
- name: app
image: registry.example.com/my-app:v2.0.0
```
### Use Case 3: Feature Flags
Dynamic feature control without deployment:
**Application Code with Feature Flag:**
```javascript
const Unleash = require('unleash-client');
const unleash = new Unleash({
url: 'http://unleash.platform/api/',
appName: 'my-application',
customHeaders: {
Authorization: process.env.UNLEASH_API_TOKEN
}
});
// Use feature flag
if (unleash.isEnabled('new-checkout-flow')) {
// New checkout implementation
renderNewCheckout();
} else {
// Old checkout implementation
renderOldCheckout();
}
```
## Integration Points
### Forgejo Integration
Forgejo serves as central source code management and CI/CD platform:
- **Source Control**: Git repositories for application code
- **CI/CD Pipelines**: Forgejo Actions for automated builds and tests
- **Container Registry**: Built-in container registry for images
- **Webhook Integration**: Triggers for external systems
- **Pull Request Workflows**: Code review and approval processes
### ArgoCD Integration
ArgoCD handles declarative application deployment:
- **GitOps Sync**: Continuous synchronization with Git state
- **Health Monitoring**: Application health status monitoring
- **Rollback Support**: Easy rollback to previous versions
- **Multi-Cluster**: Deployment to multiple clusters
- **UI and CLI**: Web interface and command-line access
### Observability Integration
Integration with monitoring and logging:
- **Metrics**: Prometheus metrics from applications
- **Logs**: Centralized log collection via Loki/ELK
- **Tracing**: Distributed tracing with Jaeger/Tempo
- **Alerting**: Alert rules for application issues
## Architecture
### Application Deployment Flow
```text
┌──────────────┐
│ Developer │
└──────┬───────┘
│ git push
▼
┌─────────────────────────────────────────┐
│ Forgejo │
│ │
│ ┌────────────────────────────────┐ │
│ │ Git Repository │ │
│ │ - Application Code │ │
│ │ - Kubernetes Manifests │ │
│ │ - Pipeline Configuration │ │
│ └─────────┬──────────────────────┘ │
│ │ │
│ │ triggers │
│ ▼ │
│ ┌────────────────────────────────┐ │
│ │ Forgejo Actions (CI/CD) │ │
│ │ - Build Container Image │ │
│ │ - Run Tests │ │
│ │ - Security Scans │ │
│ │ - Push to Registry │ │
│ └─────────┬──────────────────────┘ │
└────────────┼────────────────────────────┘
│
│ image pushed
▼
┌─────────────────────────────────────────┐
│ Container Registry │
└─────────────────────────────────────────┘
│
│ watches manifests
▼
┌─────────────────────────────────────────┐
│ ArgoCD │
│ │
│ - Monitors Git Repository │
│ - Detects Changes │
│ - Syncs to Kubernetes │
│ - Reports Health Status │
└─────────────┬───────────────────────────┘
│
│ deploys
▼
┌─────────────────────────────────────────┐
│ Kubernetes Cluster │
│ │
│ ┌────────────────────────────────┐ │
│ │ Running Application │ │
│ │ - Pods │ │
│ │ - Services │ │
│ │ - Ingress │ │
│ └────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
### CI/CD Pipeline Architecture
Typical Forgejo Actions pipeline stages:
1. **Checkout**: Clone source code
2. **Build**: Compile application and dependencies
3. **Test**: Run unit and integration tests
4. **Security Scan**: Scan dependencies and code for vulnerabilities
5. **Build Image**: Create container image
6. **Push Image**: Push to container registry
7. **Update Manifests**: Update Kubernetes manifests with new image tag
8. **Notify**: Send notifications on success/failure
## Configuration
### Forgejo Actions Configuration
Example for Node.js application:
```yaml
name: CI/CD Pipeline
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
env:
REGISTRY: registry.example.com
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test
- name: Run linter
run: npm run lint
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-results.sarif'
build-and-push:
needs: [test, security]
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to Registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=sha,prefix={{branch}}-
- name: Build and push
uses: docker/build-push-action@v4
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
```
### ArgoCD Application Configuration
Complete configuration example:
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-application
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://forgejo.example.com/myteam/my-application
targetRevision: main
path: k8s/overlays/production
# Kustomize options
kustomize:
version: v5.0.0
images:
- my-app=registry.example.com/my-app:v1.2.3
destination:
server: https://kubernetes.default.svc
namespace: production
# Sync policy
syncPolicy:
automated:
prune: true # Delete resources not in Git
selfHeal: true # Override manual changes
allowEmpty: false # Don't delete everything on empty repo
syncOptions:
- CreateNamespace=true
- PruneLast=true
- RespectIgnoreDifferences=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
# Ignore differences (avoid sync loops)
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas # Ignore if HPA manages replicas
```
## Troubleshooting
### Pipeline Fails
**Problem**: Forgejo Actions pipeline fails
**Solution**:
```bash
# 1. Check pipeline logs in Forgejo UI
# Navigate to: Repository → Actions → Select failed run
# 2. Check runner status
# In Forgejo: Site Admin → Actions → Runners
# 3. Check runner logs
kubectl logs -n forgejo-runner deployment/act-runner
# 4. Test pipeline locally with act
act -l # List available jobs
act -j build # Run specific job
```
### ArgoCD Application OutOfSync
**Problem**: Application shows "OutOfSync" status
**Solution**:
```bash
# 1. Check differences
argocd app diff my-application
# 2. View sync status details
argocd app get my-application
# 3. Manual sync
argocd app sync my-application
# 4. Hard refresh (ignore cache)
argocd app sync my-application --force
# 5. Check for ignored differences
argocd app get my-application --show-operation
```
### Application Deployment Fails
**Problem**: Application pod crashes after deployment
**Solution**:
```bash
# 1. Check pod status
kubectl get pods -n production
# 2. View pod logs
kubectl logs -n production deployment/my-application
# 3. Describe pod for events
kubectl describe pod -n production <pod-name>
# 4. Check resource limits
kubectl top pod -n production
# 5. Rollback via ArgoCD
argocd app rollback my-application
```
### Image Pull Errors
**Problem**: Kubernetes cannot pull container image
**Solution**:
```bash
# 1. Verify image exists
docker pull registry.example.com/my-app:v1.2.3
# 2. Check image pull secret
kubectl get secret -n production regcred
# 3. Create image pull secret if missing
kubectl create secret docker-registry regcred \
--docker-server=registry.example.com \
--docker-username=user \
--docker-password=password \
-n production
# 4. Reference secret in deployment
kubectl patch deployment my-application -n production \
-p '{"spec":{"template":{"spec":{"imagePullSecrets":[{"name":"regcred"}]}}}}'
```
## Best Practices
### Golden Path Templates
Provide standardized templates for common use cases:
1. **Web Application Template**: Node.js, Python, Go web services
2. **API Service Template**: RESTful API with OpenAPI
3. **Batch Job Template**: Kubernetes CronJob configurations
4. **Microservice Template**: Service mesh integration
Example repository template structure:
```text
application-template/
├── .forgejo/
│ └── workflows/
│ ├── build.yaml
│ ├── test.yaml
│ └── deploy.yaml
├── k8s/
│ ├── base/
│ └── overlays/
├── src/
│ └── ...
├── Dockerfile
├── README.md
└── .gitignore
```
### Deployment Checklist
Before deploying to production:
- ✅ All tests passing
- ✅ Security scans completed
- ✅ Resource limits defined
- ✅ Health checks configured
- ✅ Monitoring and alerts set up
- ✅ Backup strategy defined
- ✅ Rollback plan documented
- ✅ Team notified about deployment
### Configuration Management
- Use ConfigMaps for non-sensitive configuration
- Use Secrets for sensitive data
- Use External Secrets Operator for vault integration
- Never commit secrets to Git
- Use environment-specific overlays (Kustomize)
- Document all configuration options
## Status
**Maturity**: Production
**Stability**: Stable
**Support**: Internal Platform Team
## Additional Resources
### Forgejo
- [Forgejo Documentation](https://forgejo.org/docs/latest/)
- [Forgejo Actions Guide](https://forgejo.org/docs/latest/user/actions/)
- [Forgejo API Reference](https://forgejo.org/docs/latest/api/)
### ArgoCD
- [ArgoCD Documentation](https://argo-cd.readthedocs.io/)
- [ArgoCD Best Practices](https://argo-cd.readthedocs.io/en/stable/user-guide/best_practices/)
- [ArgoCD Sync Waves](https://argo-cd.readthedocs.io/en/stable/user-guide/sync-waves/)
### GitOps
- [GitOps Principles](https://opengitops.dev/)
- [GitOps Patterns](https://www.gitops.tech/)
- [Kubernetes Deployment Strategies](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy)
### CI/CD
- [GitHub Actions Documentation](https://docs.github.com/en/actions) (Forgejo Actions compatible)
- [Docker Best Practices](https://docs.docker.com/develop/dev-best-practices/)
- [Container Security Best Practices](https://kubernetes.io/docs/concepts/security/pod-security-standards/)
View git blame Copy permalink