From 8dcf1005256291dcce072f28692942fe16fc3e39 Mon Sep 17 00:00:00 2001 From: Stephan Lo Date: Thu, 10 Oct 2024 14:10:21 +0200 Subject: [PATCH 1/5] feat(use-cases): added socksshop fork from mkdev - looks interesting --- content/en/docs/concepts/3_use-cases/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/concepts/3_use-cases/_index.md b/content/en/docs/concepts/3_use-cases/_index.md index c27f9d9..83abc4d 100644 --- a/content/en/docs/concepts/3_use-cases/_index.md +++ b/content/en/docs/concepts/3_use-cases/_index.md @@ -30,6 +30,8 @@ Deploy and develop the famous socks shops: * https://medium.com/@wadecharley703/socks-shop-microservices-application-deployment-on-the-cloud-cd1017cce1c0 +* See also mkdev fork: https://github.com/mkdev-me/microservices-demo + ### Humanitec Demos * https://github.com/poc-template-org/node-js-sample From a8dcf6304cb2b4b751a00c36bb719258f915a116 Mon Sep 17 00:00:00 2001 From: Patrick Sy Date: Thu, 10 Oct 2024 18:35:31 +0200 Subject: [PATCH 2/5] feat(cnoe-verification): added description of the findings researching the validation and verification capabilities in the CNOE framework Refs: #IPCEICIS-467 --- content/en/docs/solution/tools/CNOE/_index.md | 3 + .../docs/solution/tools/CNOE/verification.md | 68 +++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100644 content/en/docs/solution/tools/CNOE/_index.md create mode 100644 content/en/docs/solution/tools/CNOE/verification.md diff --git a/content/en/docs/solution/tools/CNOE/_index.md b/content/en/docs/solution/tools/CNOE/_index.md new file mode 100644 index 0000000..0011792 --- /dev/null +++ b/content/en/docs/solution/tools/CNOE/_index.md @@ -0,0 +1,3 @@ +--- +title: CNOE +--- diff --git a/content/en/docs/solution/tools/CNOE/verification.md b/content/en/docs/solution/tools/CNOE/verification.md new file mode 100644 index 0000000..a39774c --- /dev/null +++ b/content/en/docs/solution/tools/CNOE/verification.md @@ -0,0 +1,68 @@ +--- +title: Validation and Verification +weigth: 100 +--- + +## Definition + +The CNOE docs do somewhat interchange validation and verification but for the +most part they adhere to the general definition: + +> Validation is used when you check your approach before actually executing an +> action. + +Examples: + +- Form validation before processing the data +- Compiler checking syntax +- Rust's borrow checker + +> Verification describes testing if your 'thing' complies with your spec + +Examples: + +- Unit tests +- Testing availability (ping, curl health check) +- Checking a ZKP of some computation + +--- + +## In CNOE + +It seems that both validation and verification within the CNOE framework are +not actually handled by some explicit component but should be addressed +throughout the system and workflows. + +As stated in the [docs](https://cnoe.io/docs/intro/capabilities/validation), +validation takes place in all parts of the stack by enforcing strict API usage +and policies (signing, mitigations, security scans etc, see usage of kyverno +for example), and using code generation (proven code), linting, formatting, +LSP. Consequently, validation of source code, templates, etc is more a best +practice rather than a hard fact or feature and it is up to the user +to incorporate them into their workflows and pipelines. This is probably +due to the complexity of the entire stack and the individual properties of +each component and applications. + +Verification of artifacts and deployments actually exists in a somewhat similar +state. The current CNOE reference-implementation does not provide sufficient +verification tooling. + +However, as stated in the [docs](https://cnoe.io/docs/reference-implementation/integrations/verification) +within the framework `cnoe-cli` is capable of extremely limited verification of +artifacts within kubernetes. The same verification is also available as a step +within a backstage +[plugin](https://github.com/cnoe-io/plugin-scaffolder-actions). This is pretty +much just a wrapper of the cli tool. The tool consumes CRD-like structures +defining the state of pods and CRDs and checks for their existence within a +live cluster ([example](https://github.com/cnoe-io/cnoe-cli/blob/main/pkg/cmd/prereq/ack-s3-prerequisites.yaml)). + +Depending on the aspiration of 'verification' this check is rather superficial +and might only suffice as an initial smoke test. Furthermore, it seems like the +feature is not actually used within the CNOE stacks repo. + +For a live product more in depth verification tools and schemes are necessary +to verify the correct configuration and authenticity of workloads, which is, in +the context of traditional cloud systems, only achievable to a limited degree. + +Existing tools within the stack, e.g. Argo, provide some verification +capabilities. But further investigation into the general topic is necessary. From ac17c07503552564668eb82f3228e8ffd0e64c2f Mon Sep 17 00:00:00 2001 From: Patrick Sy Date: Thu, 10 Oct 2024 20:25:58 +0200 Subject: [PATCH 3/5] feat(http-routing): added docs describing routing of http traffic in the platform Refs: #IPCEICIS-468 --- .../solution/tools/idpbuilder/http-routing.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 content/en/docs/solution/tools/idpbuilder/http-routing.md diff --git a/content/en/docs/solution/tools/idpbuilder/http-routing.md b/content/en/docs/solution/tools/idpbuilder/http-routing.md new file mode 100644 index 0000000..21e709d --- /dev/null +++ b/content/en/docs/solution/tools/idpbuilder/http-routing.md @@ -0,0 +1,57 @@ +--- +title: Http Routing +weight: 100 +--- + +### Routing switch + +The idpbuilder supports creating platforms using either path based or subdomain +based routing: + +```shell +idpbuilder create --log-level debug --package https://github.com/cnoe-io/stacks//ref-implementation +``` + +```shell +idpbuilder create --use-path-routing --log-level debug --package https://github.com/cnoe-io/stacks//ref-implementation +``` + +However, even though argo does report all deployments as green eventually, not +the entire demo is actually functional (verification?). This is due to +hardcoded values that for example point to the path-routed location of gitea to +access git repos. Thus, backstage might not be able to access them. + +Within the demo / ref-implementation, a simple search & replace is suggested to +change urls to fit the given environment. But proper scripting/templating could +take care of that as the hostnames and necessary properties should be +available. This is, however, a tedious and repetitive task one has to keep in +mind throughout the entire system, which might lead to an explosion of config +options in the future. Code that addresses correct routing is located in both +the stack templates and the idpbuilder code. + +### Cluster internal routing + +For the most part, components communicate with either the cluster API using the +default DNS or with each other via http(s) using the public DNS/hostname (+ +path-routing scheme). The latter is necessary due to configs that are visible +and modifiable by users. This includes for example argocd config for components +that has to sync to a gitea git repo. Using the same URL for internal and +external resolution is imperative. + +The idpbuilder achieves transparent internal DNS resolution by overriding the +public DNS name in the cluster's internal DNS server (coreDNS). Subsequently, +within the cluster requests to the public hostnames resolve to the IP of the +internal ingress controller service. Thus, internal and external requests take +a similar path and run through proper routing (rewrites, ssl/tls, etc). + +### Conclusion + +One has to keep in mind that some specific app features might not +work properly or without haxx when using path based routing (e.g. docker +registry in gitea). Futhermore, supporting multiple setup strategies will +become cumbersome as the platforms grows. We should probably only support one +type of setup to keep the system as simple as possible, but allow modification +if necessary. + +DNS solutions like `nip.io` or the already used `localtest.me` mitigate the +need for path based routing From 952d170d8321574222669d9270ff80fa01f9518f Mon Sep 17 00:00:00 2001 From: Stephan Lo Date: Fri, 11 Oct 2024 09:34:06 +0000 Subject: [PATCH 4/5] refactor(solution): section 'solution' refactored into a dedicated 'CNOE' subsection --- .../cicd-pipeline/review-stl.md | 11 +++++++++++ .../en/docs/solution/tools/Backstage/_index.md | 14 +++++--------- .../tools/{ => CNOE}/CNOE-competitors/_index.md | 9 +++++---- content/en/docs/solution/tools/CNOE/_index.md | 1 + .../docs/solution/tools/CNOE/idpbuilder/_index.md | 6 ++++++ .../{ => CNOE}/idpbuilder/installation/_index.md | 0 .../idpbuilder/kyverno integration/_index.md | 0 .../idpbuilder/kyverno integration/kyverno.png | Bin .../en/docs/solution/tools/CNOE/verification.md | 1 + .../en/docs/solution/tools/idpbuilder/_index.md | 6 ------ 10 files changed, 29 insertions(+), 19 deletions(-) create mode 100644 content/en/docs/concepts/4_digital-platforms/platform-components/cicd-pipeline/review-stl.md rename content/en/docs/solution/tools/{ => CNOE}/CNOE-competitors/_index.md (96%) create mode 100644 content/en/docs/solution/tools/CNOE/idpbuilder/_index.md rename content/en/docs/solution/tools/{ => CNOE}/idpbuilder/installation/_index.md (100%) rename content/en/docs/solution/tools/{ => CNOE}/idpbuilder/kyverno integration/_index.md (100%) rename content/en/docs/solution/tools/{ => CNOE}/idpbuilder/kyverno integration/kyverno.png (100%) delete mode 100644 content/en/docs/solution/tools/idpbuilder/_index.md diff --git a/content/en/docs/concepts/4_digital-platforms/platform-components/cicd-pipeline/review-stl.md b/content/en/docs/concepts/4_digital-platforms/platform-components/cicd-pipeline/review-stl.md new file mode 100644 index 0000000..ed5e701 --- /dev/null +++ b/content/en/docs/concepts/4_digital-platforms/platform-components/cicd-pipeline/review-stl.md @@ -0,0 +1,11 @@ + + +# Gitops changes the definition of 'Delivery' or 'Deployment' + +We have Gitops these days .... so there is a desired state of an environment in a repo and a reconciling mechanism done by Gitops to enforce this state on the environemnt. + +There is no continuous whatever step inbetween ... Gitops is just 'overwriting' (to avoid saying 'delivering' or 'deploying') the environment with the new state. + +This means whatever quality ensuring steps have to take part before 'overwriting' have to be defined as state changer in the repos, not in the environments. + +Conclusio: I think we only have three contexts, or let's say we don't have the contect 'continuous delivery' \ No newline at end of file diff --git a/content/en/docs/solution/tools/Backstage/_index.md b/content/en/docs/solution/tools/Backstage/_index.md index df22096..ef168be 100644 --- a/content/en/docs/solution/tools/Backstage/_index.md +++ b/content/en/docs/solution/tools/Backstage/_index.md @@ -1,9 +1,5 @@ -+++ -title = "Backstage" -weight = 2 -[params] - author = 'evgenii.dominov@telekom.de' - date = '2024-09-36' -+++ - -Here you will find information about Backstage, it's plugins and usage tutorials \ No newline at end of file +--- +title: Backstage +weight: 2 +description: Here you will find information about Backstage, it's plugins and usage tutorials +--- diff --git a/content/en/docs/solution/tools/CNOE-competitors/_index.md b/content/en/docs/solution/tools/CNOE/CNOE-competitors/_index.md similarity index 96% rename from content/en/docs/solution/tools/CNOE-competitors/_index.md rename to content/en/docs/solution/tools/CNOE/CNOE-competitors/_index.md index 2f8772e..22a54ea 100644 --- a/content/en/docs/solution/tools/CNOE-competitors/_index.md +++ b/content/en/docs/solution/tools/CNOE/CNOE-competitors/_index.md @@ -1,7 +1,8 @@ -+++ -title = "Analysis of the CNOE competitors" -weight = 1 -+++ +--- +title: Analysis of CNOE competitors +weight: 1 +description: We compare CNOW - which we see as an orchestrator - with other platform orchestring tools like Kratix and Humanitc +--- ## Kratix diff --git a/content/en/docs/solution/tools/CNOE/_index.md b/content/en/docs/solution/tools/CNOE/_index.md index 0011792..3d41c12 100644 --- a/content/en/docs/solution/tools/CNOE/_index.md +++ b/content/en/docs/solution/tools/CNOE/_index.md @@ -1,3 +1,4 @@ --- title: CNOE +description: CNOE is a platform building orchestrator, which we choosed at least to start in 2024 with to build the EDF --- diff --git a/content/en/docs/solution/tools/CNOE/idpbuilder/_index.md b/content/en/docs/solution/tools/CNOE/idpbuilder/_index.md new file mode 100644 index 0000000..72291e2 --- /dev/null +++ b/content/en/docs/solution/tools/CNOE/idpbuilder/_index.md @@ -0,0 +1,6 @@ +--- +title: idpbuilder +weight: 3 +description: Here you will find information about idpbuilder installation and usage +--- + diff --git a/content/en/docs/solution/tools/idpbuilder/installation/_index.md b/content/en/docs/solution/tools/CNOE/idpbuilder/installation/_index.md similarity index 100% rename from content/en/docs/solution/tools/idpbuilder/installation/_index.md rename to content/en/docs/solution/tools/CNOE/idpbuilder/installation/_index.md diff --git a/content/en/docs/solution/tools/idpbuilder/kyverno integration/_index.md b/content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/_index.md similarity index 100% rename from content/en/docs/solution/tools/idpbuilder/kyverno integration/_index.md rename to content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/_index.md diff --git a/content/en/docs/solution/tools/idpbuilder/kyverno integration/kyverno.png b/content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/kyverno.png similarity index 100% rename from content/en/docs/solution/tools/idpbuilder/kyverno integration/kyverno.png rename to content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/kyverno.png diff --git a/content/en/docs/solution/tools/CNOE/verification.md b/content/en/docs/solution/tools/CNOE/verification.md index a39774c..af36de9 100644 --- a/content/en/docs/solution/tools/CNOE/verification.md +++ b/content/en/docs/solution/tools/CNOE/verification.md @@ -1,6 +1,7 @@ --- title: Validation and Verification weigth: 100 +description: How does CNOE ensure equality between actual and desired state --- ## Definition diff --git a/content/en/docs/solution/tools/idpbuilder/_index.md b/content/en/docs/solution/tools/idpbuilder/_index.md deleted file mode 100644 index e302c08..0000000 --- a/content/en/docs/solution/tools/idpbuilder/_index.md +++ /dev/null @@ -1,6 +0,0 @@ -+++ -title = "idpbuilder" -weight = 3 -+++ - -Here you will find information about idpbuilder installation and usage \ No newline at end of file From a5a7c84b6eb0808525dd886e7ec01b0b9457aab4 Mon Sep 17 00:00:00 2001 From: Stephan Lo Date: Fri, 11 Oct 2024 10:01:40 +0000 Subject: [PATCH 5/5] refactor(tools): kyverno moved to tools --- .../idpbuilder => }/kyverno integration/_index.md | 8 ++++---- .../idpbuilder => }/kyverno integration/kyverno.png | Bin 2 files changed, 4 insertions(+), 4 deletions(-) rename content/en/docs/solution/tools/{CNOE/idpbuilder => }/kyverno integration/_index.md (91%) rename content/en/docs/solution/tools/{CNOE/idpbuilder => }/kyverno integration/kyverno.png (100%) diff --git a/content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/_index.md b/content/en/docs/solution/tools/kyverno integration/_index.md similarity index 91% rename from content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/_index.md rename to content/en/docs/solution/tools/kyverno integration/_index.md index 7dea80a..12ca83e 100644 --- a/content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/_index.md +++ b/content/en/docs/solution/tools/kyverno integration/_index.md @@ -1,7 +1,7 @@ -+++ -title = "Kyverno integration" -weight = 4 -+++ +--- +title: Kyverno +description: Kyverno is a policy engine for Kubernetes designed to enforce, validate, and mutate configurations of Kubernetes resources +--- ## Kyverno Overview diff --git a/content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/kyverno.png b/content/en/docs/solution/tools/kyverno integration/kyverno.png similarity index 100% rename from content/en/docs/solution/tools/CNOE/idpbuilder/kyverno integration/kyverno.png rename to content/en/docs/solution/tools/kyverno integration/kyverno.png