Commit graph

75 commits

Author SHA1 Message Date
Gabriel Adrian Samfira
e441b6ce89 Switch to log/slog
This change switches GARM to the new structured logging standard
library. This will allow us to set log levels and reduce some of
the log spam.

Given that we introduced new knobs to tweak logging, the number of
config options for logging now warrants it's own section.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-01-05 23:46:40 +00:00
Gabriel Adrian Samfira
2a5e2409b2 Add system-info instance callback
Allow runners to update their own system information. Runners can now send
back os_name, os_version and agent_id back as part of a POST to
CALLBACK_URL/system-info/.

The goal is to get better info in regard to the actual OS that's running
and to move the agent_id from the status updates to the system-info callback.

The status updates should be used only to send back info about the status of
the installation process.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-01-04 15:23:43 +00:00
Gabriel Adrian Samfira
d09f12dfd8 Add force delete runner
This branch adds the ability to forcefully remove a runner from GARM.

When the operator wishes to manually remove a runner, the workflow is as
follows:

* Check that the runner exists in GitHub. If it does, attempt to
  remove it. An error here indicates that the runner may be processing
  a job. In this case, we don't continue and the operator gets immediate
  feedback from the API.
* Mark the runner in the database as pending_delete
* Allow the consolidate loop to reap it from the provider and remove it
  from the database.

Removing the instance from the provider is async. If the provider errs out,
GARM will keep trying to remove it in perpetuity until the provider succedes.

In situations where the provider is misconfigured, this will never happen, leaving
the instance in a permanent state of pending_delete.

A provider may fail for various reasons. Either credentials have expired, the
API endpoint has changed, the provider is misconfigured or the operator may just
have removed it from the config before cleaning up the runners. While some cases
are recoverable, some are not. We cannot have a situation in which we cannot clean
resources in garm because of a misconfiguration.

This change adds the pending_force_delete instance status. Instances marked with
this status, will be removed from GARM even if the provider reports an error.

The GARM cli has been modified to give new meaning to the --force-remove-runner
option. This option in the CLI is no longer mandatory. Instead, setting it will mark
the runner with the new pending_force_delete status. Omitting it will mark the runner
with the old status of pending_delete.

Fixes: #160

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-10-12 06:15:36 +00:00
Gabriel Adrian Samfira
e53c271337 Add metadata URLs
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-09-24 13:50:21 +00:00
Gabriel Adrian Samfira
1268507ce6 Add jit config routes
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-09-24 13:50:20 +00:00
Gabriel Adrian Samfira
a26907fb91 Add root CA bundle metadata URL
Thic change adds a metadata endpoint that returns a list of root CA
certificates a runner must install in order to be able to validate all
relevant API endpoints it may require. This includes any GHES API that
runs on a self signed certificate.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-28 09:44:18 +00:00
Gabriel Adrian Samfira
1c0ff85a0d
Add flag to toggle webhook management
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:02 +03:00
Gabriel Adrian Samfira
c00048e128
Add optional keepWebhook flag when removing an entity
The user can opt to not delete the webhook (if installed) when removing
the entity from garm. Garm will only ever try to remove a webhook that
exactly matches the URL that is composed of the base webhook URL configured
in the config.toml file and the unique controller ID that is generated
when the controller is first installed. It should be safe to remove the
webhook when the entity is removed.

Of course, this behavior can be disabled.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:02 +03:00
Gabriel Adrian Samfira
bb6ee9c668
Add keepWebhook flag when deleting entities
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:01 +03:00
Gabriel Adrian Samfira
779afe980e
Add webhook show, return info and some fixes
* Added a webhook show command. This gives us info about the webhook and
    if it is installed.
  * Return webhook info when installing the webhook
  * Small typo fixes.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:01 +03:00
Gabriel Adrian Samfira
dbd41f518d
Add CLI webhook enablement
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:01 +03:00
Gabriel Adrian Samfira
7ce3f007b0
Add functions to (un)install webhooks for orgs and repos
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:01 +03:00
Gabriel Adrian Samfira
f2796f1d5a
Add admin required middleware and webhook endpoint
* Add a new middleware that tests for admin access
  * Add a new controller ID suffixed webhook endpoint. This will be used
    to accept webhook events on a webhook URL that is suffixed with our own
    controller ID.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-22 09:39:01 +03:00
Gabriel Adrian Samfira
99539edde7 Add controller info
This change adds a new controller info endpoint and associated client and
CLI command. The controller info endpoint returns information about controller
status and configuration.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-08-12 22:47:50 +00:00
Gabriel Adrian Samfira
f0332b7578 Run go generate
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-22 22:42:57 +00:00
Gabriel Adrian Samfira
e775c9c11d Move most of util package
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-22 22:39:17 +00:00
Gabriel Adrian Samfira
ed651bb7d0 Move errors to external package
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-22 22:26:47 +00:00
Mihaela Balutoiu
d2b78793f5 Update generated swagger client code
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-19 18:06:16 +03:00
Mihaela Balutoiu
802c7527fe Add more swagger annotations for apiserver
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-19 18:06:13 +03:00
Gabriel
ddd6976115
Merge pull request #139 from mihaelabalutoiu/fix-swagger-annotations
Fix swagger annotations
2023-07-18 20:03:36 +03:00
Mihaela Balutoiu
e3275b4dc1 Update generated swagger client code
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-18 19:29:02 +03:00
Mihaela Balutoiu
9ba3ae487b Update the swagger annotations for the apiserver
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-18 19:27:21 +03:00
Gabriel
0d08808c81
Merge pull request #138 from ionutbalutoiu/small-fixes
Small fixes to the GARM api server
2023-07-18 17:25:30 +03:00
Ionut Balutoiu
572094700d Fix calling /api/v1/first-run without ending /
The endpoint endpoint `/api/v1/first-run` only works when we
have ending `/`.

This commit fixes this.

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2023-07-18 17:01:17 +03:00
Mihaela Balutoiu
520f468500 Update generated swagger client code
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-18 15:20:31 +03:00
Mihaela Balutoiu
f13d19b4ec Add more functionality to swagger client library
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-18 15:19:53 +03:00
Mihaela Balutoiu
db21fb6ea9 Update generated swagger client code
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-17 14:24:54 +03:00
Mihaela Balutoiu
98e415bf11 Add more swagger annotations to apiserver
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-17 12:00:51 +03:00
Mihaela Balutoiu
a871931210 Fix apiserver/routers/routers.go typo
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-07-10 19:20:19 +03:00
Ionut Balutoiu
e5b52bea34 Update swagger client generated code
Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2023-07-05 14:25:31 +03:00
Ionut Balutoiu
a952729082 Update swagger:meta API info
* Include details about what API consumes / produces.
* Add info about GARM authentication.

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2023-07-05 14:24:37 +03:00
Ionut Balutoiu
5891216179 Update generated swagger client code
Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2023-07-05 13:48:01 +03:00
Ionut Balutoiu
ca878507b5 Add more functionality to swagger client library
* Update `go:generate` annotations to use stable swagger tag instead
  of relying on the `swagger` CLI tool already installed.
* Rename the following route IDs from repositories:
    * `Create` -> `CreateRepo`
    * `List` -> `ListRepos`
    * `Get` -> `GetRepo`
    * `Delete` -> `DeleteRepo`
  The swagger CLI spec validation will fail if the route IDs are not unique.
* Fully implement the all the API calls to:
    * `/api/v1/repositories`
    * `/api/v1/instances`

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2023-07-05 13:47:58 +03:00
Gabriel Adrian Samfira
86ed06d6ff Rename UpdateRepositoryParams to UpdateEntityParams
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-05 00:00:24 +00:00
Gabriel Adrian Samfira
a526c1024c Various fixes
* enable foreign key constraints on sqlite
  * on delete cascade for addresses and status messages
  * add debug server config option
  * fix rr allocation

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
1287a93cf2 Add job list to API and CLI
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-07-03 07:46:20 +00:00
Ionut Balutoiu
4a68f3b046 Generate initial swagger API client library code
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-06-30 19:04:13 +03:00
Ionut Balutoiu
d122f293cf Add swagger annotations to apiserver
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
2023-06-30 19:03:57 +03:00
Gabriel Adrian Samfira
829db87f15
Rename module
This change renames the module from "garm" to "github.com/cloudbase/garm".

This will make it easier to consume public functions defined in garm, by
external applications, without having to resort to replace.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-12 16:01:49 +02:00
Gabriel Adrian Samfira
8d17498ab8
Remove caches, retry fetching the hostname
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-01-29 16:03:20 +02:00
Gabriel Adrian Samfira
8f56f51598 Move some code around
Move the metrics code into its own package.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-01-27 14:57:25 +00:00
Michael Kuhnt
6cd18ff1fd
improve access to controller info 2023-01-26 22:13:18 +01:00
Michael Kuhnt
6a032bfaa2
metrics: fix review findings 2023-01-26 15:46:27 +01:00
Michael Kuhnt
ee659f509f
feat: add prometheus metrics & endpoint 2023-01-26 14:15:16 +01:00
Gabriel Adrian Samfira
e93b6d73e5
Sanitize log entries
While most of these log entries come from either github or our own
database, it's still a good idea to sanitize them.
2023-01-23 18:01:46 +02:00
Gabriel Adrian Samfira
687c48127b
Fix some linting issues and add verify-vendor
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-01-20 21:55:52 +02:00
Gabriel Adrian Samfira
3a92a5be0e Some cleanup and safety checks
* Add logging middleware
  * Remove some noise from logs
  * Add some safety checks when managing runners
2022-12-29 16:50:11 +00:00
Gabriel Adrian Samfira
dfc3c1ff5c Update providers to use new token URL
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-12-05 21:29:30 +00:00
Gabriel Adrian Samfira
0869073906 Define a metadata subrouter
Define a metadata subrouter and move the token endpoint there. We may
end up needing multiple endpoints for various purposes in the future.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-12-02 19:48:38 +00:00
Gabriel Adrian Samfira
a078645ab2
Add token endpoint
This change adds a github registration endpoint that instances can use
to fetch a github registration token.

This change also invalidates disables access to an instance to the token
and status updates endpoints once the instance transitions from
"pending" or "installing" to any other state.
2022-12-01 18:00:22 +02:00