Commit graph

271 commits

Author SHA1 Message Date
Gabriel Adrian Samfira
cb4d56773f Remove some code, move some code around
Remove code that was just wrapping other functions at this point, and
move some code around. We need to get a better idea what is actually
still needed in the pool manager, to begin to refactor it into something
that can scale out.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-04-01 14:52:37 +00:00
Gabriel Adrian Samfira
36288c65e6 Slightly simplify code
Change instance DB functions from querying by ID to querying by name. Names
are unique in GARM, so we might as well use the name instead of the ID and
spare ourselves the extra query to get the ID when a qorkflow comes in.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-30 18:22:06 +00:00
Gabriel Adrian Samfira
f9f545f060 Remove duplicate code
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-29 18:50:04 +00:00
Gabriel Adrian Samfira
9384e37bb1 Fix tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-28 18:23:49 +00:00
Gabriel Adrian Samfira
0152b21529 Implement some common logic for pool creation
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-28 10:09:20 +00:00
Gabriel Adrian Samfira
39f1be5512 Fix JIT config with empty runner group name
When no runner group is set, do not attempt to resolve the runner group.
Looking for an empty runner group will just return a not found error, which
will make GARM fall back to registration token.

This change fixes that.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-25 18:53:53 +00:00
Gabriel Adrian Samfira
f0080047a3 Remove superfluous function
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-18 10:56:49 +00:00
Gabriel Adrian Samfira
56da6a4437 Slightly better UX when dealing with webhooks
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-18 10:19:16 +00:00
Gabriel Adrian Samfira
9259f84e56 Fix getting webhook URL info
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-18 09:53:34 +00:00
Gabriel Adrian Samfira
cfb68f8928 Check webhook secret for entity
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-18 09:39:07 +00:00
Gabriel Adrian Samfira
bef8f30d63 Fix lint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-17 11:13:14 +00:00
Gabriel Adrian Samfira
fa75ecfa8e Dedupe more code
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-17 10:59:09 +00:00
Gabriel Adrian Samfira
b550d0c5b9 remove extra function
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-17 10:28:35 +00:00
Gabriel Adrian Samfira
1734e6f87c Deduplicate code
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-17 10:21:41 +00:00
Gabriel Adrian Samfira
234f71d9d1 Rename PoolType to GithubEntityType
We'll use GithubEntityType throughout the codebase to determine the
type of operation that is about to take place, so this won't belimited
to determining only pool type. We'll also use this to dedupe the label
scope as well.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-17 06:58:03 +00:00
Gabriel Adrian Samfira
206fe42c73 Remove unused code, update test
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-15 15:48:53 +00:00
Gabriel Adrian Samfira
ac29af6eff Add some unit tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-15 14:35:23 +00:00
Gabriel Adrian Samfira
d7ea80a657 Remove log message
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-15 08:12:16 +00:00
Gabriel Adrian Samfira
cdfda0321a Fix balancer type validation
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-15 07:26:04 +00:00
Gabriel Adrian Samfira
ce3c917ae5 Add pool balancing strategy
This change adds the ability to specify the pool balancing strategy to
use when processing queued jobs. Before this change, GARM would round-robin
through all pools that matched the set of tags requested by queued jobs.

When round-robin (default) is used for an entity (repo, org or enterprise)
and you have 2 pools defined for that entity with a common set of tags that
match 10 jobs (for example), then those jobs would trigger the creation of
a new runner in each of the two pools in turn. Job 1 would go to pool 1,
job 2 would go to pool 2, job 3 to pool 1, job 4 to pool 2 and so on.

When "stack" is used, those same 10 jobs would trigger the creation of a
new runner in the pool with the highest priority, every time.

In both cases, if a pool is full, the next one would be tried automatically.

For the stack case, this would mean that if pool 2 had a priority of 10 and
pool 1 would have a priority of 5, pool 2 would be saturated first, then
pool 1.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-14 20:04:34 +00:00
Gabriel Adrian Samfira
7d33e0f0cf Add job info in runner list
This change adds information about the job a runner is currently handling.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-11 15:46:18 +00:00
Gabriel Adrian Samfira
9a6770c3a3 Allow bypassing Unauthorized error when deleting runner
This change allows users to bypass GitHub Unauthorized errors when removing
github runners. This means that removing runners will now be possible even
if the pool manager is stopped.

There is a new flag added to the runner rm command and to the API that
tells GARM to bypass pool being stopped and any 401 error returned by
GitHub.

This means you will be able to remove the runners from garm and your
provider, but will mean that the runner will still exist in github as
"offline" if the credentials are not updated or the runner manually removed.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-10 15:21:39 +00:00
Gabriel Adrian Samfira
4668461603 Expose the credential type through the API
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-02 17:04:27 +00:00
Gabriel Adrian Samfira
cbb2134f0e Add GitHub App support
This change adds the ability to use GitHub Apps to authenticate against the
GitHub API. This gives us a larger quota for API requests (15k vs 5k for PATs).

Also, each GitHub App has its own quota, whereas PATs share the same user quota.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-03-01 19:47:50 +00:00
Mario Constanti
d332e7a8ca fix: gocritic linter finding
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:40:15 +01:00
Mario Constanti
4c81c16505 fix: goconst linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:39:02 +01:00
Mario Constanti
9e7ac60c09 fix: gosec linter finding
we have to keep crypto/sha1 as github is still using it

Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:34:12 +01:00
Mario Constanti
fd0550eb7f fix: godoc linter findings (TODO comments)
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:33:19 +01:00
Mario Constanti
ee3a670456 fix: var-naming linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:28:39 +01:00
Mario Constanti
7221812dfa fix: remove unused cobra args
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 17:20:05 +01:00
Mario Constanti
9f5c38ef2d fix: unused-parameter linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 16:54:38 +01:00
Mario Constanti
b8a9b6c89b fix: ignore testing package typechecks
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
3fd09f6dcd fix: assignOp linter finding
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
9f405e0e8f fix: ifElseChain linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
b0e3f78fbb fix: godoc linter warnings (TODOs)
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
acc17eafcd fix: receiver-naming linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
f6404456b9 fix: indent-error-flow linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
e5ed45c258 fix: unnecessary conversion linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
0ab86a7e51 fix: unused-parameter linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
3b9f8b555b fix: var-naming linter findings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
bd0b27ab10 fix: gci section warnings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
8fc001f5f6 fix: misspell linter warnings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
b3854eaf18 fix: whitespace linter warnings
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 15:06:53 +01:00
Mario Constanti
d68cc3bf05 fix: add missing metrics for few gh api callS
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-22 05:57:42 +01:00
Mario Constanti
b36b5137b6 feat: count github api calls
introduce metrics counter for github api calls

Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-21 14:22:45 +01:00
Mario Constanti
0faf0927bc feat: count external provider operations
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-21 14:21:41 +01:00
Mario Constanti
b1cbfac08a fix: switch to context.Background() for adminctx
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-20 16:42:10 +01:00
Mario Constanti
2a3e4d6563 fix: pass context.TODO by getting admin context
fix linter warnings

Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-20 16:39:52 +01:00
Mario Constanti
0a53b8f6d8 fix: stop metrics collector ticker on ctx.Done
Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-20 14:57:32 +01:00
Mario Constanti
17d74dfbf0 chore: rework prometheus metrics registration
fail if metric registration panics

Signed-off-by: Mario Constanti <mario.constanti@mercedes-benz.com>
2024-02-20 14:27:27 +01:00