This change caches jobs meant for an entity in the pool manager. This
allows us to avoid querying the db as much and allows us to better determine
when we should scale down.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* The Windows userdata wrapper needs to run the real script with parameters
that allow running a downloaded script and in a non-interactive way.
* The metadata endpoint to get the root CA bundle only worked for pools.
This change fixes it for scale sets as well.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add template api endpoints
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Added template bypass
Pools and scale sets will automatically migrate to the new template
system for runner install scripts. If a pool or a scale set cannot be
migrate, it is left alone. It is expected that users set a runner install
template manually for scenarios we don't yet have a template for (windows
on gitea for example).
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Integrate templates with pool create/update
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add webapp integration with templates
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add unit tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Populate all relevant context fields
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Update dependencies
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix lint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Validate uint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add CLI template management
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Some editor improvements and bugfixes
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix scale set return values post create
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix template websocket events filter
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
---------
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds some more cache helper functions, additional tests,
vastly improves memory usage when loading instances and cleans up some
code.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
On github, attempt to use the scaleset API to list all runners without
pagination. This will avoid missing runners and accidentally removing them.
Fall back to paginated API if we can't use the scaleset API.
Add ability to retrieve all instances from cache, for an entity.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Github will remove inactive scale sets after 7 days. This change
ensures the scale set exists in github before spinning up the listener.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Although runner names are unique, we still have an ID on the model
which is used as a primary key. We should allow using that ID to
reference a runner in the API.
This change allows users to specify ID or runner name.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a single page application front-end to GARM. It uses
a generated REST client, built from the swagger definitions, the websocket
interface for live updates of entities and eager loading of everything
except runners, as users may have many runners and we don't want to load
hundreds of runners in memory.
Proper pagination should be implemented in the API, in future commits,
to avoid loading lots of elements for no reason.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
There seems to be a change in the scale set message. It now includes
a jobID and sets the runner request ID to 0. This change adds separate
job ID fields for workflow jobs and scaleset jobs.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Do not look for a name when composing the scale set. Preload may not
have been called on an entity, but we still have the ID, which is the
only thing needed when GetEntity() is called.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* time.NewTicker will panic if the duration is 0. Make it return
early if duration is 0.
* Return a pre-closed channel in Wait() instead of nil. Ensures receiver
will not block forever.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the ability to filter the list of entities returned
by the API by entity owner, name or endpoint, depending on the entity
type.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change renames a lot of variables, types and functions to be more
generic. The goal is to allow GARM to add more forges in the future.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a loop that keeps a cache of credentials rate limits
as reported by the github API. The cache is updated every 30 seconds
and is purely informational for the user.
This change also adds some caching improvements. Functions that return
values from the cache as lists, will now sort by ID or creation date.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Adds a periodic cleanup function that cross checks runners between github,
the provider and the GARM database. If an inconsistency is found, GARM will
attempt to fix it.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This adds the workers needed to start listening for scale set messages.
There is no handling of messages yet.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The locking logic was added to its own package as it may need to be used
by other parts of the code.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change moves the github client to a subpackage in utils
and adds the scaleset github client code.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a backoff mechanism when deleting github runners.
If the delete operation fails, we record the event and retry with
a geometric progression of 1.5 starting from 5 seconds, which is the
pool consolidation timeout.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The User option in the systemd unit file used when JIT configs are enabled
had a hardcoded value of "runner". This change fixes that oversight.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
