GARM cares about jobs in queued state for anything that requires
decision making. Anything else is purely informational.
This change cleans up all inactionable jobs and refuses to record jobs
that are not already in the database, have an inactionable state and
which do not have a runner we own handling them.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a new "generation" field to pools, scalesets and
runners. The generation field is inherited by runners from scale sets
or pools at the time of creation.
The generation field on scalesets and pools is incremented when the
pool or scale set is updated in a way that might influence how runners
are created (flavor, image, specs, runner groups, etc).
Using this new field, we can determine if existing runners have diverged
from the settings of the pool/scale set that spawned them.
In the CLI we now have a new set of commands available for both
pools and scalesets that lists runners, with an optional --outdated
flag and a new "rotate" flag that removes all idle runners. Optionally
the --outdated flag can be passed to the rotate command to only remove
outdated runners.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the ability to manage garm-agent tools downloads. Users
can:
* Set an upstream releases page (github releases api)
* Enable sync from upstream. In this case, GARM will automatically download
garm-agent tools from the releases page and save them in the internal
object store
* Manually upload tools. Manually uploaded tools for an OS/arch combination
will never be overwritten by auto-sync. Usrs will need to delete manually
uploaded tools to enable sync for that os/arch release.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a new "agent mode" to GARM. The agent enables GARM to
set up a persistent websocket connection between the garm server and the
runners it spawns. The goal is to be able to easier keep track of state,
even without subsequent webhooks from the forge.
The Agent will report via websockets when the runner is actually online,
when it started a job and when it finished a job.
Additionally, the agent allows us to enable optional remote shell between
the user and any runner that is spun up using agent mode. The remote shell
is multiplexed over the same persistent websocket connection the agent
sets up with the server (the agent never listens on a port).
Enablement has also been done in the web UI for this functionality.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the API endpoints, the CLI commands and the web UI elements
needed to manage objects in GARMs internal storage.
This storage system is meant to be used to distribute the garm-agent and as a
single source of truth for provider binaries, when we will add the ability for GARM
to scale out.
Potentially, we can also use this in air gapped systems to distribute the runner binaries
for forges that don't have their own internal storage system (like GHES).
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This commit adds the DB models and functions needed to create, read,
search through, update and delete files within sqlite3.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add template api endpoints
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Added template bypass
Pools and scale sets will automatically migrate to the new template
system for runner install scripts. If a pool or a scale set cannot be
migrate, it is left alone. It is expected that users set a runner install
template manually for scenarios we don't yet have a template for (windows
on gitea for example).
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Integrate templates with pool create/update
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add webapp integration with templates
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add unit tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Populate all relevant context fields
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Update dependencies
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix lint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Validate uint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add CLI template management
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Some editor improvements and bugfixes
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix scale set return values post create
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix template websocket events filter
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
---------
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Although runner names are unique, we still have an ID on the model
which is used as a primary key. We should allow using that ID to
reference a runner in the API.
This change allows users to specify ID or runner name.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a single page application front-end to GARM. It uses
a generated REST client, built from the swagger definitions, the websocket
interface for live updates of entities and eager loading of everything
except runners, as users may have many runners and we don't want to load
hundreds of runners in memory.
Proper pagination should be implemented in the API, in future commits,
to avoid loading lots of elements for no reason.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the ability to filter the list of entities returned
by the API by entity owner, name or endpoint, depending on the entity
type.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change renames a lot of variables, types and functions to be more
generic. The goal is to allow GARM to add more forges in the future.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Close response body in scaleset client
* Wait for message listener loop to exit before attempting restart
* Add LastMessageID field to scaleset model and function to update it
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This adds the workers needed to start listening for scale set messages.
There is no handling of messages yet.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change scopes all github entities to a github endpoint, allowing
users to have the same repo/org/enterprise created for each endpoint.
This way, if your username is the same on github.com and on your GHES
server, and you have the same repository name or org in both places,
GARM can now handle that situation.
This change also fixes a leaky watcher in the pool manager.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Adds a simple database watcher. At this point it's just one process, but
the plan is to allow different implementations that inform the local running
workers of changes that have occured on entities of interest in the database.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
