This change adds metrics for rate limits. Rate limits are now recorded
via a rate limit check loop (as before), but in addition, we are now
taking the rate limit info that gets returned in all github responses
and we're recording that as it happens as opposed to every 30 seconds.
The loop remains to update rate limits even for credentials that are
used rarely.
This change also adds a credentials details page in the webUI.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* The "CA Certificate Bundle (Optional)" showed no indication of a
certificate being selected. This change fixes that.
* The gitea tools cache worker should not fall back to the default releases
page if the custom page set by the user returned an error.
* Selecting "Use Internal Tools Metadata" in the gitea endpoint edit modal
now greys out the "Tools Metadata URL (optional)" text field.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the ability to manage garm-agent tools downloads. Users
can:
* Set an upstream releases page (github releases api)
* Enable sync from upstream. In this case, GARM will automatically download
garm-agent tools from the releases page and save them in the internal
object store
* Manually upload tools. Manually uploaded tools for an OS/arch combination
will never be overwritten by auto-sync. Usrs will need to delete manually
uploaded tools to enable sync for that os/arch release.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a new "agent mode" to GARM. The agent enables GARM to
set up a persistent websocket connection between the garm server and the
runners it spawns. The goal is to be able to easier keep track of state,
even without subsequent webhooks from the forge.
The Agent will report via websockets when the runner is actually online,
when it started a job and when it finished a job.
Additionally, the agent allows us to enable optional remote shell between
the user and any runner that is spun up using agent mode. The remote shell
is multiplexed over the same persistent websocket connection the agent
sets up with the server (the agent never listens on a port).
Enablement has also been done in the web UI for this functionality.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Use uncompressed tools for gitea. Gitea compresses using .xz, including for
Windows, which does not have a native, built-in tool to uncompress that
format.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The code that attempts to fetch tools from upstream, must return the error
if the current setting is the upstream repo.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds 2 new options to gitea forge endpoints:
* Tools metadata URL
* Use internal tools URLs
By default, GARM looks in the releases page of the gitea arc_runner
to determine where it can download the runner binary from for a particular
OS/arch. The tools metadata URL option can be set on an endpoint and can point
to a mirror of the upstream repo. The requirement is that the asset names
exactly mirror upstream naming conventions.
The second option disables GARM calling out to the tools metadata URL entirely.
GARM has some hardcoded values for nightly binaries. If this option is checked,
GARM will use those values, without making any kind of outgoing API call to
determine availability. This is useful in air-gapped environments.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add template api endpoints
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Added template bypass
Pools and scale sets will automatically migrate to the new template
system for runner install scripts. If a pool or a scale set cannot be
migrate, it is left alone. It is expected that users set a runner install
template manually for scenarios we don't yet have a template for (windows
on gitea for example).
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Integrate templates with pool create/update
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add webapp integration with templates
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add unit tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Populate all relevant context fields
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Update dependencies
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix lint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Validate uint
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Add CLI template management
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Some editor improvements and bugfixes
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix scale set return values post create
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* Fix template websocket events filter
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
---------
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Filter out gitea tools to only consider archived downloads. This
should help in situations where bandwidth is more important than
CPU time used to unarchive the tools.
Also a drive by fix for scale sets cleanup.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Do not look for a name when composing the scale set. Preload may not
have been called on an entity, but we still have the ID, which is the
only thing needed when GetEntity() is called.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* time.NewTicker will panic if the duration is 0. Make it return
early if duration is 0.
* Return a pre-closed channel in Wait() instead of nil. Ensures receiver
will not block forever.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds the ability to filter the list of entities returned
by the API by entity owner, name or endpoint, depending on the entity
type.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
* We were passing the wrong type to GORM for events
* We now expose entity events in the API and CLI
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change renames a lot of variables, types and functions to be more
generic. The goal is to allow GARM to add more forges in the future.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a loop that keeps a cache of credentials rate limits
as reported by the github API. The cache is updated every 30 seconds
and is purely informational for the user.
This change also adds some caching improvements. Functions that return
values from the cache as lists, will now sort by ID or creation date.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds an update routine in the cache worker, for github tools
downloads.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
