This change scopes all github entities to a github endpoint, allowing
users to have the same repo/org/enterprise created for each endpoint.
This way, if your username is the same on github.com and on your GHES
server, and you have the same repository name or org in both places,
GARM can now handle that situation.
This change also fixes a leaky watcher in the pool manager.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Improper use of time.After can lead to memory leaks if the timer never
gets a chance to fire.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change adds a new websocket endpoint for database events. The events
endpoint allows clients to stream events as they happen in GARM. Events
are defined as a structure containning the event type (create, update, delete),
the database entity involved (instances, pools, repos, etc) and the payload
consisting of the object involved in the event. The payload translates
to the types normally returned by the API and can be deserialized as one
of the types present in the params package.
The events endpoint is a websocket endpoint and it accepts filters as
a simple json send over the websocket connection. The filters allows the
user to specify which entities are of interest, and which operations should
be returned. For example, you may be interested in changes made to pools
or runners, in which case you could create a filter that only returns
update operations for pools. Or update and delete operations.
The filters can be defined as:
{
"filters": [
{
"entity_type": "instance",
"operations": ["update", "delete"]
},
{
"entity_type": "pool"
},
],
"send_everything": false
}
This would return only update and delete events for instances and all events
for pools. Alternatively you can ask GARM to send you everything:
{
"send_everything": true
}
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The websocket client and hub interaction has been simplified a bit.
The hub now acts only as a tee writer to the various clients that
register. Clients must register and unregister explicitly. The hub
is no longer passed in to the client.
Websocket clients now watch for password changes or jwt token expiration
times. Clients are disconnected if auth token expires or if the password
is changed.
Various aditional safety checks have been added.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
GARM has a backoff interval when consuming queued jobs. This backoff
is intended to allow any potential idle runners to pick up a job before
GARM attempts to spin up a new one. This change allows users to set a
custom backoff interval or disable it altogether by setting it to 0.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change uses the database watcher to watch for changes to the
github entities, credentials and controller info.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The watcher uses a buffered channel, so we may get stray events that
we need to consume before generating new ones in the tests.
This is just for testing purposes. In actual use cases, we never expect
to only have one event generated.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Adds a simple database watcher. At this point it's just one process, but
the plan is to allow different implementations that inform the local running
workers of changes that have occured on entities of interest in the database.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change moves the callback_url, metadata_url and webhooks_url from
the config to the database. The goal is to move as much as possible from
the config to the DB, in preparation for a potential refactor that will
allow GARM to scale out. This would allow multiple nodes to share a single
source of truth.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This change removes a check that denies the creation of a pool
if the new pool has the same image and flavor set on the same
provider. The reasoning for that check was that if you need to
create another pool with identical settings to an existing one,
you might as well scale up the min-idle-runners on the old one.
This was done when runner groups were not yet added. This in
turn has forced users to alias images with new names in their
provider, leading to terrible UX. In the end, being too
opinionated in this case has caused more harm than good.
Fixes#245
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
No point in making a DB query if we know we don't want to be able to
delete/update the default endpoint.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Do not rely on the entity object to hold updated or detailed credentials,
fetch them from the DB every time.
This change also ensures that we pass in the user context instead of the
runner context to the DB methods.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
When updating credentials on an entity, we must ensure that the new credentials
belong to the same endpoint as the entity.
When an entity is created, the endpoint is determined by the credentials that
were used during the create operation. From that point forward the entity is
associated with an endpoint, and that cannot change.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Add database models that deal with github credentials. This change
adds models for github endpoints (github.com, GHES, etc). This change
also adds code to migrate config credntials to the DB.
Tests need to be fixed and new tests need to be written. This will come
in a later commit.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
