Commit graph

857 commits

Author SHA1 Message Date
Gabriel Adrian Samfira
bc4285dc80 Update garm-provider-common
Use the websocket reader from within garm-provider-common.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-08-03 16:07:21 +00:00
Gabriel
a5b15789a1
Merge pull request #281 from gabriel-samfira/cleanup-websocket-urls
Rename websocket URLs
2024-08-03 01:29:05 +03:00
Gabriel Adrian Samfira
79c1e47427 Rename websocket URLs
Given that we now have multiple websocket URLs (logs and events), this
change categorizes them under the same prefix.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-08-02 22:19:51 +00:00
Gabriel
7fe2a3ad34
Merge pull request #280 from gabriel-samfira/generalize-ws-reader
Generalize the websocket reader
2024-08-03 00:27:43 +03:00
Gabriel Adrian Samfira
5c45f54ef4 Generalize the websocket reader
This change adds a new message handler that users of the reader can use
to handle websocket messages. Packages should never print to console by
themselves.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-08-02 21:18:31 +00:00
Gabriel
6835419daf
Merge pull request #279 from gabriel-samfira/scope-entities-to-endpoint
Fix: Scope entities to endpoint
2024-08-01 13:19:42 +03:00
Gabriel Adrian Samfira
cc6e985629 Fix: Scope entities to endpoint
This change scopes all github entities to a github endpoint, allowing
users to have the same repo/org/enterprise created for each endpoint.

This way, if your username is the same on github.com and on your GHES
server, and you have the same repository name or org in both places,
GARM can now handle that situation.

This change also fixes a leaky watcher in the pool manager.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-29 17:35:57 +00:00
Gabriel
f64ffa8d92
Merge pull request #277 from gabriel-samfira/slight-docs-reword
Reword the section about labels
2024-07-07 00:05:27 +03:00
Gabriel Adrian Samfira
411dcea4fb Reword the section about labels
This change removes a stray line in the quickstart and rewords
the section about labels a bit.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-06 21:02:54 +00:00
Gabriel
59de9cdf93
Merge pull request #275 from gabriel-samfira/add-event-stream
Add event stream
2024-07-05 23:06:09 +03:00
Gabriel Adrian Samfira
2554f70b89 Replace time.After with time.NewTimer
Improper use of time.After can lead to memory leaks if the timer never
gets a chance to fire.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
49f1b7aa14 Set read limit to 16 KB
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
a2c606b87d Simplify logic
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
5b24b9e56b Fix shadowing bug in job create function
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
5c5b2256bb Create common utility function for ws
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
246f826b76 Validate filter payload
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
9f8659abd6 Add events websocket endpoint
This change adds a new websocket endpoint for database events. The events
endpoint allows clients to stream events as they happen in GARM. Events
are defined as a structure containning the event type (create, update, delete),
the database entity involved (instances, pools, repos, etc) and the payload
consisting of the object involved in the event. The payload translates
to the types normally returned by the API and can be deserialized as one
of the types present in the params package.

The events endpoint is a websocket endpoint and it accepts filters as
a simple json send over the websocket connection. The filters allows the
user to specify which entities are of interest, and which operations should
be returned. For example, you may be interested in changes made to pools
or runners, in which case you could create a filter that only returns
update operations for pools. Or update and delete operations.

The filters can be defined as:

{
  "filters": [
    {
      "entity_type": "instance",
      "operations": ["update", "delete"]
    },
    {
      "entity_type": "pool"
    },
  ],
  "send_everything": false
}

This would return only update and delete events for instances and all events
for pools. Alternatively you can ask GARM to send you everything:

{
  "send_everything": true
}

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel Adrian Samfira
dd1740c189 Refactor the websocket client and add fixes
The websocket client and hub interaction has been simplified a bit.
The hub now acts only as a tee writer to the various clients that
register. Clients must register and unregister explicitly. The hub
is no longer passed in to the client.

Websocket clients now watch for password changes or jwt token expiration
times. Clients are disconnected if auth token expires or if the password
is changed.

Various aditional safety checks have been added.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:55:35 +00:00
Gabriel
ca7f20b62d
Merge pull request #276 from gabriel-samfira/add-version-to-api-response
Add version to controller info response
2024-07-05 15:54:45 +03:00
Gabriel Adrian Samfira
42ae3c52d1 Add version to controller info response
This change adds the GARM server version to the controller info response.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-05 12:49:16 +00:00
Gabriel
dcee09281b
Merge pull request #274 from gabriel-samfira/make-durations-configuratble
Allow configuration of job backoff interval
2024-07-02 12:03:05 +03:00
Gabriel Adrian Samfira
892a62bfe4 Allow configuration of job backoff interval
GARM has a backoff interval when consuming queued jobs. This backoff
is intended to allow any potential idle runners to pick up a job before
GARM attempts to spin up a new one. This change allows users to set a
custom backoff interval or disable it altogether by setting it to 0.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-07-01 10:27:31 +00:00
Gabriel
8f0d44742e
Merge pull request #269 from gabriel-samfira/remove-update-state
Use watcher and get rid of RefreshState()
2024-06-21 16:51:54 +03:00
Gabriel Adrian Samfira
daaca0bd8f Use watcher and get rid of RefreshState()
This change uses the database watcher to watch for changes to the
github entities, credentials and controller info.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-21 13:47:48 +00:00
Gabriel
38127af747
Merge pull request #268 from gabriel-samfira/lower-log-level
Lower the log level of ignored jobs
2024-06-20 18:35:24 +03:00
Gabriel Adrian Samfira
1dfa74efd8 Lower the log level of ignored jobs
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-20 15:30:25 +00:00
Gabriel
98b8a7f15f
Merge pull request #267 from fabi200123/fix-set-watcher
Remove duplicate of watcher.SetWatcher()
2024-06-20 17:59:25 +03:00
Fabian Fulga
61c9470bcf Remove duplicate of watcher.SetWatcher 2024-06-20 17:37:11 +03:00
Gabriel
5eef3b68cb
Merge pull request #266 from gabriel-samfira/consume-watcher-events
Consume events before testing
2024-06-20 15:50:30 +03:00
Gabriel Adrian Samfira
8bf979fe1c Consume events before testing
The watcher uses a buffered channel, so we may get stray events that
we need to consume before generating new ones in the tests.

This is just for testing purposes. In actual use cases, we never expect
to only have one event generated.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-20 12:38:36 +00:00
Gabriel
c188a6f2c1
Merge pull request #263 from gabriel-samfira/add-database-watcher
Add database watcher
2024-06-20 15:01:53 +03:00
Gabriel Adrian Samfira
230f002902 Add more tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-20 11:01:57 +00:00
Gabriel Adrian Samfira
b7d138d2ac Add notifications for jobs
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-19 13:44:24 +00:00
Gabriel Adrian Samfira
5f07bc2d7c Check if producer was registered
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-19 12:40:56 +00:00
Gabriel Adrian Samfira
0c8c6f5668 Add more tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-19 12:27:48 +00:00
Gabriel Adrian Samfira
cc9ecf5847 Add more tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-18 19:28:34 +00:00
Gabriel Adrian Samfira
8a79d9e8f9 Add more watcher tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-18 17:45:48 +00:00
Gabriel Adrian Samfira
37f6434ed8 Fix race condition and add some tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-18 16:42:24 +00:00
Gabriel Adrian Samfira
b51683f1ae Add some tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-17 19:42:50 +00:00
Gabriel
a66cbccdd9
Merge pull request #261 from fabi200123/integration-tests
Refactor Integration tests
2024-06-17 13:57:42 +03:00
Fabian Fulga
9d4c0a953c Adding garm test suite 2024-06-17 12:36:00 +03:00
Gabriel Adrian Samfira
6051629810 Add watcher for github creds and endpoints
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-14 20:35:02 +00:00
Gabriel Adrian Samfira
7f9db2e413 Send notify on update controller
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-14 20:24:45 +00:00
Gabriel Adrian Samfira
8d57fc8fa2 Add rudimentary database watcher
Adds a simple database watcher. At this point it's just one process, but
the plan is to allow different implementations that inform the local running
workers of changes that have occured on entities of interest in the database.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-14 19:47:12 +00:00
Gabriel
4c7c9b0e1e
Merge pull request #259 from gabriel-samfira/fix-readme-typos
Fix typos
2024-06-08 12:38:50 +03:00
Gabriel Adrian Samfira
1058c40c43 Fix typos
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-07 18:24:58 +00:00
Gabriel
214cb05072
Merge pull request #256 from gabriel-samfira/move-urls-to-db
Move URLs from default section of config to DB
2024-06-07 13:09:04 +03:00
Gabriel Adrian Samfira
aea328bab9 Remove URLs from sample config
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-07 10:04:34 +00:00
Gabriel Adrian Samfira
37ae7520b8 Update docs
Update the quickstart and the "using garm" sections.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-07 09:27:38 +00:00
Gabriel Adrian Samfira
3992f97d8c Fix tests and make URLs optional in config
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2024-06-07 09:27:38 +00:00