DevFW-CICD/garm
15
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
385 commits 3 branches 10 tags 27 MiB
Commit graph

385 commits

This branch
This branch
All branches
Author SHA1 Message Date
Gabriel Adrian Samfira
b6a593b20c fix sqlite connection URI 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
3796c25228 Amend some log messages 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
8f604c8a32 Fix test 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
bf90eb323a Add back update locks 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
4ec684b493 use journal mode WAL 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
4a2ba68867 Use the same db connection in runner 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
b3c754c015 set max connections to 1 when using sqlite3 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
faf4f18e0e Remove lock for update 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
b52f107bde Update log messages 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
c04a93dde9 Add basic round robin for pools 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
4b9c20e1be Reduce timeout to 10 seconds 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
28360fd662 Do not record jobs not meant for us 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
a15a91b974 Break lock and lower scale down timeout 
Break the lock on a job if it's still queued and the runner that it
triggered was assigned to another job. This may cause leftover runners
to be created, but we scale those down in ~3 minutes.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
1287a93cf2 Add job list to API and CLI 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
b6a02db446 Remove completed jobs and slight optimization 
* Removes completed jobs from the db
  * Skip ensure min idle runners for pools with min idle runners set to 0

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
5153738359 Small fixes 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel Adrian Samfira
fbffd8157b Add job tracking 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:46:20 +00:00
Gabriel
8abf94ef85
Merge pull request #121 from gabriel-samfira/use-errgroup 
Replace wait implementation with errgroup
 2023-07-03 10:45:48 +03:00
Gabriel Adrian Samfira
5bca63eeb1 Replace wait implementation with errgroup 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-07-03 07:40:57 +00:00
Gabriel
32464c7b6a
Merge pull request #120 from SystemKeeper/add-performance-considerations 
Add doc about performance considerations
 2023-07-02 00:19:41 +03:00
Marcel Müller
4d09a7977a Add doc about performance considerations 
Signed-off-by: Marcel Müller <marcel-mueller@gmx.de>
 2023-07-01 16:29:56 +02:00
Gabriel
681702e3db
Merge pull request #119 from mihaelabalutoiu/add-swagger-client 
Generate client library via swagger
 2023-06-30 22:30:19 +03:00
Ionut Balutoiu
4a68f3b046 Generate initial swagger API client library code 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-30 19:04:13 +03:00
Ionut Balutoiu
d122f293cf Add swagger annotations to apiserver 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-30 19:03:57 +03:00
Gabriel
b506df20da
Merge pull request #118 from gabriel-samfira/add-logging 
Add logging and fix backoff loop
 2023-06-30 09:54:45 +03:00
Gabriel Adrian Samfira
67b871488d Log the actual error 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-30 09:14:10 +03:00
Gabriel Adrian Samfira
0a27acd818 Remove extra loop and add logging 
* removes an extra loop. The fetch tools loop does the same job
  * add a lot of log messages

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-30 08:52:16 +03:00
Gabriel Adrian Samfira
7358beb2b9 Merge Unlock() and UnlockAndDelete() 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-30 08:48:29 +03:00
Gabriel
aa44bf4f98
Merge pull request #116 from ionutbalutoiu/sighup-log-rotate 
Rotate log file on SIGHUP
 2023-06-27 20:12:55 +03:00
Ionut Balutoiu
721cbefd6a Rotate log file on SIGHUP 
Add functionality to rotate the log file when `SIGHUP` signal is received.

Also, a doc is added with few details about logging.

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
 2023-06-27 20:04:20 +03:00
Gabriel
c45bd1d1d8
Merge pull request #115 from gabriel-samfira/use-su-to-install-runner 
Use su to install the runner
 2023-06-27 18:04:05 +03:00
Gabriel Adrian Samfira
c0347b0e9b Use su to install the runner 
This change executes the runner install script provided via userdata using
su. The script itself has also been changed to assume it's running as the
runner user.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-27 07:15:31 +00:00
Gabriel
442e76e278
Merge pull request #112 from gabriel-samfira/use-select-with-wait 
Refactor pool manager loop
 2023-06-23 18:49:48 +03:00
Gabriel Adrian Samfira
1edb9247a8 Add per instance mux 
Lock operations per instance name. This should avoid go routines trying
to update the same instance when operations may be slow.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-23 15:43:31 +00:00
Gabriel Adrian Samfira
a9cf5127a9 More granular loops, update go-github 
This commit adds:

  * more granular loops for various operations
  * update go-github to latest version
  * skip trying to fetch runner info for canceled or skipped jobs
  * loops use waitgroups to signal exit

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-23 08:16:41 +00:00
Gabriel Adrian Samfira
f84da48f5b Accomodate older versions of curl 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-22 22:46:09 +00:00
Gabriel Adrian Samfira
4921692ee2 Wrap errgroup in select 
This commit:

  * swaps WaitGroups with errgroups
  * wraps errgroup.Wait() in a select to prevent situations in which an
    operation takes a long time and prevents garm from being restarted.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-23 01:07:55 +03:00
Gabriel
f5226aec17
Merge pull request #111 from mihaelabalutoiu/fix-pools-typo 
Fix `runner/pools.go` typo
 2023-06-21 12:35:42 +03:00
Mihaela Balutoiu
d698e2815e Fix runner/pools.go typo 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-21 10:56:51 +03:00
Gabriel
1ebbbd6915
Merge pull request #110 from mihaelabalutoiu/add-more-pools-unit-tests 
Add more `runner/pools.go` unit tests
 2023-06-20 23:25:58 +03:00
Mihaela Balutoiu
00c0ada0aa Add more runner/pools.go unit tests 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-19 23:39:07 +03:00
Gabriel Adrian Samfira
8f31db1f67
Reword title 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-16 17:36:27 +03:00
Gabriel Adrian Samfira
3a5c939612
Add slack link 
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
 2023-06-16 12:00:32 +03:00
Gabriel
e44786f1d2
Merge pull request #109 from mihaelabalutoiu/fix-pools-typo 
Fix `runner/pools.go` typo
 2023-06-15 15:57:48 +03:00
Mihaela Balutoiu
7ac2455379 Fix runner/pools.go typo 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-15 14:12:07 +03:00
Gabriel
05168ac994
Merge pull request #106 from gabriel-samfira/reap-failed-agent 
Reap failed agent
 2023-06-14 22:19:20 +03:00
Gabriel
22b410c9b7
Merge pull request #108 from mihaelabalutoiu/add-db-users-unit-tests 
Add more test cases for the `/database/sql/users.go`
 2023-06-14 20:05:00 +03:00
Mihaela Balutoiu
d340200333 Add more test cases for the /database/sql/users.go 
Add more coverage for the `users.go` using the SQL database interactions.

Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-14 19:24:42 +03:00
Gabriel
a4933b2836
Merge pull request #107 from mihaelabalutoiu/add-pools-unit-tests 
Add test cases for the `runner/pools.go`
 2023-06-14 15:45:42 +03:00
Mihaela Balutoiu
e1ad300f79 Add test cases for the runner/pools.go 
Signed-off-by: Mihaela Balutoiu <mbalutoiu@cloudbasesolutions.com>
 2023-06-14 14:17:47 +03:00