This reverts commit 996ac343ee.
The lock is still needed to guard against the following scenario.
d92a892ece/act/runner/run_context.go (L225-L236)
- two or more jobs start
- one of them creates the act template (`lxc_build_template $(lxc_template_release) $name`) - lxc-helpers now has transactions and they won't race against each other
- once it is built all jobs will then try to install node in the container that was just built and race against each other
with a global lock only the first one will build and populate the act template. The other will then do nothing because it already exists.
The bug can trivially be reproduced with:
```yaml
on:
pull_request:
jobs:
test1:
runs-on: lxc
steps:
- run: echo OK1
test2:
runs-on: lxc
steps:
- run: echo OK2
test3:
runs-on: lxc
steps:
- run: echo OK3
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1103): <!--number 1103 --><!--line 0 --><!--description UmV2ZXJ0ICJjaG9yZShjbGVhbnVwKTogbHhjLWhlbHBlcnMgZG9lcyBub3QgbmVlZCBhIGdsb2JhbCBsb2NrICgjMTA0Nyki-->Revert "chore(cleanup): lxc-helpers does not need a global lock (#1047)"<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1103
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: limiting-factor <limiting-factor@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Following on to #1065, the reason that the reverse proxy cannot be started (such as an invalid cache server URL) wasn't being logged.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1089): <!--number 1089 --><!--line 0 --><!--description Y2hvcmU6IGxvZyByZWFzb24gcmV2ZXJzZSBwcm94eSBjYW4ndCBiZSBzdGFydGVk-->chore: log reason reverse proxy can't be started<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1089
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Andrew Cassidy <drewcassidy@me.com>
Co-committed-by: Andrew Cassidy <drewcassidy@me.com>
## Problem
When using reusable workflows, the Forgejo runner prematurely revokes `GITHUB_TOKEN` after the first step completes, causing subsequent steps to fail with authentication errors.
### Reproduction
When the reusable workflow contains multiple steps that require authentication:
1. First step (e.g., checkout) completes successfully
2. Reporter receives completion banner from child workflow
3. Token is revoked prematurely
4. Second step fails with authentication error
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1081): <!--number 1081 --><!--line 0 --><!--description Zml4OiBwcmV2ZW50IHByZW1hdHVyZSB0b2tlbiByZXZvY2F0aW9uIGluIHJldXNhYmxlIHdvcmtmbG93cw==-->fix: prevent premature token revocation in reusable workflows<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1081
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Roman K. <devops@syncstack.net>
Co-committed-by: Roman K. <devops@syncstack.net>
An attempt to address #1074 in a way that makes everyone happy
Right now, any server URL that doesnt start with `http://` or `https://` automatically gets `https://` prepended to it. When no instance is set while running `forgejo-runner exec` this results in the `{FORGEJO,GITHUB}_SERVER_URL` environment variables being set to simply "https://". Any action using the GitHub actions toolkit chokes on this as an invalid URL and fails the run with a nondescript error. This PR instead leaves empty strings alone, which the actions toolkit seems much happier with
Alternate fixes:
* default the `--forgejo-instance` flag to a valid but unresolved url like 'http://example.com' so the actions don't crash
* don't set the environment variables at all when the server URL is an empty string. This causes the actions toolkit to assume GitHub.com as a fallback
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1083): <!--number 1083 --><!--line 0 --><!--description Zml4OiBkb24ndCBwcmVwZW5kIHNlcnZlciBVUkwgd2l0aCBodHRwczovLyBpZiBpdCdzIGFuIGVtcHR5IHN0cmluZw==-->fix: don't prepend server URL with https:// if it's an empty string<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1083
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Andrew Cassidy <drewcassidy@me.com>
Co-committed-by: Andrew Cassidy <drewcassidy@me.com>
Many emoji technically have two forms, the colorful form and the monochrome text form. For some reason, on macOS, ☁️ defaults to the latter, which is also really tiny for some reason?. Adding the invisible emoji variant selector (U+FE0F) causes it to always render as the colorful full-width graphic
Before:
After:
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1076): <!--number 1076 --><!--line 0 --><!--description ZGlzYW1iaWd1YXRlIGNsb3VkIGVtb2pp-->disambiguate cloud emoji<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1076
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Andrew Cassidy <drewcassidy@me.com>
Co-committed-by: Andrew Cassidy <drewcassidy@me.com>
When using a reusable workflow such as:
```yaml
name: "use-inputs-impl"
on:
workflow_call:
inputs:
image:
description: "Image to be used in job"
type: string
required: false
default: "docker.io/alpine:3.22"
condition:
description: "Only run a job if this is true"
type: string
required: false
default: false
jobs:
works:
runs-on: docker
container:
image: alpine:3.22
env:
INPUT_IMAGE: ${{ inputs.image }}
INPUT_CONDITION: ${{ inputs.condition }}
steps:
- run: echo "Hello world from Alpine"
- run: cat /etc/alpine-release
- run: echo "INPUT_IMAGE=$INPUT_IMAGE"
- run: echo "INPUT_CONDITION=$INPUT_CONDITION"
```
The runner will currently output un-evaluated expressions:
```
INPUT_IMAGE=${{ inputs.image }}
INPUT_CONDITION=${{ inputs.condition }}
```
Internally, the workflow is using `INPUT_...` environment variables to transfer data into local reusable workflows, and the order of evaluation in `getEvaluatorInputs` was preferring to use the un-evaluated value, over the evaluated value.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1080): <!--number 1080 --><!--line 0 --><!--description Zml4OiBhbGxvdyBtYXBwaW5nIGludG8gZW52IHZhcmlhYmxlIElOUFVUX04gZnJvbSAke3sgaW5wdXRzLm4gfX0=-->fix: allow mapping into env variable INPUT_N from ${{ inputs.n }}<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1080
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
First hand experience to implement this feature can be found at https://codeberg.org/forgejo/forgejo/issues/9406. In a nutshell it is a service container in the CI that randomly uses massive amounts of memory (>50GB RSS) but normally less than 100MB.
---
See also the [matching documentation pull request](https://codeberg.org/forgejo/docs/pulls/1539).
---
It is already possible to limit the memory used by all containers in the config file:
```yaml
container:
options: --memory 200M
```
This limit can be further reduced (but not increased) by the same option in a job:
```yaml
jobs:
job:
runs-on: docker
container:
image: code.forgejo.org/oci/node:20-bookworm
options: --memory 200M
steps:
- run: echo OK
```
or a service container:
```yaml
job:
my-job:
runs-on: docker
services:
pgsql:
image: postgres:15
options: --memory 1G
```
Refs https://docs.docker.com/engine/containers/resource_constraints/#limit-a-containers-access-to-memory
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- features
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1079): <!--number 1079 --><!--line 0 --><!--description ZmVhdDogYWxsb3cgLS1tZW1vcnkgaW4gd29ya2Zsb3cgY29udGFpbmVyIG9wdGlvbnM=-->feat: allow --memory in workflow container options<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1079
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
With https://codeberg.org/forgejo/forgejo/pulls/9530, the error messages from `ParseRawOn` are user-facing and need a pass to improve their meaning.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- features
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1063): <!--number 1063 --><!--line 0 --><!--description ZmVhdDogaW1wcm92ZSByZWFkYWJpbGl0eSBvZiBlcnJvciBtZXNzYWdlcyBmcm9tIFBhcnNlUmF3T24=-->feat: improve readability of error messages from ParseRawOn<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1063
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- https://code.forgejo.org/forgejo/lxc-helpers/src/tag/v1.1.1 has locking and atomicity
- this lock was only good for when two release templates are populated simultaneously and not when it was interrupted
Refs forgejo/lxc-helpers#8
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1047): <!--number 1047 --><!--line 0 --><!--description V0lQOiBjaG9yZShjbGVhbnVwKTogbHhjLWhlbHBlcnMgZG9lcyBub3QgbmVlZCBhIGdsb2JhbCBsb2Nr-->WIP: chore(cleanup): lxc-helpers does not need a global lock<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1047
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: limiting-factor <limiting-factor@posteo.com>
Co-committed-by: limiting-factor <limiting-factor@posteo.com>
- remove the root of the LXC container after destroying it, with sudo as it may contain files owned by root while the runner id is not root
- os.RemoveAll only for native host runs, it is no longer needed for the LXC backend
- remove the CleanUp function that is an indirection with no use
Resolvesforgejo/runner#442
When running the test from a non-root user and without this fix, it fails as follow:
```
go test -v -count=1 -run='TestRunnerLXC' ./internal/app/run
=== RUN TestRunnerLXC
...
time="2025-10-03T15:05:12+02:00" level=debug msg=stopHostEnvironment
time="2025-10-03T15:05:13+02:00" level=debug msg="HostEnvironment.Remove /tmp/TestRunnerLXC1841090130/001/d29c1256e2912892/hostexecutor"
time="2025-10-03T15:05:13+02:00" level=error msg="Error while stop job container FORGEJO-ACTIONS-TASK-0_WORKFLOW-3ede81fbc69d42e6db70bef5820490fc3e7dc4d9dcbfb64981f2d00f08a30d6e_JOB-job: unlinkat /tmp/TestRunnerLXC1841090130/001/d29c1256e2912892/hostexecutor/some/directory/owned/by/root: permission denied"
=== NAME TestRunnerLXC
runner_test.go:469:
Error Trace: /home/earl-warren/software/runner/internal/app/run/runner_test.go:469
/home/earl-warren/software/runner/internal/app/run/runner_test.go:496
Error: Received unexpected error:
Error occurred running finally: unlinkat /tmp/TestRunnerLXC1841090130/001/d29c1256e2912892/hostexecutor/some/directory/owned/by/root: permission denied (original error: <nil>)
Test: TestRunnerLXC
Messages: OK
=== NAME TestRunnerLXC/OK
testing.go:1679: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
=== NAME TestRunnerLXC
testing.go:1267: TempDir RemoveAll cleanup: unlinkat /tmp/TestRunnerLXC1841090130/001/d29c1256e2912892/hostexecutor/some/directory/owned/by/root: permission denied
--- FAIL: TestRunnerLXC (6.84s)
--- FAIL: TestRunnerLXC/OK (6.84s)
FAIL
FAIL code.forgejo.org/forgejo/runner/v11/internal/app/run 6.847s
FAIL
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1054): <!--number 1054 --><!--line 0 --><!--description Zml4OiByZW1vdmUgTFhDIGJhY2tlbmQgbGVmdG92ZXJzIHdoZW4gdGhlIGpvYiBjb21wbGV0ZXM=-->fix: remove LXC backend leftovers when the job completes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1054
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Additional logging to support #1044.
Manual testing only. Cases tested:
Cancel a job from Forgejo UI; this seems like the most likely missing piece in #1044 as two jobs were simultaneously marked as "Failed". There are codepaths in Forgejo that can set this state to both cancelled and failed, but the runner didn't provide log output indicating that's why a job was stopping:
```
time="2025-10-02T13:22:53-06:00" level=info msg="UpdateTask returned task result RESULT_CANCELLED for a task that was in local state RESULT_UNSPECIFIED - beginning local task termination" func="[ReportState]" file="[reporter.go:410]"
```
Host-based executor hits step timeout in exec, or, is cancelled. This occurred but only logged the `err` from `exec`, not the context error indicating whether it was a timeout or a cancellation:
```
[Test Action/job1] this step has been cancelled: ctx: context deadline exceeded, exec: RUN signal: killed
[Test Action/job1] this step has been cancelled: ctx: context canceled, exec: RUN signal: killed
```
Unable to `ReportState` due to Forgejo inaccessible. If the runner isn't able to update state to Forgejo a job could be considered a zombie; this would trigger one of the codepaths where the job would be marked as failed. If connectivity was later restored, then the runner could identify it was marked as failed and cancel the job context. (This combination doesn't seem likely, but, I think it's reasonable to consider these failures as warnings because there may be unexpected errors here that we're not aware of).
```
time="2025-10-02T13:27:19-06:00" level=warning msg="ReportState error: unavailable: 502 Bad Gateway" func="[RunDaemon]" file="[reporter.go:207]"
```
Runner shutdown logging; just changed up to `Info` level:
```
time="2025-10-02T13:31:36-06:00" level=info msg="forcing the jobs to shutdown" func="[Shutdown]" file="[poller.go:93]"
[Test Action/job1] ❌ Failure - Main sleep 120
[Test Action/job1] this step has been cancelled: ctx: context canceled, exec: RUN signal: killed
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1048): <!--number 1048 --><!--line 0 --><!--description Zml4OiBpbXByb3ZlIGxvZ2dpbmcgdG8gZGlhZ25vc2UgbXlzdGVyeSBqb2IgdGVybWluYXRpb25z-->fix: improve logging to diagnose mystery job terminations<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1048
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Fixes#1039.
Rather than opening and closing the Bolt DB instance constantly, the cache now maintains one open `*bolthold.Store` for its lifetime, allowing GC, cache read, and cache write operations to occur concurrently.
The major risk is this change is, "is it safe to use one Bolt instance across goroutines concurrently?" [Bolt does document its concurrency requirements](https://github.com/boltdb/bolt?tab=readme-ov-file#transactions), and an analysis of our DB interactions looks to me like it introduces very little risk.
Most of the cache operations perform multiple touches to the database; for example `useCache` performs a read to fetch a cache object, and then an update to set its `UsedAt` timestamp. If we wanted to ensure consistency in these operations, they should use a Bolt ReadWrite transaction -- but concurrent access would just be setting the field to the same value anyway.
The `gcCache` is the complex operation where a transaction might be warranted -- but doing so would also cause the same bug that #1039 indicates. I believe it is safe to run without a transaction because it is protected by an application-level mutex (to prevent multiple concurrent GCs), it is the only code that performs deletes from the database -- these should guarantee that all its delete attempts are successful. In the event of unexpected failure to do the DB write, `gcCache` deletes from the storage before deleting from the DB, so it should just attempt to cleanup again next run.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1040): <!--number 1040 --><!--line 0 --><!--description Zml4OiBhbGxvdyBHQyAmIGNhY2hlIG9wZXJhdGlvbnMgdG8gb3BlcmF0ZSBjb25jdXJyZW50bHk=-->fix: allow GC & cache operations to operate concurrently<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1040
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
The test is covered by TestRunnerLXC which verifies the script actually runs on LXC.
Resolvesforgejo/runner#1035
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1037): <!--number 1037 --><!--line 0 --><!--description Zml4OiBkbyBub3QgYXR0ZW1wdCB0byBydW4gdGhlIExYQyBzdG9wIHNjcmlwdCB3aXRoIHNlbGYtaG9zdGVk-->fix: do not attempt to run the LXC stop script with self-hosted<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1037
Reviewed-by: fnetX <fnetx@noreply.code.forgejo.org>
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
- Changes `EvaluateConcurrency` to `EvaluateWorkflowConcurrency`, which has no job-related arguments
- Changes gitContext to be sent as an object rather than a map
- Allows `nil` to be returned for `cancelInProgress`, which indicates that the value wasn't specified in the input yaml -- required for distinguishing the `cancel-in-progress: false` case from not being specified at all.
ReadWorkflowRawConcurrency & EvaluateWorkflowConcurrency were never used in forgejo yet, so this shouldn't break the forgejo build.
Prerequisite for https://codeberg.org/forgejo/forgejo/pulls/9434.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- features
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1026): <!--number 1026 --><!--line 0 --><!--description ZmVhdDogc3VwcG9ydCBldmFsdWF0aW5nIHdvcmtmbG93LWxldmVsIGNvbmN1cnJlbmN5IGJsb2NrcyBpbiBqb2JwYXJzZXI=-->feat: support evaluating workflow-level concurrency blocks in jobparser<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1026
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This fixes a bug currently preventing use of an external cache server in the Codeberg runner setup.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1027): <!--number 1027 --><!--line 0 --><!--description Zml4OiBDb3JyZWN0bHkgb3ZlcnJpZGUgdGhlIHZhbHVlIG9mIGBGb3JnZWpvLUNhY2hlLUhvc3RgIHdoZW4gQUNUSU9OU19DQUNIRV9VUkwgaXMgb3ZlcnJpZGRlbi4=-->fix: Correctly override the value of `Forgejo-Cache-Host` when ACTIONS_CACHE_URL is overridden.<!--description-->
<!--end release-notes-assistant-->
Co-authored-by: Vivian <me@vivstuff.gay>
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1027
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Kwonunn <kwonunn@noreply.code.forgejo.org>
Co-committed-by: Kwonunn <kwonunn@noreply.code.forgejo.org>
Resolvesforgejo/runner#1014
---
Manual testing can also be done using the [reproducer from the issue](https://code.forgejo.org/forgejo/runner/issues/1014#issuecomment-60694).
## Before
The first step of the local composite action sets the step result of the job to success which confuses Forgejo display.
## After
Forgejo displays the progress of the composite action in the step calling it.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1019): <!--number 1019 --><!--line 0 --><!--description Zml4OiBhIGNvbXBvc2l0ZSBhY3Rpb24gbXVzdCBub3QgY2hhbmdlIHRoZSByZXN1bHQgb2YgdGhlIGNhbGxpbmcgc3RlcCBiZWZvcmUgaXQgY29tcGxldGVz-->fix: a composite action must not change the result of the calling step before it completes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1019
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
The working directory was not cleaned up upon completion of a LXC job because rc.stopJobContainer() -> rc.cleanUpJobContainer() -> rc.JobContainer.Remove() was never called for LXC containers.
- stopContainer() and closeContainer() must not call
rc.stopHostEnvironment(ctx) for LXC containers because
- it will needlessly be called twice
- it intercepts the call to
- rc.stopJobContainer()
- rc.JobContainer.Close()
- rc.stopHostEnvironment(ctx) must be called in rc.cleanUpJobContainer which is indirectly called by rc.stopJobContainer()
- since rc.JobContainer.Close() is a noop, not calling it for LXC containers had no consequence
Resolvesforgejo/runner#442
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/1003): <!--number 1003 --><!--line 0 --><!--description Zml4OiByZW1vdmUgTFhDIHdvcmtpbmcgZGlyZWN0b3J5IHdoZW4gaXQgY29tcGxldGVz-->fix: remove LXC working directory when it completes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/1003
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Fixes#994.
First commit ensures that the interpolateResults method is invoked before data is sent to the reporter. Second commit changes how data is sent to the reporter to include both the result and the job outputs.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/995): <!--number 995 --><!--line 0 --><!--description Zml4OiBzZW5kIGpvYiBvdXRwdXRzICYgam9iIHJlc3VsdCB0byBGb3JnZWpvIGluIHN5bmMgd2l0aCBlYWNoIG90aGVy-->fix: send job outputs & job result to Forgejo in sync with each other<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/995
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- enforce timeout-minutes timeout for jobs in a way similar to how it is done for steps
- minimal refactor of evaluateStepTimeout evaluateTimeout so it can be used by jobs as well, with additional debug information and error logging if parsing fails
- add integration tests for both step and job timeout-minutes, verifying expressions are allowed and evaluated
Resolvesforgejo/runner#979
---
Manually verified to work as expected https://v13.next.forgejo.org/earl-warren/testtimeout-minutes/actions/runs/3/jobs/0/attempt/1
```yaml
on: [push]
jobs:
test:
runs-on: docker
timeout-minutes: 1
steps:
- run: |
set -x
while : ; do
sleep 30
done
```
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/982): <!--number 982 --><!--line 0 --><!--description Zml4OiBlbmZvcmNlIGpvYi48am9iLWlkPi50aW1lb3V0LW1pbnV0ZXM=-->fix: enforce job.<job-id>.timeout-minutes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/982
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
- create the caches interface and matching cachesImpl
- move the cache logic out of handler
- openDB
- readCache
- useCache
- gcCache
- access to the storage struct
- serve
- commit
- exist
- write
- add getCaches / setCaches to the handler interface so it can be
used by tests. The caches test should be implemented independently
in the future but this is a different kind of cleanup.
- no functional change, minimal refactor
- responseFatalJSON(w, r, err) replaces responseJSON(w, r, 500, err)
and calls fatal() when the following fail because they are
not recoverable. There may be other non-recoverable errors but
it is difficult to be 100% sure they cannot be engineered by the
caller of the API for DoS purposes.
- openDB
- findCache
- cache.Repo != repo
- wrap errors in
- openDB() - it was missing
- readCache() - it was missing
- useCache() - it was missing
- findCache() - some had identical messages
- in gc
- replace logger.Warnf with h.fatal
- differentiate errors that have identical messages
- call fatal if openDB fails instead of returning
in case of an error that is not recoverable (e.g. failing to open the
bolthold database), the cache can call fatal() to log the error and
send a TERM signal that will gracefully shutdown the daemon.
the license change from MIT to GPLv3+ is a breaking change
Refs forgejo/runner#773
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/940): <!--number 940 --><!--line 0 --><!--description Y2hvcmU6IGJ1bXAgdmVyc2lvbiB0byB2MTE=-->chore: bump version to v11<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/940
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
It was raised during embargo review of #925 that there are two implementations of `computeMac`; this PR fixes that.
As all the tests for `computeMac` were in the `artifactcache` package, it made more sense to keep the method there. That required reversing the dependency `artifactcache->cacheproxy` package dependency -- it makes more sense to me for the proxy to depend on the cache, rather than vice-versa.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/936): <!--number 936 --><!--line 0 --><!--description cmVmYWN0b3I6IHJlbW92ZSBkdXBsaWNhdGUgY29tcHV0ZU1hYyBmdW5jdGlvbg==-->refactor: remove duplicate computeMac function<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/936
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- the Handler struct becomes handler (lowercase)
- the Handler interface is defined to be the existing methods
- isClosed() is added and used only in tests
- setgcAt() is added and used only in tests
---
This is to allow mocking the Handler interface for testing.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- other
- [PR](https://code.forgejo.org/forgejo/runner/pulls/934): <!--number 934 --><!--line 0 --><!--description Y2hvcmU6IHJlZmFjdG9yIGFjdC9hcnRpZmFjdGNhY2hlIEhhbmRsZXIgdG8gYW4gaW50ZXJmYWNl-->chore: refactor act/artifactcache Handler to an interface<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/934
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.code.forgejo.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
Container images built by the runner are tagged with a unique name:
- based on the specified `uses` URL for remote actions.
- random for local actions.
In the case of local actions, this will create new tags for each run
but the images (and their layers) will be shared and not be
duplicated. The least recently used tags can be garbage collected by
tools such as https://github.com/stepchowfun/docuum.
Using a different method for creating the tag name for the remote
actions is to help with maintenance by establishing a direct relation
with the `uses` field. It was instead relying on a name transformed
multiple times which makes it more difficult to verify name collision
are not accidentally made possible by one of those transformations.
Without this fix, when a workflow ran a local [docker action](https://forgejo.org/docs/next/user/actions/actions/#docker-actions)
(e.g. the [example in the end-to-end
tests](8f920b4b7a/actions/example-force-rebuild/.forgejo/workflows/test.yml)),
it used an image tag that could collide with other workflows that
happen to use the same name.
The workaround for older runner versions is to set
[`[container].force_rebuild: true`](https://forgejo.org/docs/next/admin/actions/runner-installation/#configuration)
in the runner configuration file.
