Commit graph

562 commits

Author SHA1 Message Date
Earl Warren
07c56a4032 chore(tests): add ./pkg/container tests
remove a failed test verifying exposed port which are not used in the
Forgejo runner anyway
2024-12-27 11:59:35 +01:00
Michael Kriese
d8376ed890 fix(cache-server): use consistent uint64 2024-11-22 01:01:12 +01:00
Earl Warren
ef243fbf2f fix: log skipped job and step result as info instead of debug
This allows the Forgejo runner to obtain the job result from the
logs even when it is not in debug mode.
2024-11-21 12:18:04 +00:00
Earl Warren
c42fa5f412 fix: [FORGEJO] do not share the act-toolcache volume
In the context of Forgejo the act-toolcache must not be mounted
otherwise independent workflows will race against each other when
writing to it. For instance if a setup-go action is run at the same
time as another, there is a probability that they both write the same
file at the same time, which could lead to a truncated or invalid
content.
2024-11-20 09:42:15 +00:00
ChristopherHX
017db5edae fix: cache adjust restore order of exact key matches (#2267)
* wip: adjust restore order

* fixup

* add tests

* cleanup

* fix typo

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
(cherry picked from commit 88ac88c5413a430b41d767a942c8e70e778e1d61)
2024-11-07 16:25:40 +01:00
Jason Song
b0c54edc78 Support overwriting caches (#2265)
* feat: support overwrite caches

* test: fix case

* test: fix get_with_multiple_keys

* chore: use atomic.Bool

* test: improve get_with_multiple_keys

* chore: use ping to improve path

* fix: wrong CompareAndSwap

* test: TestHandler_gcCache

* chore: lint code

* chore: lint code

(cherry picked from commit 087b28afc56351b93dd68d7e59a2c8740f6c0e44)
2024-11-07 16:25:26 +01:00
ChristopherHX
4b2554db86 fix: docker buildx cache restore not working (#2236)
* To take effect artifacts v4 pr is needed with adjusted claims

(cherry picked from commit c606759e8c0c2d5036c5bb15d7ec87beca1150cf)
2024-11-07 16:25:15 +01:00
Earl Warren
4b5c96dc22 fix: return an error when the argument count is wrong
Closes forgejo/runner#307
2024-10-31 16:02:17 +01:00
Earl Warren
d544eb76a8 fix: debug is leaking host container and network names
Closes forgejo/runner#295
2024-10-19 08:48:50 +02:00
Earl Warren
61b7859f12 [FORGEJO] when a workflow decode error happen, log and do not crash 2024-09-15 15:46:01 +02:00
Michael Kriese
5304d71a96 fix(jobparser): support workflow_dispatch.inputs 2024-08-22 15:02:05 +02:00
TheFox0x7
ae5411b096 fix(jobparser): template job name if it's defined 2024-07-01 20:23:17 +02:00
Tobias Bölz
6657ce2520 Fix NewDockerNetworkCreateExecutor in docker_stub 2024-04-03 13:55:42 +02:00
TheFox0x7
2ca53dd660 [FORGEJO] add forge alias for github (#37)
I left gitea for backwards compatibility

Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/37
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>
Co-committed-by: TheFox0x7 <thefox0x7@gmail.com>
2024-04-02 18:23:26 +00:00
Earl Warren
eb76c80193 [FORGEJO] a network of "" is not the same as "host"
The comment that introduced this change suggests it was motivated by a
border case by which the image would be empty. It is however unclear
why it should have any impact on how the network name is determined.

The hunk is reverted.

https://github.com/nektos/act/pull/1949/files#r1315163582
2024-03-24 12:09:40 +01:00
Earl Warren
18b78d6299 [FORGEJO] Revert "Don't set GITHUB_TOKEN (#2089)"
It is a breaking change.

This reverts commit d2b2ed2d66.
2024-03-11 17:46:38 +07:00
s3lph
0d8856b0ce [FORGEJO] feat(docker): Add flag to enable IPv6 in auto-created networks (#24)
Implements one part of forgejo/runner#119. The other part is a corresponding PR in forgejo/runner: forgejo/runner#120.

Reviewed-on: https://code.forgejo.org/forgejo/act/pulls/24
Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org>
Co-authored-by: s3lph <codeberg@s3lph.me>
Co-committed-by: s3lph <codeberg@s3lph.me>
2024-03-11 17:10:33 +07:00
Earl Warren
e9bb483fb8 [LXC] global lock on start
Since the start script may create LXC templates that are shared, they
may race against each other when running for the first time. A lock
global to the host needs to be used to guarantee that does not happen.
2024-03-11 15:30:55 +07:00
Earl Warren
44d6d5784c [LXC] split platform into template, release and config 2024-03-11 15:30:55 +07:00
Earl Warren
4553d1750c [FORGEJO] implement lxc separately from self-hosted 2024-03-11 15:30:55 +07:00
Earl Warren
22f27b8bd8 [FORGEJO] wrap self-hosted platform steps in an LXC container
act PR https://github.com/nektos/act/pull/1682

* shell script to start the LXC container
* create and destroy a LXC container
* run commands with lxc-attach
* expose additional devices for docker & libvirt to work
* install node 16 & git for checkout to work

[FORGEJO] start/stop lxc working directory is /tmp

[FORGEJO] use lxc-helpers to create/destroy containers

[FORGEJO] do not setup LXC

(cherry picked from commit 5b94ff3226848791b93e72d2e0f0ee4bba29a989)

Conflicts:
	pkg/container/host_environment.go

Conflicts:
	pkg/container/host_environment.go

[FORGJEO] upgrade to node20
2024-03-11 15:23:41 +07:00
Earl Warren
593a495381 [FORGEJO] sync lxc-helpers 231215c11d38df793521766dcce0e80e824614ca 2024-03-11 15:23:38 +07:00
Zettat123
90ce417f01 Support cloning remote actions from insecure Gitea instances (#92)
Related to https://github.com/go-gitea/gitea/issues/28693

Reviewed-on: https://gitea.com/gitea/act/pulls/92
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2024-03-05 08:07:29 +00:00
Claudio Nicora
2f624404fd Patched options() to let container options propagate to job containers (#80)
This PR let "general" container config to be propagated to each job container.

See:
- https://gitea.com/gitea/act_runner/issues/265#issuecomment-744382
- https://gitea.com/gitea/act_runner/issues/79
- https://gitea.com/gitea/act_runner/issues/378

Reviewed-on: https://gitea.com/gitea/act/pulls/80
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Claudio Nicora <claudio.nicora@gmail.com>
Co-committed-by: Claudio Nicora <claudio.nicora@gmail.com>
2024-03-04 02:56:52 +00:00
sillyguodong
291a501295 Make runs-on support variable expression (#91)
Partial implementation of https://gitea.com/gitea/act_runner/issues/445, the Gitea side also needs a PR for the entire functionality.
Gitea side: https://github.com/go-gitea/gitea/pull/29468

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/91
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-committed-by: sillyguodong <gedong_1994@163.com>
2024-03-01 04:29:58 +00:00
Chongyi Zheng
a7c4e92822 Merge branch 'nektos/master' into bump-nektos 2024-02-17 13:19:51 -05:00
Markus Wolf
f99698eb76 fix: use correct path to toolcache (#1494)
The toolcache on GitHub Actions need to be in
/opt/hostedtoolcache. This is the case for all
environment variables set by act, but it's not the
case for the volume mounted into the container.

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-30 22:43:52 +00:00
ChristopherHX
5e0a9ffa53 refactor: filecollector into new package (#2174)
* refactor: filecollector into new package

* Add test for symlinks

* add test fix bug of GetContainerArchive

* add test data
2024-01-30 00:46:45 +00:00
Josh Soref
79e54641f4 fix: improve warning about remote not found (#2169)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 19:21:21 +00:00
ChristopherHX
ce4598b0af fix: improve new-action-cache fetch failure error (#2172)
- include repoURL and repoRef in error
- map NoErrAlreadyUptodate to `couldn't find remote ref` for branchOrtag
  fetch request

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 17:02:15 +00:00
ChristopherHX
c5cae8a568 fix: subpath actions via new artifact cache (#2170)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 16:49:47 +00:00
ChristopherHX
5f0f24d596 fix: improve action not found error (#2171) 2024-01-28 16:37:19 +00:00
Matthew
c358c37324 Add containerd's normalized architectures to archMapper (#2168) 2024-01-24 02:44:48 +00:00
胖梁
853c5a22a4 WorkflowDispatchConfig supports multiple yaml node kinds (#2123)
* WorkflowDispatchConfig supports ScalarNode and SequenceNode yaml node kinds

* Avoid using log.Fatal

* package slices is not in golang 1.20

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 14:07:36 +00:00
Kristoffer
a25c37e83c fix: match cache restore-keys in creation reverse order (#2153)
* Match cache restore-keys in creation reverse order

* Match full prefix when selecting cache

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 12:11:50 +00:00
TKaxv_7S
141b614c36 feat: support offline mode (#2128)
* Add: Actions Offline Mode

* Add: Actions Offline Mode

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 00:20:15 +00:00
ChristopherHX
57a16ab0f7 feat: cli option to enable the new action cache (#1954)
* Enable the new action cache

* fix

* fix: CopyTarStream (Docker)

* suppress panic in test

* add a cli option for opt in

* fixups

* add package

* fix

* rc.Config nil in test???

* add feature flag

* patch

* Fix respect --action-cache-path

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>

* add remote reusable workflow to ActionCache

* fixup

---------

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-19 23:49:35 +00:00
Leonardo Taccari
4485482c1f Add support for NetBSD (#2023)
NetBSD can run Docker CLI and then use Docker on some remote machine
via DOCKER_HOST.

(This can be probably extended to all other Unix-es capable of running
just Docker CLI.)

Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-01-08 19:26:03 +00:00
Jon Jensen
182e1be469 Fix noisy runs-on error logging (#2102)
Move the logging back up a level to fix a minor logging issue introduced in #2088

`RunContext`s for composite actions have dummy/blank `Job`s with no `runs-on`,
meaning their calls to `withGithubEnv` would result in an inaccurate log message
complaining that `'runs-on' key not defined in ...`

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-12-16 23:04:54 +00:00
ChristopherHX
98afefd092 fix: IsHost is defined as false on windows (#2093)
* fix: IsHost is defined as false on windows

* Update docker_run.go

* Update docker_run.go
2023-12-16 14:46:17 -08:00
Jon Jensen
21fcd3b3dd Evaluate if condition when calling a reusable workflow (#2087)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 20:01:32 +00:00
raffis
0956a75aca feat: support runs-on labels and group (#2062)
Signed-off-by: Raffael Sahli <raffael.sahli@doodle.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 19:46:38 +00:00
Björn Brauer
bfde430c64 Evaluate all service values (#2054)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 18:30:21 +00:00
ChristopherHX
78a021bc14 fix: (#2075)
network-scoped alias is supported only for containers in user defined networks

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 18:09:25 +00:00
Jon Jensen
d2b2ed2d66 Don't set GITHUB_TOKEN (#2089)
This needs to be explicitly in the `env` to be consistent with GitHub

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 17:52:08 +00:00
Jon Jensen
16d6000e9a Support array expressions in runs-on (#2088)
* Support array expressions in runs-on

* Simplify appproach to use EvaluateYamlNode, fix case-sensitivity bug

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 17:40:06 +00:00
Andreas Taylor
078c67a6ee Use unique name for reusable workflow (#2015)
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 17:21:41 +00:00
Jason Song
0fe2aa1cd9 fix: panic (#2071)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-04 14:10:53 +00:00
Jason Song
3322bc0cda fix: panic 2023-10-31 22:32:21 +08:00
Sam Foo
2de6a8e3aa Add support for service containers (#1949)
* Support services (#42)

Removed createSimpleContainerName and AutoRemove flag

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/42
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support services options (#45)

Reviewed-on: https://gitea.com/gitea/act/pulls/45
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support intepolation for `env` of `services` (#47)

Reviewed-on: https://gitea.com/gitea/act/pulls/47
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support services `credentials` (#51)

If a service's image is from a container registry requires authentication, `act_runner` will need `credentials` to pull the image, see [documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idcredentials).
Currently, `act_runner` incorrectly uses the `credentials` of `containers` to pull services' images and the `credentials` of services won't be used, see the related code: ba7ef95f06/pkg/runner/run_context.go (L228-L269)

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/51
Reviewed-by: Jason Song <i@wolfogre.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Add ContainerMaxLifetime and ContainerNetworkMode options

from: 1d92791718

* Fix container network issue (#56)

Follow: https://gitea.com/gitea/act_runner/pulls/184
Close https://gitea.com/gitea/act_runner/issues/177

- `act` create new networks only if the value of `NeedCreateNetwork` is true, and remove these networks at last. `NeedCreateNetwork` is passed by `act_runner`. 'NeedCreateNetwork' is true only if  `container.network` in the configuration file of the `act_runner` is empty.
- In the `docker create` phase, specify the network to which containers will connect. Because, if not specify , container will connect to `bridge` network which is created automatically by Docker.
  - If the network is user defined network ( the value of `container.network` is empty or `<custom-network>`.  Because, the network created by `act` is also user defined network.), will also specify alias by `--network-alias`. The alias of service is `<service-id>`. So we can be access service container by `<service-id>:<port>` in the steps of job.
- Won't try to `docker network connect ` network after `docker start` any more.
  - Because on the one hand,  `docker network connect` applies only to user defined networks, if try to `docker network connect host <container-name>` will return error.
  - On the other hand, we just specify network in the stage of `docker create`, the same effect can be achieved.
- Won't try to remove containers and networks berfore  the stage of `docker start`, because the name of these containers and netwoks won't be repeat.

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/56
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-committed-by: sillyguodong <gedong_1994@163.com>

* Check volumes (#60)

This PR adds a `ValidVolumes` config. Users can specify the volumes (including bind mounts) that can be mounted to containers by this config.

Options related to volumes:
- [jobs.<job_id>.container.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes)
- [jobs.<job_id>.services.<service_id>.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idvolumes)

In addition, volumes specified by `options` will also be checked.

Currently, the following default volumes (see f78a6206d3/pkg/runner/run_context.go (L116-L166)) will be added to `ValidVolumes`:
- `act-toolcache`
- `<container-name>` and `<container-name>-env`
- `/var/run/docker.sock` (We need to add a new configuration to control whether the docker daemon can be mounted)

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/60
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Remove ContainerMaxLifetime; fix lint

* Remove unused ValidVolumes

* Remove ConnectToNetwork

* Add docker stubs

* Close docker clients to prevent file descriptor leaks

* Fix the error when removing network in self-hosted mode (#69)

Fixes https://gitea.com/gitea/act_runner/issues/255

Reviewed-on: https://gitea.com/gitea/act/pulls/69
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Move service container and network cleanup to rc.cleanUpJobContainer

* Add --network flag; default to host if not using service containers or set explicitly

* Correctly close executor to prevent fd leak

* Revert to tail instead of full path

* fix network duplication

* backport networkingConfig for aliaes

* don't hardcode netMode host

* Convert services test to table driven tests

* Add failing tests for services

* Expose service container ports onto the host

* Set container network mode in artifacts server test to host mode

* Log container network mode when creating/starting a container

* fix: Correctly handle ContainerNetworkMode

* fix: missing service container network

* Always remove service containers

Although we usually keep containers running if the workflow errored
(unless `--rm` is given) in order to facilitate debugging and we have
a flag (`--reuse`) to always keep containers running in order to speed
up repeated `act` invocations, I believe that these should only apply
to job containers and not service containers, because changing the
network settings on a service container requires re-creating it anyway.

* Remove networks only if no active endpoints exist

* Ensure job containers are stopped before starting a new job

* fix: go build -tags WITHOUT_DOCKER

---------

Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: ZauberNerd <zaubernerd@zaubernerd.de>
2023-10-19 09:24:52 +00:00