DevFW-CICD/garm
15
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Update docs

Browse source 
Reword some phrases, update links and steps.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This commit is contained in:
Gabriel Adrian Samfira 2024-08-03 16:03:50 +00:00
parent 56b0e6065a
commit 35f4bea269
15 changed files with 57 additions and 1119 deletions
Download patch file Download diff file Expand all files Collapse all files

12
README.md
View file

@ -6,13 +6,13 @@ Welcome to GARM!
GARM enables you to create and automatically maintain pools of [self-hosted GitHub runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners), with auto-scaling that can be used inside your github workflow runs.
The goal of ```GARM``` is to be simple to set up, simple to configure and simple to use. It is a single binary that can run on any GNU/Linux machine without any other requirements other than the providers it creates the runners in. It is intended to be easy to deploy in any environment and can create runners in any system you can write a provider for. There is no complicated setup process and no extremely complex concepts to understand. Once set up, it's meant to stay out of your way.
The goal of ```GARM``` is to be simple to set up, simple to configure and simple to use. The server itself is a single binary that can run on any GNU/Linux machine without any other requirements other than the providers you want to enable in your setup. It is intended to be easy to deploy in any environment and can create runners in virtually any system you can write a provider for. There is no complicated setup process and no extremely complex concepts to understand. Once set up, it's meant to stay out of your way.
GARM supports creating pools on either GitHub itself or on your own deployment of [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@3.5/admin/overview/about-github-enterprise-server). For instructions on how to use ```GARM``` with GHE, see the [credentials](/doc/github_credentials.md) section of the documentation.
GARM supports creating pools in either GitHub itself or in your own deployment of [GitHub Enterprise Server](https://docs.github.com/en/enterprise-server@3.5/admin/overview/about-github-enterprise-server). For instructions on how to use ```GARM``` with GHE, see the [credentials](/doc/github_credentials.md) section of the documentation.
Through the use of providers, `GARM` can create runners in a variety of environments using the same `GARM` instance. Whether you want to create pools of runners in your OpenStack cloud, your Azure cloud and your Kubernetes cluster, that is easily achieved by just installing the appropriate providers, configuring them in `GARM` and creating pools that use them. You can create zero-runner pools for instances with high costs (large VMs, GPU enabled instances, etc) and have them spin up on demand, or you can create large pools of k8s backed runners that can be used for your CI/CD pipelines at a moment's notice. You can mix them up and create pools in any combination of providers or resource allocations you want.
Through the use of providers, `GARM` can create runners in a variety of environments using the same `GARM` instance. Whether you want to create pools of runners in your OpenStack cloud, your Azure cloud or your Kubernetes cluster, that is easily achieved by just installing the appropriate providers, configuring them in `GARM` and creating pools that use them. You can create zero-runner pools for instances with high costs (large VMs, GPU enabled instances, etc) and have them spin up on demand, or you can create large pools of eagerly creaated k8s backed runners that can be used for your CI/CD pipelines at a moment's notice. You can mix them up and create pools in any combination of providers or resource allocations you want.
:warning: **Important note**: The README and documentation in the `main` branch are relevant to the not yet released code that is present in `main`. Following the documentation from the `main` branch for a stable release of GARM, may lead to errors. To view the documentation for the latest stable release, please switch to the appropriate tag. For information about setting up `v0.1.4`, please refer to the [v0.1.4 tag](https://github.com/cloudbase/garm/tree/v0.1.4).
:warning: **Important note**: The README and documentation in the `main` branch are relevant to the not yet released code that is present in `main`. Following the documentation from the `main` branch for a stable release of GARM, may lead to errors. To view the documentation for the latest stable release, please switch to the appropriate tag. For information about setting up `v0.1.5`, please refer to the [v0.1.5 tag](https://github.com/cloudbase/garm/tree/v0.1.5).
## Join us on slack
@ -28,7 +28,7 @@ Check out the [quickstart](/doc/quickstart.md) document for instructions on how
### On Kubernetes
Thanks to the efforts of the amazing folks at @mercedes-benz, GARM can now be integrated into k8s via their operator. Check out the [GARM operator](https://github.com/mercedes-benz/garm-operator/) for more details.
Thanks to the efforts of the amazing folks at [@mercedes-benz](https://github.com/mercedes-benz/), GARM can now be integrated into k8s via their operator. Check out the [GARM operator](https://github.com/mercedes-benz/garm-operator/) for more details.
## Configuration
@ -44,7 +44,7 @@ The ```GARM``` configuration is a simple ```toml```. The sample config file in [
## Using GARM
GARM is designed with simplicity in mind. At least we try to keep it as simple as possible. We're aware that adding a new tool in your workflow can be painful, especially when you already have to deal with so many. The cognitive load for OPS has reached a level where it feels overwhelming at times to even wrap your head around a new tool. As such, we believe that tools should be simple, should take no more than a few hours to understand and set up and if you absolutely need to interact with the tool, it should be as intuitive as possible.
GARM is designed with simplicity in mind. At least we try to keep it as simple as possible. We're aware that adding a new tool in your workflow can be painful, especially when you already have to deal with so many. The cognitive load for OPS has reached a level where it feels overwhelming at times to even wrap your head around a new tool. As such, we believe that tools should be simple, should take no more than a few hours to understand and set up and if you absolutely need to interact with the tool, it should be as intuitive as possible. Although we try our best to make this happen, we're aware that GARM has some rough edges, especially for new users. If you encounter issues or feel like the setup process was too complicated, please let us know. We're always looking to improve the user experience.
We've written a short introduction into some of the commands that GARM has and some of the concepts involved in setting up GARM, managing runners and how GitHub does some of the things it does.

81
contrib/providers.d/azure/cloudconfig/install_runner.tpl
View file

@ -1,81 +0,0 @@
#!/bin/bash
set -e
set -o pipefail
METADATA_URL="GARM_METADATA_URL"
CALLBACK_URL="GARM_CALLBACK_URL"
BEARER_TOKEN="GARM_CALLBACK_TOKEN"
DOWNLOAD_URL="GH_DOWNLOAD_URL"
DOWNLOAD_TOKEN="GH_TEMP_DOWNLOAD_TOKEN"
FILENAME="GH_FILENAME"
TARGET_URL="GH_TARGET_URL"
RUNNER_NAME="GH_RUNNER_NAME"
RUNNER_LABELS="GH_RUNNER_LABELS"
TEMP_TOKEN=""
if [ -z "$METADATA_URL" ];then
echo "no token is available and METADATA_URL is not set"
exit 1
fi
function call() {
PAYLOAD="$1"
curl --fail -s -X POST -d "${PAYLOAD}" -H 'Accept: application/json' -H "Authorization: Bearer ${BEARER_TOKEN}" "${CALLBACK_URL}" || echo "failed to call home: exit code ($?)"
}
function sendStatus() {
MSG="$1"
call "{\"status\": \"installing\", \"message\": \"$MSG\"}"
}
function success() {
MSG="$1"
ID=$2
call "{\"status\": \"idle\", \"message\": \"$MSG\", \"agent_id\": $ID}"
}
function fail() {
MSG="$1"
call "{\"status\": \"failed\", \"message\": \"$MSG\"}"
exit 1
}
if [ ! -z "$DOWNLOAD_TOKEN" ]; then
TEMP_TOKEN="Authorization: Bearer $DOWNLOAD_TOKEN"
fi
sendStatus "downloading tools from ${DOWNLOAD_URL}"
curl --fail -L -H "${TEMP_TOKEN}" -o "/home/runner/${FILENAME}" "${DOWNLOAD_URL}" || fail "failed to download tools"
mkdir -p /home/runner/actions-runner || fail "failed to create actions-runner folder"
sendStatus "extracting runner"
tar xf "/home/runner/${FILENAME}" -C /home/runner/actions-runner/ || fail "failed to extract runner"
chown runner:runner -R /home/runner/actions-runner/ || fail "failed to change owner"
sendStatus "installing dependencies"
cd /home/runner/actions-runner
sudo ./bin/installdependencies.sh || fail "failed to install dependencies"
sendStatus "fetching runner registration token"
GITHUB_TOKEN=$(curl --fail -s -X GET -H 'Accept: application/json' -H "Authorization: Bearer ${BEARER_TOKEN}" "${METADATA_URL}" || fail "failed to get runner registration token")
sendStatus "configuring runner"
sudo -u runner -- ./config.sh --unattended --url "${TARGET_URL}" --token "${GITHUB_TOKEN}" --name "${RUNNER_NAME}" --labels "${RUNNER_LABELS}" --ephemeral || fail "failed to configure runner"
sendStatus "installing runner service"
./svc.sh install runner || fail "failed to install service"
sendStatus "starting service"
./svc.sh start || fail "failed to start service"
set +e
AGENT_ID=$(grep "agentId" /home/runner/actions-runner/.runner | tr -d -c 0-9)
if [ $? -ne 0 ];then
fail "failed to get agent ID"
fi
set -e
success "runner successfully installed" $AGENT_ID

31
contrib/providers.d/azure/cloudconfig/userdata.tpl
View file

@ -1,31 +0,0 @@
#cloud-config
package_upgrade: true
packages:
- curl
- tar
system_info:
default_user:
name: runner
home: /home/runner
shell: /bin/bash
groups:
- sudo
- adm
- cdrom
- dialout
- dip
- video
- plugdev
- netdev
- docker
- lxd
sudo: ALL=(ALL) NOPASSWD:ALL
runcmd:
- /install_runner.sh
- rm -f /install_runner.sh
write_files:
- encoding: b64
content: RUNNER_INSTALL_B64
owner: root:root
path: /install_runner.sh
permissions: "755"

8
contrib/providers.d/azure/config.sh
View file

@ -1,8 +0,0 @@
# Azure service principal credentials
export AZURE_SUBSCRIPTION_ID="<SUBSCRIPTION_ID>"
export AZURE_TENANT_ID="<TENANT_ID>"
export AZURE_CLIENT_ID="<CLIENT_ID>"
export AZURE_CLIENT_SECRET="<CLIENT_SECRET>"
# GARM config
export LOCATION="westeurope"

370
contrib/providers.d/azure/garm-external-provider
View file

@ -1,370 +0,0 @@
#!/bin/bash
set -e
set -o pipefail
if [ ! -t 0 ]
then
INPUT=$(cat -)
fi
MYPATH=$(realpath ${BASH_SOURCE[0]})
MYDIR=$(dirname "${MYPATH}")
TEMPLATES="$MYDIR/cloudconfig"
# Defaults
LOCATION=${LOCATION:"westeurope"}
# END Defaults
if [ -z "$GARM_PROVIDER_CONFIG_FILE" ]
then
echo "no config file specified in env"
exit 1
fi
source "$GARM_PROVIDER_CONFIG_FILE"
declare -A GARM_TO_GH_ARCH_MAP
GARM_TO_GH_ARCH_MAP["amd64"]="x64"
GARM_TO_GH_ARCH_MAP["arm"]="arm"
GARM_TO_GH_ARCH_MAP["arm64"]="arm64"
declare -A AZURE_OS_TO_GH_OS_MAP
AZURE_OS_TO_GH_OS_MAP["Linux"]="linux"
AZURE_OS_TO_GH_OS_MAP["Windows"]="win"
# https://docs.microsoft.com/en-us/azure/virtual-machines/states-billing#power-states-and-billing
declare -A AZURE_POWER_STATE_MAP
AZURE_POWER_STATE_MAP["VM starting"]="pending_create"
AZURE_POWER_STATE_MAP["VM running"]="running"
AZURE_POWER_STATE_MAP["VM stopping"]="stopped"
AZURE_POWER_STATE_MAP["VM stopped"]="stopped"
AZURE_POWER_STATE_MAP["VM deallocating"]="stopped"
AZURE_POWER_STATE_MAP["VM deallocated"]="stopped"
# https://docs.microsoft.com/en-us/azure/virtual-machines/states-billing#provisioning-states
declare -A AZURE_PROVISION_STATE_MAP
AZURE_PROVISION_STATE_MAP["Creating"]="pending_create"
AZURE_PROVISION_STATE_MAP["Updating"]="pending_create"
AZURE_PROVISION_STATE_MAP["Migrating"]="pending_create"
AZURE_PROVISION_STATE_MAP["Failed"]="error"
AZURE_PROVISION_STATE_MAP["Succeeded"]="running"
AZURE_PROVISION_STATE_MAP["Deleting"]="pending_delete"
function checkValNotNull() {
if [ -z "$1" -o "$1" == "null" ]; then
echo "failed to fetch value $2"
return 1
fi
return 0
}
function requestedArch() {
ARCH=$(echo "$INPUT" | jq -c -r '.arch')
checkValNotNull "${ARCH}" "arch" || return $?
echo "${ARCH}"
}
function downloadURL() {
[ -z "$1" -o -z "$2" ] && return 1
GH_OS="${AZURE_OS_TO_GH_OS_MAP[$1]}"
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
URL=$(echo "$INPUT" | jq -c -r --arg OS "$GH_OS" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).download_url')
checkValNotNull "${URL}" "download URL" || return $?
echo "${URL}"
}
function tempDownloadToken() {
[ -z "$1" -o -z "$2" ] && return 1
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
TOKEN=$(echo "$INPUT" | jq -c -r --arg OS "$1" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).temp_download_token')
echo "${TOKEN}"
}
function runnerTokenURL() {
METADATA_URL=$(echo "$INPUT" | jq -c -r '."metadata-url"')
checkValNotNull "${METADATA_URL}" "metadata-url" || return $?
echo "${METADATA_URL}/runner-registration-token/"
}
function downloadFilename() {
[ -z "$1" -o -z "$2" ] && return 1
GH_OS="${AZURE_OS_TO_GH_OS_MAP[$1]}"
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
FN=$(echo "$INPUT" | jq -c -r --arg OS "$GH_OS" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).filename')
checkValNotNull "${FN}" "download filename" || return $?
echo "${FN}"
}
function poolID() {
POOL_ID=$(echo "$INPUT" | jq -c -r '.pool_id')
checkValNotNull "${POOL_ID}" "pool_id" || return $?
echo "${POOL_ID}"
}
function vmSize() {
VM_SIZE=$(echo "$INPUT" | jq -c -r '.flavor')
checkValNotNull "${VM_SIZE}" "flavor" || return $?
echo "${VM_SIZE}"
}
function imageUrn() {
IMG=$(echo "$INPUT" | jq -c -r '.image')
checkValNotNull "${IMG}" "image" || return $?
echo "${IMG}"
}
function getOSImageDetails() {
IMAGE=$(echo "$INPUT" | jq -r -c '.image')
IMAGE_DETAILS=$(az vm image show --urn "$IMAGE" -o json --only-show-errors)
echo "$IMAGE_DETAILS"
}
function repoURL() {
REPO=$(echo "$INPUT" | jq -c -r '.repo_url')
checkValNotNull "${REPO}" "repo_url" || return $?
echo "${REPO}"
}
function callbackURL() {
CB_URL=$(echo "$INPUT" | jq -c -r '."callback-url"')
checkValNotNull "${CB_URL}" "callback-url" || return $?
echo "${CB_URL}"
}
function callbackToken() {
CB_TK=$(echo "$INPUT" | jq -c -r '."instance-token"')
checkValNotNull "${CB_TK}" "instance-token" || return $?
echo "${CB_TK}"
}
function instanceName() {
NAME=$(echo "$INPUT" | jq -c -r '.name')
checkValNotNull "${NAME}" "name" || return $?
echo "${NAME}"
}
function labels() {
LBL=$(echo "$INPUT" | jq -c -r '.labels | join(",")')
checkValNotNull "${LBL}" "labels" || return $?
echo "${LBL}"
}
function vmStatus() {
[ -z "$1" -o -z "$2" ] && return 1
RG_DETAILS=$(az group show -n "$1" -o json --only-show-errors)
RG_STATE=$(echo "$RG_DETAILS" | jq -r '.properties.provisioningState')
STATUS="${AZURE_PROVISION_STATE_MAP[$RG_STATE]}"
if [[ "$STATUS" != "running" ]]; then
echo "$STATUS"
return 0
fi
VM_DETAILS=$(az vm show -g "$1" -n "$2" --show-details -o json --only-show-errors)
VM_STATE=$(echo "$VM_DETAILS" | jq -r '.provisioningState')
STATUS="${AZURE_PROVISION_STATE_MAP[$VM_STATE]}"
if [[ "$STATUS" != "running" ]]; then
echo "$STATUS"
return 0
fi
VM_POWER_STATE=$(echo "$VM_DETAILS" | jq -r '.powerState')
VM_STATUS="${AZURE_POWER_STATE_MAP[$VM_POWER_STATE]}"
if [[ -z "${VM_STATUS}" ]]; then
echo "unknown"
return 0
fi
echo "${VM_STATUS}"
}
function getCloudConfig() {
IMAGE_DETAILS=$(getOSImageDetails)
OS_TYPE=$(echo "${IMAGE_DETAILS}" | jq -c -r '.osDiskImage.operatingSystem')
checkValNotNull "${OS_TYPE}" "operatingSystem" || return $?
ARCH=$(requestedArch)
DW_URL=$(downloadURL "${OS_TYPE}" "${ARCH}")
DW_TOKEN=$(tempDownloadToken "${OS_TYPE}" "${ARCH}")
DW_FILENAME=$(downloadFilename "${OS_TYPE}" "${ARCH}")
LABELS=$(labels)
TMP_SCRIPT=$(mktemp)
TMP_CC=$(mktemp)
INSTALL_TPL=$(cat "${TEMPLATES}/install_runner.tpl")
CC_TPL=$(cat "${TEMPLATES}/userdata.tpl")
echo "$INSTALL_TPL" | sed -e "s|GARM_CALLBACK_URL|$(callbackURL)|g" \
-e "s|GARM_CALLBACK_TOKEN|$(callbackToken)|g" \
-e "s|GH_DOWNLOAD_URL|${DW_URL}|g" \
-e "s|GH_FILENAME|${DW_FILENAME}|g" \
-e "s|GH_TARGET_URL|$(repoURL)|g" \
-e "s|GARM_METADATA_URL|$(runnerTokenURL)|g" \
-e "s|GH_RUNNER_NAME|$(instanceName)|g" \
-e "s|GH_TEMP_DOWNLOAD_TOKEN|${DW_TOKEN}|g" \
-e "s|GH_RUNNER_LABELS|${LABELS}|g" > ${TMP_SCRIPT}
AS_B64=$(base64 -w0 ${TMP_SCRIPT})
echo "${CC_TPL}" | sed "s|RUNNER_INSTALL_B64|${AS_B64}|g" > ${TMP_CC}
echo "${TMP_CC}"
}
function CreateInstance() {
if [ -z "$INPUT" ]; then
echo "expected build params in stdin"
exit 1
fi
CC_FILE=$(getCloudConfig)
VM_SIZE=$(vmSize)
INSTANCE_NAME=$(instanceName)
IMAGE_URN=$(imageUrn)
IMAGE_DETAILS=$(getOSImageDetails)
OS_TYPE=$(echo "${IMAGE_DETAILS}" | jq -c -r '.osDiskImage.operatingSystem' | tr '[:upper:]' '[:lower:]')
checkValNotNull "${OS_TYPE}" "os_type" || return $?
OS_NAME=$(echo "${IMAGE_URN}" | cut -d ':' -f2)
OS_VERSION=$(echo "${IMAGE_URN}" | cut -d ':' -f3)
ARCH="amd64"
TAGS="garm_controller_id=${GARM_CONTROLLER_ID} garm_pool_id=${GARM_POOL_ID} os_type=${OS_TYPE} os_name=${OS_NAME} os_version=${OS_VERSION} os_arch=${ARCH}"
set +e
az group create -n $INSTANCE_NAME -l $LOCATION --tags $TAGS --only-show-errors -o none
az vm create -g $INSTANCE_NAME -n $INSTANCE_NAME -l $LOCATION --size $VM_SIZE --image $IMAGE_URN --tags $TAGS --nsg-rule none --public-ip-address "" --user-data "${CC_FILE}" -o none --only-show-errors
if [[ $? -ne 0 ]]; then
az group delete -n $INSTANCE_NAME --no-wait --y -o none --only-show-errors
echo "Failed to create Azure VM"
exit 1
fi
rm -f "${CC_FILE}"
set -e
STATUS=$(vmStatus $INSTANCE_NAME $INSTANCE_NAME)
FAULT_VAL=""
jq -rnc \
--arg PROVIDER_ID "${INSTANCE_NAME}" \
--arg NAME "${INSTANCE_NAME}" \
--arg OS_TYPE "${OS_TYPE}" \
--arg OS_NAME "${OS_NAME}" \
--arg OS_VERSION "${OS_VERSION}" \
--arg ARCH "${ARCH}" \
--arg STATUS "${STATUS}" \
--arg POOL_ID "${GARM_POOL_ID}" \
--arg FAULT "${FAULT_VAL}" \
'{"provider_id": $PROVIDER_ID, "name": $NAME, "os_type": $OS_TYPE, "os_name": $OS_NAME, "os_version": $OS_VERSION, "os_arch": $ARCH, "status": $STATUS, "pool_id": $POOL_ID, "provider_fault": $FAULT}'
}
function DeleteInstance() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ]; then
echo "missing instance ID in env"
return 1
fi
set +e
rg_info=$(az group show -n "${instance_id}" -o json --only-show-errors 2>&1)
if [ $? -ne 0 ]; then
CODE=$?
set -e
if echo "${rg_info}" | grep -q "ResourceGroupNotFound"; then
return 0
fi
return $CODE
fi
set -e
az group delete -n "${instance_id}" --no-wait --y --only-show-errors
}
function StartInstance() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ]; then
echo "missing instance ID in env"
return 1
fi
az vm start -g "${instance_id}" -n "${instance_id}" -o none --only-show-errors
}
function StopServer() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ]; then
echo "missing instance ID in env"
return 1
fi
az vm deallocate -g "${instance_id}" -n "${instance_id}" -o none --only-show-errors
}
function GetInstance() {
local instance_id="${GARM_INSTANCE_ID}"
info=$(az vm show -d -n $instance_id -g $instance_id -o json --only-show-errors 2>&1)
echo $info | jq -r '
{
provider_id: .name,
name: .name,
os_type: .tags.os_type,
os_name: .tags.os_name,
os_version: .tags.os_version,
os_arch: .tags.os_arch,
pool_id: .tags.garm_pool_id,
status: {"VM starting": "pending_create", "VM running": "running", "VM stopping": "stopped", "VM stopped": "stopped", "VM deallocating": "stopped", "VM deallocated": "stopped"}[.powerState]
}'
}
function ListInstances() {
INSTANCES=$(az vm list --query "[?tags.garm_pool_id == '${GARM_POOL_ID}']" -o json --only-show-errors 2>&1)
echo $info | jq -r '[
.[] | {
provider_id: .name,
name: .name,
os_type: .tags.os_type,
os_name: .tags.os_name,
os_version: .tags.os_version,
os_arch: .tags.os_arch,
pool_id: .tags.garm_pool_id,
status: {"Creating": "pending_create", "Migrating": "pending_create", "Failed": "error", "Succeeded": "running", "Deleting": "pending_delete"}[.provisioningState]
}]'
}
# Login to Azure
checkValNotNull "${AZURE_SUBSCRIPTION_ID}" "AZURE_SUBSCRIPTION_ID"
checkValNotNull "${AZURE_TENANT_ID}" "AZURE_TENANT_ID"
checkValNotNull "${AZURE_CLIENT_ID}" "AZURE_CLIENT_ID"
checkValNotNull "${AZURE_CLIENT_SECRET}" "AZURE_CLIENT_SECRET"
export AZURE_CONFIG_DIR="${MYDIR}/.azure"
az login --service-principal -u $AZURE_CLIENT_ID -p $AZURE_CLIENT_SECRET --tenant $AZURE_TENANT_ID -o none --only-show-errors
az account set -s $AZURE_SUBSCRIPTION_ID -o none --only-show-errors
case "$GARM_COMMAND" in
"CreateInstance")
CreateInstance
;;
"DeleteInstance")
DeleteInstance
;;
"GetInstance")
GetInstance
;;
"ListInstances")
ListInstances
;;
"StartInstance")
StartInstance
;;
"StopInstance")
StopServer
;;
"RemoveAllInstances")
echo "RemoveAllInstances not implemented"
exit 1
;;
*)
echo "Invalid GARM provider command: \"$GARM_COMMAND\""
exit 1
;;
esac

12
contrib/providers.d/openstack/README.md
View file

@ -1,12 +0,0 @@
# OpenStack external provider for garm
This is an example external provider, written for OpenStack. It is a simple bash script that implements the external provider interface, in order to supply ```garm``` with compute instances. This is just an example, complete with a sample config file.
Not all functions are implemented, just the bare minimum to get it to work with the current feature set of ```garm```. It is not meant for production, as it needs a lot more error checking, retries, and potentially more flexibility to be of any use in a real environment.
Images that are used with garm require the following properties set on the image:
* os_type (one of: windows, linux)
* os_distro
* os_version
* architecture (one of: x86_64, armv7l, mips64, mips64el, mips, mipsel)

81
contrib/providers.d/openstack/cloudconfig/install_runner.tpl
View file

@ -1,81 +0,0 @@
#!/bin/bash
set -e
set -o pipefail
METADATA_URL="GARM_METADATA_URL"
CALLBACK_URL="GARM_CALLBACK_URL"
BEARER_TOKEN="GARM_CALLBACK_TOKEN"
DOWNLOAD_URL="GH_DOWNLOAD_URL"
DOWNLOAD_TOKEN="GH_TEMP_DOWNLOAD_TOKEN"
FILENAME="GH_FILENAME"
TARGET_URL="GH_TARGET_URL"
RUNNER_NAME="GH_RUNNER_NAME"
RUNNER_LABELS="GH_RUNNER_LABELS"
TEMP_TOKEN=""
if [ -z "$METADATA_URL" ];then
echo "no token is available and METADATA_URL is not set"
exit 1
fi
function call() {
PAYLOAD="$1"
curl --fail -s -X POST -d "${PAYLOAD}" -H 'Accept: application/json' -H "Authorization: Bearer ${BEARER_TOKEN}" "${CALLBACK_URL}" || echo "failed to call home: exit code ($?)"
}
function sendStatus() {
MSG="$1"
call "{\"status\": \"installing\", \"message\": \"$MSG\"}"
}
function success() {
MSG="$1"
ID=$2
call "{\"status\": \"idle\", \"message\": \"$MSG\", \"agent_id\": $ID}"
}
function fail() {
MSG="$1"
call "{\"status\": \"failed\", \"message\": \"$MSG\"}"
exit 1
}
if [ ! -z "$DOWNLOAD_TOKEN" ]; then
TEMP_TOKEN="Authorization: Bearer $DOWNLOAD_TOKEN"
fi
sendStatus "downloading tools from ${DOWNLOAD_URL}"
curl --fail -L -H "${TEMP_TOKEN}" -o "/home/runner/${FILENAME}" "${DOWNLOAD_URL}" || fail "failed to download tools"
mkdir -p /home/runner/actions-runner || fail "failed to create actions-runner folder"
sendStatus "extracting runner"
tar xf "/home/runner/${FILENAME}" -C /home/runner/actions-runner/ || fail "failed to extract runner"
chown runner:runner -R /home/runner/actions-runner/ || fail "failed to change owner"
sendStatus "installing dependencies"
cd /home/runner/actions-runner
sudo ./bin/installdependencies.sh || fail "failed to install dependencies"
sendStatus "fetching runner registration token"
GITHUB_TOKEN=$(curl --fail -s -X GET -H 'Accept: application/json' -H "Authorization: Bearer ${BEARER_TOKEN}" "${METADATA_URL}" || fail "failed to get runner registration token")
sendStatus "configuring runner"
sudo -u runner -- ./config.sh --unattended --url "${TARGET_URL}" --token "${GITHUB_TOKEN}" --name "${RUNNER_NAME}" --labels "${RUNNER_LABELS}" --ephemeral || fail "failed to configure runner"
sendStatus "installing runner service"
./svc.sh install runner || fail "failed to install service"
sendStatus "starting service"
./svc.sh start || fail "failed to start service"
set +e
AGENT_ID=$(grep "agentId" /home/runner/actions-runner/.runner | tr -d -c 0-9)
if [ $? -ne 0 ];then
fail "failed to get agent ID"
fi
set -e
success "runner successfully installed" $AGENT_ID

31
contrib/providers.d/openstack/cloudconfig/userdata.tpl
View file

@ -1,31 +0,0 @@
#cloud-config
package_upgrade: true
packages:
- curl
- tar
system_info:
default_user:
name: runner
home: /home/runner
shell: /bin/bash
groups:
- sudo
- adm
- cdrom
- dialout
- dip
- video
- plugdev
- netdev
- docker
- lxd
sudo: ALL=(ALL) NOPASSWD:ALL
runcmd:
- /install_runner.sh
- rm -f /install_runner.sh
write_files:
- encoding: b64
content: RUNNER_INSTALL_B64
owner: root:root
path: /install_runner.sh
permissions: "755"

445
contrib/providers.d/openstack/garm-external-provider
View file

@ -1,445 +0,0 @@
#!/bin/bash
set -e
set -o pipefail
if [ ! -t 0 ]
then
INPUT=$(cat -)
fi
MYPATH=$(realpath ${BASH_SOURCE[0]})
MYDIR=$(dirname "${MYPATH}")
TEMPLATES="$MYDIR/cloudconfig"
# Defaults
# set this variable to 0 in the provider config to disable.
BOOT_FROM_VOLUME=${BOOT_FROM_VOLUME:-1}
# END Defaults
if [ -z "$GARM_PROVIDER_CONFIG_FILE" ]
then
echo "no config file specified in env"
exit 1
fi
source "$GARM_PROVIDER_CONFIG_FILE"
declare -A OS_TO_GH_ARCH_MAP
OS_TO_GH_ARCH_MAP["x86_64"]="x64"
OS_TO_GH_ARCH_MAP["armv7l"]="arm64"
OS_TO_GH_ARCH_MAP["mips64"]="arm64"
OS_TO_GH_ARCH_MAP["mips64el"]="arm64"
OS_TO_GH_ARCH_MAP["mips"]="arm"
OS_TO_GH_ARCH_MAP["mipsel"]="arm"
declare -A OS_TO_GARM_ARCH_MAP
OS_TO_GARM_ARCH_MAP["x86_64"]="amd64"
OS_TO_GARM_ARCH_MAP["armv7l"]="arm64"
OS_TO_GARM_ARCH_MAP["mips64"]="arm64"
OS_TO_GARM_ARCH_MAP["mips64el"]="arm64"
OS_TO_GARM_ARCH_MAP["mips"]="arm"
OS_TO_GARM_ARCH_MAP["mipsel"]="arm"
declare -A GARM_TO_GH_ARCH_MAP
GARM_TO_GH_ARCH_MAP["amd64"]="x64"
GARM_TO_GH_ARCH_MAP["arm"]="arm"
GARM_TO_GH_ARCH_MAP["arm64"]="arm64"
declare -A STATUS_MAP
STATUS_MAP["ACTIVE"]="running"
STATUS_MAP["SHUTOFF"]="stopped"
STATUS_MAP["BUILD"]="pending_create"
STATUS_MAP["ERROR"]="error"
STATUS_MAP["DELETING"]="pending_delete"
function checkValNotNull() {
if [ -z "$1" -o "$1" == "null" ];then
echo "failed to fetch value $2"
return 1
fi
return 0
}
function getOSImageDetails() {
IMAGE_ID=$(echo "$INPUT" | jq -r -c '.image')
OS_IMAGE=$(openstack image show "$IMAGE_ID" -f json)
echo "$OS_IMAGE"
}
function getOpenStackNetworkID() {
if [ -z "$OPENSTACK_PRIVATE_NETWORK" ]
then
echo "no network specified in config"
return 1
fi
NET_ID=$(openstack network show ${OPENSTACK_PRIVATE_NETWORK} -f value -c id)
if [ -z "$NET_ID" ];then
echo "failed to find network $OPENSTACK_PRIVATE_NETWORK"
fi
echo ${NET_ID}
}
function getVolumeSizeFromFlavor() {
local flavor="$1"
FLAVOR_DETAILS=$(openstack flavor show "${flavor}" -f json)
DISK_SIZE=$(echo "$FLAVOR_DETAILS" | jq -c -r '.disk')
if [ -z "$DISK_SIZE" ];then
echo "failed to get disk size from flavor"
return 1
fi
echo ${DISK_SIZE}
}
function waitForVolume() {
local volumeName=$1
set +e
status=$(openstack volume show "${volumeName}" -f json | jq -r -c '.status')
if [ $? -ne 0 ];then
CODE=$?
set -e
return $CODE
fi
set -e
while [ "${status}" != "available" -a "${status}" != "error" ];do
status=$(openstack volume show "${volumeName}" -f json | jq -r -c '.status')
done
}
function createVolumeFromImage() {
local image="$1"
local disk_size="$2"
local instance_name="$3"
if [ -z ${image} -o -z ${disk_size} -o -z "${instance_name}" ];then
echo "missing image, disk size or instance name in function call"
return 1
fi
# Instance names contain a UUID. It should be safe to create a volume with the same name and
# expect it to be unique.
set +e
VOLUME_INFO=$(openstack volume create -f json --image "${image}" --size "${disk_size}" "${instance_name}")
if [ $? -ne 0 ]; then
CODE=$?
openstack volume delete "${instance_name}" || true
set -e
return $CODE
fi
waitForVolume "${instance_name}"
echo "${VOLUME_INFO}"
}
function requestedArch() {
ARCH=$(echo "$INPUT" | jq -c -r '.arch')
checkValNotNull "${ARCH}" "arch" || return $?
echo "${ARCH}"
}
function downloadURL() {
[ -z "$1" -o -z "$2" ] && return 1
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
URL=$(echo "$INPUT" | jq -c -r --arg OS "$1" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).download_url')
checkValNotNull "${URL}" "download URL" || return $?
echo "${URL}"
}
function tempDownloadToken() {
[ -z "$1" -o -z "$2" ] && return 1
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
TOKEN=$(echo "$INPUT" | jq -c -r --arg OS "$1" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).temp_download_token')
echo "${TOKEN}"
}
function runnerTokenURL() {
METADATA_URL=$(echo "$INPUT" | jq -c -r '."metadata-url"')
checkValNotNull "${METADATA_URL}" "metadata-url" || return $?
echo "${METADATA_URL}/runner-registration-token/"
}
function downloadFilename() {
[ -z "$1" -o -z "$2" ] && return 1
GH_ARCH="${GARM_TO_GH_ARCH_MAP[$2]}"
FN=$(echo "$INPUT" | jq -c -r --arg OS "$1" --arg ARCH "$GH_ARCH" '(.tools[] | select( .os == $OS and .architecture == $ARCH)).filename')
checkValNotNull "${FN}" "download filename" || return $?
echo "${FN}"
}
function poolID() {
POOL_ID=$(echo "$INPUT" | jq -c -r '.pool_id')
checkValNotNull "${POOL_ID}" "pool_id" || return $?
echo "${POOL_ID}"
}
function flavor() {
FLAVOR=$(echo "$INPUT" | jq -c -r '.flavor')
checkValNotNull "${FLAVOR}" "flavor" || return $?
echo "${FLAVOR}"
}
function image() {
IMG=$(echo "$INPUT" | jq -c -r '.image')
checkValNotNull "${IMG}" "image" || return $?
echo "${IMG}"
}
function repoURL() {
REPO=$(echo "$INPUT" | jq -c -r '.repo_url')
checkValNotNull "${REPO}" "repo_url" || return $?
echo "${REPO}"
}
function callbackURL() {
CB_URL=$(echo "$INPUT" | jq -c -r '."callback-url"')
checkValNotNull "${CB_URL}" "callback-url" || return $?
echo "${CB_URL}"
}
function callbackToken() {
CB_TK=$(echo "$INPUT" | jq -c -r '."instance-token"')
checkValNotNull "${CB_TK}" "instance-token" || return $?
echo "${CB_TK}"
}
function instanceName() {
NAME=$(echo "$INPUT" | jq -c -r '.name')
checkValNotNull "${NAME}" "name" || return $?
echo "${NAME}"
}
function labels() {
LBL=$(echo "$INPUT" | jq -c -r '.labels | join(",")')
checkValNotNull "${LBL}" "labels" || return $?
echo "${LBL}"
}
function getCloudConfig() {
IMAGE_DETAILS=$(getOSImageDetails)
OS_TYPE=$(echo "${IMAGE_DETAILS}" | jq -c -r '.properties.os_type')
checkValNotNull "${OS_TYPE}" "os_type" || return $?
ARCH=$(requestedArch)
DW_URL=$(downloadURL "${OS_TYPE}" "${ARCH}")
DW_TOKEN=$(tempDownloadToken "${OS_TYPE}" "${ARCH}")
DW_FILENAME=$(downloadFilename "${OS_TYPE}" "${ARCH}")
LABELS=$(labels)
TMP_SCRIPT=$(mktemp)
TMP_CC=$(mktemp)
INSTALL_TPL=$(cat "${TEMPLATES}/install_runner.tpl")
CC_TPL=$(cat "${TEMPLATES}/userdata.tpl")
echo "$INSTALL_TPL" | sed -e "s|GARM_CALLBACK_URL|$(callbackURL)|g" \
-e "s|GARM_CALLBACK_TOKEN|$(callbackToken)|g" \
-e "s|GH_DOWNLOAD_URL|${DW_URL}|g" \
-e "s|GH_FILENAME|${DW_FILENAME}|g" \
-e "s|GH_TARGET_URL|$(repoURL)|g" \
-e "s|GARM_METADATA_URL|$(runnerTokenURL)|g" \
-e "s|GH_RUNNER_NAME|$(instanceName)|g" \
-e "s|GH_TEMP_DOWNLOAD_TOKEN|${DW_TOKEN}|g" \
-e "s|GH_RUNNER_LABELS|${LABELS}|g" > ${TMP_SCRIPT}
AS_B64=$(base64 -w0 ${TMP_SCRIPT})
echo "${CC_TPL}" | sed "s|RUNNER_INSTALL_B64|${AS_B64}|g" > ${TMP_CC}
echo "${TMP_CC}"
}
function waitForServer() {
local srv_id="$1"
srv_info=$(openstack server show -f json "${srv_id}")
[ $? -ne 0 ] && return $?
status=$(echo "${srv_info}" | jq -r -c '.status')
while [ "${status}" != "ERROR" -a "${status}" != "ACTIVE" ];do
sleep 0.5
srv_info=$(openstack server show -f json "${srv_id}")
[ $? -ne 0 ] && return $?
status=$(echo "${srv_info}" | jq -r -c '.status')
done
echo "${srv_info}"
}
function CreateInstance() {
if [ -z "$INPUT" ];then
echo "expected build params in stdin"
exit 1
fi
CC_FILE=$(getCloudConfig)
FLAVOR=$(flavor)
IMAGE=$(image)
INSTANCE_NAME=$(instanceName)
NET=$(getOpenStackNetworkID)
IMAGE_DETAILS=$(getOSImageDetails)
OS_TYPE=$(echo "${IMAGE_DETAILS}" | jq -c -r '.properties.os_type')
checkValNotNull "${OS_TYPE}" "os_type" || return $?
DISTRO=$(echo "${IMAGE_DETAILS}" | jq -c -r '.properties.os_distro')
checkValNotNull "${DISTRO}" "os_distro" || return $?
VERSION=$(echo "${IMAGE_DETAILS}" | jq -c -r '.properties.os_version')
checkValNotNull "${VERSION}" "os_version" || return $?
ARCH=$(echo "${IMAGE_DETAILS}" | jq -c -r '.properties.architecture')
checkValNotNull "${ARCH}" "architecture" || return $?
GH_ARCH=${OS_TO_GH_ARCH_MAP[${ARCH}]}
if [ -z "${GH_ARCH}" ];then
GH_ARCH=${ARCH}
fi
SOURCE_ARGS=""
if [ "${BOOT_FROM_VOLUME}" -eq 1 ];then
VOL_SIZE=$(getVolumeSizeFromFlavor "${FLAVOR}")
VOL_INFO=$(createVolumeFromImage "${IMAGE}" "${VOL_SIZE}" "${INSTANCE_NAME}")
if [ $? -ne 0 ];then
openstack volume delete "${INSTANCE_NAME}" || true
fi
SOURCE_ARGS="--volume ${INSTANCE_NAME}"
else
SOURCE_ARGS="--image ${IMAGE}"
fi
set +e
TAGS="--tag garm-controller-id=${GARM_CONTROLLER_ID} --tag garm-pool-id=${GARM_POOL_ID}"
PROPERTIES="--property os_type=${OS_TYPE} --property os_name=${DISTRO} --property os_version=${VERSION} --property os_arch=${GH_ARCH} --property pool_id=${GARM_POOL_ID}"
SRV_DETAILS=$(openstack server create --os-compute-api-version 2.52 ${SOURCE_ARGS} ${TAGS} ${PROPERTIES} --flavor "${FLAVOR}" --user-data="${CC_FILE}" --network="${NET}" "${INSTANCE_NAME}")
if [ $? -ne 0 ];then
openstack volume delete "${INSTANCE_NAME}" || true
exit 1
fi
SRV_DETAILS=$(waitForServer "${INSTANCE_NAME}")
if [ $? -ne 0 ];then
CODE=$?
# cleanup
rm -f "${CC_FILE}" || true
openstack server delete "${INSTANCE_NAME}" || true
openstack volume delete "${INSTANCE_NAME}" || true
set -e
FAULT=$(echo "${SRV_DETAILS}"| jq -rc '.fault')
echo "Failed to create server: ${FAULT}"
exit $CODE
fi
set -e
rm -f "${CC_FILE}" || true
SRV_ID=$(echo "${SRV_DETAILS}" | jq -r -c '.id')
STATUS=$(echo "${SRV_DETAILS}" | jq -r -c '.status')
FAULT=$(echo "${SRV_DETAILS}" | jq -r -c '.fault')
FAULT_VAL=""
if [ ! -z "${FAULT}" -a "${FAULT}" != "null" ];then
FAULT_VAL=$(echo "${FAULT}" | base64 -w0)
fi
jq -rnc \
--arg PROVIDER_ID ${SRV_ID} \
--arg NAME "${INSTANCE_NAME}" \
--arg OS_TYPE "${OS_TYPE}" \
--arg OS_NAME "${DISTRO}" \
--arg OS_VERSION "${VERSION}" \
--arg ARCH "${GH_ARCH}" \
--arg STATUS "${STATUS_MAP[${STATUS}]}" \
--arg POOL_ID "${GARM_POOL_ID}" \
--arg FAULT "${FAULT_VAL}" \
'{"provider_id": $PROVIDER_ID, "name": $NAME, "os_type": $OS_TYPE, "os_name": $OS_NAME, "os_version": $OS_VERSION, "os_arch": $ARCH, "status": $STATUS, "pool_id": $POOL_ID, "provider_fault": $FAULT}'
}
function DeleteInstance() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ];then
echo "missing instance ID in env"
return 1
fi
set +e
instance_info=$(openstack server show "${instance_id}" -f json 2>&1)
if [ $? -ne 0 ];then
CODE=$?
set -e
if [ "${instance_info}" == "No server with a name or ID of*" ];then
return 0
fi
return $CODE
fi
set -e
VOLUMES=$(echo "${instance_info}" | jq -r -c '.volumes_attached[] | .id')
openstack server delete "${instance_id}"
for vol in "$VOLUMES";do
waitForVolume "${vol}"
openstack volume delete $vol || true
done
}
function StartInstance() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ];then
echo "missing instance ID in env"
return 1
fi
openstack server start "${instance_id}"
}
function StopServer() {
local instance_id="${GARM_INSTANCE_ID}"
if [ -z "${instance_id}" ];then
echo "missing instance ID in env"
return 1
fi
openstack server stop "${instance_id}"
}
function ListInstances() {
INSTANCES=$(openstack server list --os-compute-api-version 2.52 --tags garm-pool-id=${GARM_POOL_ID} --long -f json)
echo ${INSTANCES} | jq -r '[
.[] | .Properties * {
provider_id: .ID,
name: .Name,
status: {"ACTIVE": "running", "SHUTOFF": "stopped", "BUILD": "pending_create", "ERROR": "error", "DELETING": "pending_delete"}[.Status]
}]'
}
function GetInstance() {
INSTANCE=$(openstack server show --os-compute-api-version 2.52 ${GARM_INSTANCE_ID} -f json)
echo ${INSTANCES} | jq -r '.properties * {
provider_id: .id,
name: .name,
status: {"ACTIVE": "running", "SHUTOFF": "stopped", "BUILD": "pending_create", "ERROR": "error", "DELETING": "pending_delete"}[.status]
}'
}
case "$GARM_COMMAND" in
"CreateInstance")
CreateInstance
;;
"DeleteInstance")
DeleteInstance
;;
"GetInstance")
GetInstance
;;
"ListInstances")
ListInstances
;;
"StartInstance")
StartInstance
;;
"StopInstance")
StopServer
;;
"RemoveAllInstances")
echo "RemoveAllInstances not implemented"
exit 1
;;
*)
echo "Invalid GARM provider command: \"$GARM_COMMAND\""
exit 1
;;
esac

16
contrib/providers.d/openstack/keystonerc
View file

@ -1,16 +0,0 @@
# OpenStack client config
export OS_REGION_NAME=RegionOne
export OS_AUTH_VERSION=3
export OS_AUTH_URL=http://10.0.8.36:5000/v3
export OS_PROJECT_DOMAIN_NAME=admin_domain
export OS_USERNAME=admin
export OS_AUTH_TYPE=password
export OS_USER_DOMAIN_NAME=admin_domain
export OS_PROJECT_NAME=admin
export OS_PASSWORD=Iegeehahth4suSie
export OS_IDENTITY_API_VERSION=3
# GARM config
export OPENSTACK_PRIVATE_NETWORK="int_net"
export BOOT_FROM_VOLUME=1

12
doc/github_credentials.md
View file

@ -47,7 +47,7 @@ ubuntu@garm:~/garm$ garm-cli github endpoint list
GARM has the option to use both [Personal Access Tokens (PAT)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) or a [GitHub App](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app).
If you'll use a PAT, you'll have to grant access for the following scopes:
If you'll use a PAT (classic), you'll have to grant access for the following scopes:
* ```public_repo``` - for access to a repository
* ```repo``` - for access to a private repository
@ -56,6 +56,16 @@ If you'll use a PAT, you'll have to grant access for the following scopes:
* ```admin:repo_hook``` - if you want to allow GARM to install webhooks on repositories (optional)
* ```admin:org_hook``` - if you want to allow GARM to install webhooks on organizations (optional)
Fine grained PATs are also supported as long as you grant the required privileges:
* **Repository permissions**:
* `Administration: Read & write` - needed to generate JIT config/registration token, remove runners, etc.
* `Metadata: Read-only` - automatically enabled by above
* `Webhooks: Read & write` - needed to install webhooks on repositories
* **Organization permissions**:
* `Self-hosted runners: Read & write` - needed to manage runners in an organization
* `Webhooks: Read & write` - needed to install webhooks on organizations
If you plan to use github apps, you'll need to select the following permissions:
* **Repository permissions**:

2
doc/labels.md
View file

@ -12,4 +12,4 @@ Before version 2.305.0 of the runner and before JIT runners were introduced, the
This made scheduling and using runners a bit awkward in some situations. For example, in large organizations with many teams, often times workflows would simply target the `self-hosted` label. This would match all runners regardless of any other custom labels. This had the side effect that workflows would potentially use expensive runners for simple jobs or would select low resource runners for tasks that would require a lot of resources.
Version 2.305.0 of the runner introduced the `--no-default-labels` flag when registering the runner. When JIT is not available (GHES version < 3.10), GARM will now register the runner with the `--no-default-labels` flag. If you still need the default labels, you can add them to the pool specification.
Version 2.305.0 of the runner introduced the `--no-default-labels` flag when registering the runner. When JIT is not available (GHES version < 3.10), GARM will now register the runner with the `--no-default-labels` flag. If you still need the default labels, you can still add them when creating the pool as part of the `--tags` command line option.

22
doc/providers.md
View file

@ -1,15 +1,13 @@
# Provider configuration
GARM was designed to be extensible. Providers can be written as external executables. External providers are executables that implement the needed interface to create/delete/list compute systems that are used by ```GARM``` to create runners.
GARM was designed to be extensible. Providers can be written as external executables which implement the needed interface to create/delete/list compute systems that are used by ```GARM``` to create runners.
- [External provider](#external-provider)
- [Providers](#providers)
- [Available external providers](#available-external-providers)
## External provider
## Providers
The external provider is a special kind of provider. It delegates the functionality needed to create the runners to external executables. These executables can be either binaries or scripts. As long as they adhere to the needed interface, they can be used to create runners in any target IaaS. This is identical to what ```containerd``` does with ```CNIs```.
There are currently two sample external providers available in the [contrib folder of this repository](../contrib/providers.d/). The providers are written in ```bash``` and are meant as examples of how a provider could be written in ```bash```. Production ready providers would need more error checking and idempotency, but they serve as an example of what can be done. As it stands, they are functional.
GARM delegates the functionality needed to create the runners to external executables. These executables can be either binaries or scripts. As long as they adhere to the needed interface, they can be used to create runners in any target IaaS. You might find this behavior familiar if you've ever had to deal with installing `CNIs` in `containerd`. The principle is the same.
The configuration for an external provider is quite simple:
@ -28,17 +26,23 @@ provider_type = "external"
# anything (bash, a binary, python, etc). See documentation in this repo on how to write an
# external provider.
provider_executable = "/etc/garm/providers.d/openstack/garm-external-provider"
# This option will pass all environment variables that start with AWS_ to the provider.
# To pass in individual variables, you can add the entire name to the list.
environment_variables = ["AWS_"]
```
The external provider has two options:
The external provider has three options:
* ```provider_executable```
* ```config_file```
* `provider_executable`
* `config_file`
* `environment_variables`
The ```provider_executable``` option is the absolute path to an executable that implements the provider logic. GARM will delegate all provider operations to this executable. This executable can be anything (bash, python, perl, go, etc). See [Writing an external provider](./external_provider.md) for more details.
The ```config_file``` option is a path on disk to an arbitrary file, that is passed to the external executable via the environment variable ```GARM_PROVIDER_CONFIG_FILE```. This file is only relevant to the external provider. GARM itself does not read it. In the case of the sample OpenStack provider, this file contains access information for an OpenStack cloud (what you would typically find in a ```keystonerc``` file) as well as some provider specific options like whether or not to boot from volume and which tenant network to use. You can check out the [sample config file](../contrib/providers.d/openstack/keystonerc) in this repository.
The `environment_variables` option is a list of environment variables that will be passed to the external provider. By default GARM will pass a clean env to providers, consisting only of variables that the [provider interface](./external_provider.md) expects. However, in some situations, provider may need access to certain environment variables set in the env of GARM itself. This might be needed to enable access to IAM roles (ec2) or managed identity (azure). This option takes a list of environment variables or prefixes of environment variables that will be passed to the provider. For example, if you want to pass all environment variables that start with `AWS_` to the provider, you can set this option to `["AWS_"]`.
If you want to implement an external provider, you can use this file for anything you need to pass into the binary when ```GARM``` calls it to execute a particular operation.
### Available external providers

23
doc/quickstart.md
View file

@ -18,8 +18,6 @@
<!-- /TOC -->
This doc will be updated at a future date with the exact permissions needed in case you want to use a fine grained PAT.
## Create the config folder
All of our config files and data will be stored in `/etc/garm`. Let's create that folder:
@ -28,7 +26,7 @@ All of our config files and data will be stored in `/etc/garm`. Let's create tha
sudo mkdir -p /etc/garm
```
Coincidentally, this is also where the docker container [looks for the config](../Dockerfile#L29) when it starts up. You can either use `Docker` or you can set up garm directly on your system. I'll show you both ways. In both cases, we need to first create the config folder and a proper config file.
Coincidentally, this is also where the docker container [looks for the config](../Dockerfile#L29) when it starts up. You can either use `Docker` or you can set up garm directly on your system. We'll walk you through both options. In both cases, we need to first create the config folder and a proper config file.
## The config file
@ -100,15 +98,13 @@ This is where you have a decision to make. GARM has a number of providers you ca
* [Google Cloud Platform (GCP)](https://github.com/cloudbase/garm-provider-gcp)
* [Oracle Cloud Infrastructure (OCI)](https://github.com/cloudbase/garm-provider-oci)
All currently available providers are `external`.
The easiest provider to set up is probably the LXD or Incus provider. Incus is a fork of LXD so the functionality is identical (for now). For the purpose of this document, we'll continue with LXD. You don't need an account on an external cloud. You can just use your machine.
You will need to have LXD installed and configured. There is an excellent [getting started guide](https://documentation.ubuntu.com/lxd/en/latest/getting_started/) for LXD. Follow the instructions there to install and configure LXD, then come back here.
Once you have LXD installed and configured, you can add the provider section to your config file. If you're connecting to the `local` LXD installation, the [config snippet for the LXD provider](https://github.com/cloudbase/garm-provider-lxd/blob/4ee4e6fc579da4a292f40e0f7deca1e396e223d0/testdata/garm-provider-lxd.toml) will work out of the box. We'll be connecting using the unix socket so no further configuration will be needed.
Go ahead and create a new config somwhere where GARM can access it and paste that entire snippet. For the purposes of this doc, we'll assume you created a new file called `/etc/garm/garm-provider-lxd.toml`. That config file will be used by the provider itself. Remember, the providers are external executables that are called by GARM. They may have their own configs.
Go ahead and create a new config in a location where GARM can access it and paste that entire snippet. For the purposes of this doc, we'll assume you created a new file called `/etc/garm/garm-provider-lxd.toml`. That config file will be used by the provider itself. Remember, the providers are external executables that are called by GARM. They have their own configs which are relevant only to those executables, not GARM itself.
We now need to define the provider in the GARM config file and tell GARM how it can find both the provider binary and the provider specific config file. To do that, open the GARM config file `/etc/garm/config.toml` in your favorite editor and paste the following config snippet at the end:
@ -171,7 +167,7 @@ Adding the `garm` user to the LXD group will allow it to connect to the LXD unix
Next, download the latest release from the [releases page](https://github.com/cloudbase/garm/releases).
```bash
wget -q -O - https://github.com/cloudbase/garm/releases/download/v0.1.4/garm-linux-amd64.tgz | tar xzf - -C /usr/local/bin/
wget -q -O - https://github.com/cloudbase/garm/releases/download/v0.1.5/garm-linux-amd64.tgz | tar xzf - -C /usr/local/bin/
```
We'll be running under an unprivileged user. If we want to be able to listen on any port under `1024`, we'll have to set some capabilities on the binary:
@ -204,7 +200,7 @@ Copy the sample `systemd` service file:
```bash
wget -O /etc/systemd/system/garm.service \
https://raw.githubusercontent.com/cloudbase/garm/v0.1.4/contrib/garm.service
https://raw.githubusercontent.com/cloudbase/garm/v0.1.5/contrib/garm.service
```
Reload the `systemd` daemon and start the service:
@ -228,19 +224,18 @@ signal.NotifyContext(context.Background, [interrupt terminated])
2023/07/17 22:21:33 Loading provider lxd_local
2023/07/17 22:21:33 registering prometheus metrics collectors
2023/07/17 22:21:33 setting up metric routes
2023/07/17 22:21:35 ignoring unknown event
```
Excellent! We have a working GARM installation. Now we need to initialize the controller and set up the webhook in GitHub.
## Initializing GARM
Before we can start using GARM, we need initialize it. This will create the `admin` user and generate a unique controller ID that will identify this GARM installation. This process allows us to use multiple GARM installations with the same GitHub account. GARM will use the controller ID to identify the runners it creates. This way we won't run the risk of accidentally removing runners we don't manage.
Before we can start using GARM, we need initialize it. This will create the `admin` user and generate a unique controller ID that will identify this GARM installation. This process allows us to use multiple GARM installations with the same GitHub account, if we want or need to. GARM will use the controller ID to identify the runners it creates. This way we won't run the risk of accidentally removing runners we don't manage.
To initialize GARM, we'll use the `garm-cli` tool. You can download the latest release from the [releases page](https://github.com/cloudbase/garm/releases):
```bash
wget -q -O - https://github.com/cloudbase/garm/releases/download/v0.1.4/garm-cli-linux-amd64.tgz | tar xzf - -C /usr/local/bin/
wget -q -O - https://github.com/cloudbase/garm/releases/download/v0.1.5/garm-cli-linux-amd64.tgz | tar xzf - -C /usr/local/bin/
```
Now we can initialize GARM:
@ -339,7 +334,7 @@ In this exampe, we add a new github endpoint called `example`. The `ca-cert-path
Before we can add a new entity, we need github credentials to interact with that entity (manipulate runners, create webhooks, etc). Credentials are tied to a specific github endpoint. In this section we'll be adding credentials that are valid for either [github.com](https://github.com) or your own GHES server (if you added one in the previous section).
When creating a new entity (repo, org, enterprise) using the credentials you define here, GARM will automatically associate that entity with the gitHub endpoint the credentials use.
When creating a new entity (repo, org, enterprise) using the credentials you define here, GARM will automatically associate that entity with the gitHub endpoint that the credentials use.
If you want to swap the credentials for an entity, the new credentials will need to be associated with the same endpoint as the old credentials.
@ -431,7 +426,7 @@ gabriel@rock:~$ garm-cli repo ls
+--------------------------------------+----------+--------------+------------------+--------------------+------------------+
| ID | OWNER | NAME | CREDENTIALS NAME | POOL BALANCER TYPE | POOL MGR RUNNING |
+--------------------------------------+----------+--------------+------------------+--------------------+------------------+
| 0c91d9fd-2417-45d4-883c-05daeeaa8272 | gsamfira | scripts | gabriel | pack | true |
| 0c91d9fd-2417-45d4-883c-05daeeaa8272 | gsamfira | scripts | gabriel | roundrobin | true |
+--------------------------------------+----------+--------------+------------------+--------------------+------------------+
```
@ -516,7 +511,7 @@ gabriel@rock:~$ garm-cli pool ls -a
+--------------------------------------+---------------------------+--------------+-----------------+------------------+-------+---------+---------------+----------+
```
This pool is enabled, but the `min-idle-runners` option is set to 0. This means that it will not create any lingering runners. It will only create runners when a job is started. If your provider is slow to boot up new instances, you may want to set this to a value higher than 0.
This pool is enabled, but the `min-idle-runners` option is set to 0. This means that it will not create any idle runners. It will only create runners when a job is started and a webhook is sent to our GARM server. Optionally, you can set `min-idle-runners` to a value greater than 0, but keep in mind that depending on the provider you use, this may incur cost.
For the purposes of this guide, we'll increase it to 1 so we have a runner created.

14
doc/using_garm.md
View file

@ -56,16 +56,18 @@ You can list the controller info by running the following command:
```bash
garm-cli controller show
+------------------------+----------------------------------------------------------------------------+
+-------------------------+----------------------------------------------------------------------------+
| FIELD | VALUE |
+------------------------+----------------------------------------------------------------------------+
+-------------------------+----------------------------------------------------------------------------+
| Controller ID | a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
| Hostname | garm |
| Metadata URL | https://garm.example.com/api/v1/metadata |
| Callback URL | https://garm.example.com/api/v1/callbacks |
| Webhook Base URL | https://garm.example.com/webhooks |
| Controller Webhook URL | https://garm.example.com/webhooks/a4dd5f41-8e1e-42a7-af53-c0ba5ff6b0b3 |
+------------------------+----------------------------------------------------------------------------+
| Minimum Job Age Backoff | 30 |
| Version | v0.1.5 |
+-------------------------+----------------------------------------------------------------------------+
```
There are several things of interest in this output.
@ -76,6 +78,8 @@ There are several things of interest in this output.
* `Callback URL` - This URL is configured by the user, and is the URL that is presented to the runners via userdata when they get set up. Runners will connect to this URL and send status updates and system information (OS version, OS name, github runner agent ID, etc) to the controller. Runners must be able to connect to this URL.
* `Webhook Base URL` - This is the base URL for webhooks. It is configured by the user in the GARM config file. This URL can be called into by GitHub itself when hooks get triggered by a workflow. GARM needs to know when a new job is started in order to schedule the creation of a new runner. Job webhooks sent to this URL will be recorded by GARM and acted upon. While you can configure this URL directly in your GitHub repo settings, it is advised to use the `Controller Webhook URL` instead, as it is unique to each controller, and allows you to potentially install multiple GARM controller inside the same repo. Github must be able to connect to this URL.
* `Controller Webhook URL` - This is the URL that GitHub will call into when a webhook is triggered. This URL is unique to each GARM controller and is the preferred URL to use in order to receive webhooks from GitHub. It serves the same purpose as the `Webhook Base URL`, but is unique to each controller, allowing you to potentially install multiple GARM controllers inside the same repo. Github must be able to connect to this URL.
* `Minimum Job Age Backoff` - This is the job age in seconds, after which GARM will consider spinning up a new runner to handle it. By default GARM waits for 30 seconds after receiving a new job, before it spins up a runner. This delay is there to allow any existing idle runners (managed by GARM or not) to pick up the job, before reacting to it. This way we avoid being too eager and spin up a runner for a job that would have been picked up by an existing runner anyway. You can set this to 0 if you want GARM to react immediately.
* `Version` - This is the version of GARM that is running.
We will see the `Controller Webhook URL` later when we set up the GitHub repo to send webhooks to GARM.
@ -141,7 +145,7 @@ Each of these providers can be used to set up a runner pool for a repository, or
GARM can be used to manage runners for repos, orgs and enterprises hosted on `github.com` or on a GitHub Enterprise Server.
Endpoints are the way that GARM identifies where the credentials and entities you create, are located and where the API endpoints for the GitHub API can be reached, along with a possible CA certificate that validates the connection. There is a default endpoint for `github.com`, so you don't need to add it. But if you're using GHES, you'll need to add an endpoint for it.
Endpoints are the way that GARM identifies where the credentials and entities you create are located and where the API endpoints for the GitHub API can be reached, along with a possible CA certificate that validates the connection. There is a default endpoint for `github.com`, so you don't need to add it. But if you're using GHES, you'll need to add an endpoint for it.
### Creating a GitHub Endpoint
@ -214,7 +218,7 @@ garm-cli github endpoint show github.com
### Deleting a GitHub Endpoint
You can delete an endpoint unless one of the following conditions is met:
You can delete an endpoint unless any of the following conditions are met:
* The endpoint is the default endpoint for `github.com`
* The endpoint is in use by a repository, organization or enterprise