Runner Node Requirements

A runner node runs sandbox virtual machines. Every sandbox is a real Linux VM under QEMU, so the node needs virtualization support, three devices, and a handful of kernel settings.

Summary

Choose the accelerator

The runner takes no guess. Set sandboxRunner.acceleration to kvm or to tcg. There is no default and no probe.

Take kvm unless the host cannot offer it. The explicit setting is deliberate: a node that quietly drops to emulation still serves sandboxes, so the fault shows up as "Tektona is slow" weeks later instead of as a failed rollout.

The speed of the host decides the speed of the sandbox

Nested virtualization is the single biggest lever. A runner that is itself a virtual machine boots sandboxes about 2.4 times slower than bare metal, even on newer silicon.

Same sandbox image, 4 vCPU and 4 GiB, on two clusters:

Compute is at parity. The loss is in the interrupt-heavy boot path, because a hypervisor usually does not pass interrupt virtualization through to a nested guest. So a sandbox that starts, pauses and resumes often pays the cost many times, while a long-running build barely notices it.

Software emulation costs more again. Use tcg for a proof of concept, not for people waiting on a sandbox.

Linux configuration

The following applies to Ubuntu 24.04 LTS, which is the tested distribution.

Kernel modules

Load these, and make them persist across a reboot in /etc/modules-load.d/tektona.conf:

Without vhost_vsock the runner pod never starts: it mounts /dev/vhost-vsock as a character device, and the mount fails.

Devices

Kernel settings

In /etc/sysctl.d/:

net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
kernel.apparmor_restrict_unprivileged_userns = 0
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 524288

The inotify limits matter: each sandbox holds watches, and the default limits run out well before the node does.

Packages

curl, jq, socat, conntrack, ipset, open-iscsi, ca-certificates.

Other host settings

• Swap off. The kubelet requires it, and a swapping VM host is unusable.
• Disable rpcbind and remove nfs-common, unless you need them. Port 111 is reflected in denial of service attacks.
• Power policy set to performance. Deep C-states add latency to VM entry and exit.

Check a candidate node

# 1. Bare metal, or a virtual machine?
systemd-detect-virt          # "none" means bare metal

# 2. Is KVM usable?
ls -l /dev/kvm
kvm-ok                       # from the cpu-checker package

# 3. Which acceleration features does KVM expose?
#    Use kvm_amd and npt on AMD hosts.
systool -m kvm_intel -v 2>/dev/null | grep -E "ept|unrestricted_guest|enable_apicv|flexpriority|pml"

# 4. The other devices.
ls -l /dev/vsock /dev/net/tun
lsmod | grep -E "vhost_vsock|kvm"

A node that reports none for the first command, and Y for ept, unrestricted_guest and enable_apicv, is a good runner.

Label the node

The runner workloads only schedule on labelled nodes:

kubectl label node <node> tektona.ai/sandbox-runner=true

Mixed fleets

sandboxRunner.acceleration applies to the whole release. A fleet with both kinds of node needs two runner releases, each with its own node label and its own value. Do not mix them under one release: every node in it would take the same accelerator, and half of them would be wrong.