E2E Testing on Self-Hosted Runners

This guide covers how to set up a self-hosted runner for E2E tests and how to run tests locally.

iOS Support Coming Soon

iOS E2E testing is not yet available. This guide currently covers Android only. iOS support will be added in a future update.

Architecture

Self-hosted runner (macOS)
  ├── Android Emulator
  └── Appium server (UiAutomator2 driver)
        └── Runs test specs against emulator

Runner Requirements

• macOS (Apple Silicon or Intel)
• Android Studio with SDK, emulator, and platform-tools
• Node.js (version matching project)
• Appium installed globally: npm install -g appium
• Appium driver: appium driver install uiautomator2
• GitHub CLI (gh) installed and authenticated

Quick Setup

Use the setup script to configure one or more runners on a machine:

./scripts/e2e-runner/setup-parallel-runners.sh

This script:

• Downloads and configures GitHub Actions runners
• Creates slot-specific environment variables (ports, AVD names)
• Registers runners with the repository
• Optionally installs runners as launchd services (auto-start on boot)

Prerequisites for the script

1. Authenticate GitHub CLI: gh auth login
2. Create the required AVDs (see below)

Android Emulator Setup

# Install system image
sdkmanager "system-images;android-34;google_apis;arm64-v8a"

# Create AVD for first runner
avdmanager create avd -n Pixel_6 -k "system-images;android-34;google_apis;arm64-v8a" -d pixel_6

# Create AVD for second runner (if scaling)
avdmanager create avd -n Pixel_6_Runner2 -k "system-images;android-34;google_apis;arm64-v8a" -d pixel_6

# Verify
emulator -list-avds

Scaling to Multiple Runners

Multiple runners can run on the same machine by using different ports and AVDs. The setup script configures two runner slots by default:

Slot	AVD	Appium Port	ADB Port	System Port
1	Pixel_6	4723	5554	8200
2	Pixel_6_Runner2	4724	5556	8201

GitHub Actions automatically distributes jobs across idle runners. When two PRs trigger E2E tests simultaneously, each runs on a separate runner slot.

To add more slots, edit the slot configurations in scripts/setup-parallel-runners.sh.

Environment Variables

Each runner slot needs unique ports to avoid conflicts:

Variable	Default	Purpose
APPIUM_PORT	4723	Appium server port
ANDROID_AVD_NAME	Pixel_6	Android Virtual Device name
ANDROID_ADB_PORT	5554	ADB console port for emulator
ANDROID_SYSTEM_PORT	8200	UiAutomator2 system port

The setup script creates a .env file for each runner and injects these into the launchd service plist.

Running Tests Locally

1. Start the emulator

$ANDROID_HOME/emulator/emulator -avd Pixel_6 -no-window -no-audio -no-snapshot -gpu swiftshader_indirect &

# Wait for boot
adb wait-for-device
adb shell getprop sys.boot_completed  # Should return "1"

2. Start Appium

appium --relaxed-security -p 4723 &

# Verify it's running
curl -s http://localhost:4723/status

3. Run tests

cd e2eAppium
npm install
npm run appium:android -- --spec tests/ui/login/channelSwitcher.js

Using non-default configuration

APPIUM_PORT=4724 ANDROID_AVD_NAME=Pixel_7 npm run appium:android

App Builds

Tests require an APK in e2eAppium/appBuilds/android/. The CI workflow downloads this automatically from EAS. For local testing:

cd e2eAppium
ANDROID_BUILD_URL=<eas-build-url> npm run download:androidApp

Secrets

Test credentials are stored in Doppler. For local runs, create e2eAppium/.env manually (see e2eAppium/.env.example for required keys).

Managing Runners

Start runners

# If installed as a service
cd ~/actions-runners/e2e-runner-1
./svc.sh start

# Or run interactively (useful for debugging)
cd ~/actions-runners/e2e-runner-1
./run.sh

Stop runners

# If running as a service
cd ~/actions-runners/e2e-runner-1
./svc.sh stop

# If running interactively, press Ctrl+C or:
pkill -f "Runner.Listener"

Check status

cd ~/actions-runners/e2e-runner-1
./svc.sh status

# Or check all runner processes
ps aux | grep Runner.Listener

Remove a runner

# Get a removal token
TOKEN=$(gh api -X POST repos/zuno-tech/installer-app/actions/runners/registration-token --jq '.token')

cd ~/actions-runners/e2e-runner-1
./svc.sh uninstall  # If installed as service
./config.sh remove --token "$TOKEN"

Security Considerations

Self-hosted runners execute code from the repository, so securing them is critical.

Repository access

• Never use self-hosted runners on public repositories. Anyone who can open a PR can run arbitrary code on the runner.
• Limit who can trigger workflows. Use pull_request_target with caution—it runs in the context of the base branch but can be exploited.

Runner isolation

• Dedicated machine recommended. Don't run self-hosted runners on developer laptops or machines with access to sensitive systems.
• Avoid storing secrets on the runner filesystem. Use GitHub Actions secrets or Doppler instead.
• Consider running each job in a fresh environment. The runner reuses the working directory between jobs—clean up sensitive artifacts.

Network security

• Place the runner on a network segment with limited access to internal systems.
• The runner only needs outbound HTTPS to GitHub (no inbound ports required).
• Consider a firewall rule allowing only GitHub's IP ranges if your network permits.

Credential hygiene

• Don't store long-lived credentials on the runner. Prefer short-lived tokens.
• The runner's registration token expires in 1 hour—don't commit it anywhere.
• Rotate the runner periodically by removing and re-registering it.

Monitoring

• Monitor runner logs for unexpected activity: ~/actions-runners/e2e-runner-1/_diag/
• Set up alerts if the runner goes offline unexpectedly.
• Review workflow runs periodically for suspicious commands.

GitHub recommendations

See GitHub's official guidance: Self-hosted runner security

Troubleshooting

Appium fails to start

# Check if port is already in use
lsof -i :4723
# Kill existing process
lsof -ti:4723 | xargs kill -9

Emulator not found

# Verify AVD exists
emulator -list-avds
# Verify ANDROID_HOME is set
echo $ANDROID_HOME

Tests hang or timeout

# Check emulator is booted
adb devices
adb shell getprop sys.boot_completed

# Check Appium is responding
curl http://localhost:4723/status