Android GHA hosted POC
Objective
Evaluate GitHub Actions hosted runners as an alternative to the Mac Mini for Android E2E testing, following the finding that the Mac Mini is limited to 2 parallel Android runners and requires ongoing maintenance to stay stable.
How it works
GHA's ubuntu-latest runners support KVM hardware virtualisation, so the Android guest runs at near-native speed.
Each test suite runs as its own isolated VM via a matrix strategy. There's no shared hardware, no slot management, no port conflicts between jobs. The workflow expands the test list into parallel jobs, boots a fresh emulator per job, runs the suite, and tears everything down.
Runner spec: 2 vCPU · 8GB RAM · x86_64 · ubuntu-latest
Emulator config: API 36 · google_apis · x86_64 · Pixel 2 profile
GPU rendering: swiftshader_indirect (CPU-based)
Observed Timings
From an initial run (E2E: Login):
| Phase | Duration |
|---|---|
| Boot emulator + run test | 10m 24s |
| Total job (including setup, reporting) | 12m 50s |
These timings will vary by suite; Login is a representative mid-weight suite. Heavier suites may run longer.
GHA charges $0.006/min for a standard Linux runner (pricing). At ~13 min per job, a single suite costs ~$0.08. A full 21-suite regression run costs ~$1.68, all running in parallel, so total wall time is still ~13 min.
WARNING
A full regression run was not conducted. The screen size and OS version blockers prevented the full suite from running; so the estimates are very estimated.
Parallelism
Because each suite runs in its own VM, there's no parallelism cap imposed by shared hardware. All suites can run simultaneously; the only ceiling is GHA's concurrent job limit, which is plan-dependent.
This is the key advantage over the Mac Mini: instead of batching suites into waves (4 at a time for iOS, 2 for Android), every suite starts at once and the total run time is just however long the slowest suite takes, rather than multiplying suite duration by the number of waves.
Advantages Over Self-Hosted
| Mac Mini (Android) | GHA | |
|---|---|---|
| Max parallel runners | 2 | No shared hardware ceiling |
| Full regression time (estimated) | ~110 min (2 runners, 11 waves) | ~13 min (all parallel) |
| KVM acceleration | No (QEMU only) | Yes |
| Ephemeral environment | No (shared persistent state) | Yes (fresh VM per run) |
| Maintenance | Weekly reboots, slot scripts, memory management | None |
| Cost model | ~€125/month fixed | ~$0.006/min per job |
| Screen recording | Not feasible (CPU/RAM ceiling) | Viable (can capture per job) |
Limitations
No persistent state
Each job gets a fresh emulator with no pre-provisioned data. On the Mac Mini, admin users were hard-coded per runner slot. On GHA, admin users must be created dynamically as part of the test setup, a known blocker tracked in E2E Blockers.
Concurrent job limits
GHA enforces a concurrent job limit per plan. For a large test suite triggering on every PR push, it's worth checking whether the org's plan limit would cause jobs to queue rather than run immediately.
Why iOS stays on the Mac Mini
GHA macOS runners cost $0.062/min, over 10x the Linux rate.
At ~13 min per suite, that's ~$0.81/suite vs ~$0.08 for Android. A full 21-suite iOS regression would cost ~$17 per run, compared to ~$1.68 for Android.
The Mac Mini at a fixed monthly cost is significantly cheaper for iOS at any meaningful test volume.
Final Recommendation
GHA hosted runners are the right choice for Android E2E. The Mac Mini approach was limited to 2 parallel runners, required maintenance to stay stable, and would take ~110 min for a full regression run. GHA runs all suites in parallel, needs no maintenance, and costs ~$1.68 per full regression.