bruno-dasilva/bar-benchmarks
Run a Beyond All Reason performance scenario across N identical GCP Batch VMs and collect aggregate stats. Wraps `bar-bench run` as a composite action; expects Google Cloud ADC to already be configured in the job (e.g. via google-github-actions/auth@v2).
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Activelast commit Jun 18, 2026
- License
- None
Pinned Snippet
uses: bruno-dasilva/bar-benchmarks@9a28fb23a34f34fe86d0a4d723559f225f858f52 # no releases — HEAD as of 2026-07-11tags can be moved; commit SHAs can't. why a SHA?
Inputs
| name | description | required | default |
|---|---|---|---|
| scenario | Folder name under benchmarks/ (e.g. lategame1). | yes | — |
| map | Map catalog name (e.g. hellas-basin-v1.4). | yes | — |
| engine | Engine catalog name from artifacts.toml. Mutually exclusive with engine-commit. | no | "" |
| engine-commit | RecoilEngine commit SHA (>=7 chars hex). Ad-hoc; synthesizes a catalog entry. | no | "" |
| engine-repo | GitHub repo to query for the "Build Engine v2" run + artifact when building from a commit. Forwarded to scripts/build-engine.sh --repo. Only meaningful with engine-commit. Defaults to the calling workflow's own repo so the action works unchanged on a fork; set explicitly to point at a different RecoilEngine fork. | no | ${{ github.repository }} |
| bar-content | Bar-content catalog name. Mutually exclusive with bar-content-version. | no | "" |
| bar-content-version | Full BAR content version string, e.g. 'Beyond All Reason test-29871-90f4bc1'. Ad-hoc; synthesizes a catalog entry. | no | "" |
| count | Number of Batch VMs (tasks) to spawn. | no | 20 |
| iterations | Engine invocations per VM after staging. Each iteration produces a separate benchmark result, at the cost of roughly one engine wall time per iter. Staging cost (engine + bar-content + overlay + map) is paid once. Bump max-run-duration accordingly. | no | 1 |
| description | One-line free-form description shown in the aggregate report. | no | "" |
| gcp-project | GCP project ID that owns the Batch job + buckets. | yes | — |
| gcp-region | GCP region for the Batch job. | no | us-central1 |
| artifacts-bucket | gs:// URI of the artifacts bucket. | no | gs://bar-experiments-bench-artifacts |
| results-bucket | gs:// URI of the results bucket. | no | gs://bar-experiments-bench-results |
| machine-type | GCE machine type for each VM. Default is c2d-standard-16 (16 vCPU, AMD EPYC Milan), which matches the orchestrator's per-task CPU claim so each benchmark gets a whole VM. | no | c2d-standard-16 |
| service-account | Service account email to attach to Batch VMs. Default derives from gcp-project. | no | "" |
| max-run-duration | Per-task Batch timeout in seconds. | no | 1800 |
| github-token | Token used by scripts/build-engine.sh to download RecoilEngine GitHub Actions artifacts. Required only when engine-commit is set and the engine tarball is not already cached. For cross-repo access you may need a PAT with `actions:read` on beyond-all-reason/RecoilEngine. | no | ${{ github.token }} |
| lookup-prior-results | If 'true', aggregate the rolling window of recent matching runs (same engine, bar-content, map, scenario, machine-type) and skip a fresh Batch submission when the pool already has at least lookup-min-samples valid results. The rolling aggregate is what the action's mean-ms / p95-ms / etc. outputs reflect either way — with or without a fresh run on top. | no | false |
| lookup-scan-limit | Max recent job UIDs to inspect when lookup-prior-results is true. | no | 250 |
| lookup-min-samples | Skip the Batch submission when the rolling window has at least this many valid results. | no | 50 |
Outputs
| name | description |
|---|---|
| job-uid | Batch job UID, e.g. bar-bench-1713831234-a1b2c3. |
| prior-run-hit | 'true' if lookup-prior-results found a match and skipped the Batch run. |
| results-gcs-uri | gs:// prefix under which per-task results.json blobs live. |
| report-json-path | Path to the full structured BatchReport JSON on the runner. |
| mean-ms | Across-VM mean sim frame time in milliseconds. |
| p95-ms | Across-VM p95 sim frame time in milliseconds. |
| median-ms | Across-VM median sim frame time in milliseconds. |
| stddev-ms | Pooled across-VM sim frame time stddev in milliseconds. |
| valid-count | Number of VMs that produced a valid results.json. |
| invalid-count | Number of VMs that failed or uploaded an invalid results.json. |