jadenfix/Beater Eval Gate
Run a Beater statistical eval gate in CI and post a pass / fail / INCONCLUSIVE verdict — with effect size, confidence interval, p-value, and (when underpowered) the minimum detectable effect and how many more cases would make the comparison conclusive — to the step summary and, optionally, a sticky PR comment. Runs entirely against the local `--data-dir` store: no Beater server, no API keys.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| data-dir | Beater data directory holding gates.sqlite / experiments.sqlite, produced by the eval steps earlier in the job (or by demo-fixture mode). | no | .beater |
| tenant-id | Tenant id the gate belongs to. | no | demo |
| project-id | Project id the gate belongs to. | no | demo |
| gate-id | Gate id to run. | no | main |
| experiment-run-id | Specific experiment run to gate on. Empty = the gate's latest experiment run. | no | "" |
| beaterctl-path | Path to a prebuilt beaterctl binary. Empty = build it from this action's checkout with cargo (requires a Rust toolchain on the runner; the first uncached build is slow — prefer a prebuilt binary or a cargo cache in latency-sensitive workflows). | no | "" |
| demo-fixture | "true" seeds the deterministic demo gate + experiments (`beaterctl gate-run-fixture`) into data-dir before running. Zero-config demo/self-test mode; uses tenant/project/gate demo/demo/main. | no | false |
| fail-on-gate-failure | "true" (default) fails the step when the gate does not pass, so the gate can be a required check. "false" always succeeds and only reports — the verdict is still available via outputs. | no | true |
| github-token | Token used to upsert a sticky verdict comment on the pull request. Empty = no comment (the step summary is always written). Needs `pull-requests: write`. | no | "" |
| comment-tag | Marker distinguishing this gate's sticky comment when a workflow runs several gates on one PR. | no | default |
Outputs
| name | description |
|---|---|
| verdict | pass | fail | inconclusive |
| passed | true when the gate passed under its inconclusive policy |
| decision | Raw GateDecision: pass | fail_regression | inconclusive |
| reason | The gate's human-readable reason string |
| report-path | Path to the raw gate-run JSON report |