lennardzuendorf/pytest Benchmark Baseline Check
Regression-gate pytest-benchmark with per-branch baselines, PR comparison comments, and an override label.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Activelast commit Jul 5, 2026
- License
- MIT
Pinned Snippet
uses: lennardzuendorf/pytest-bench-action@4e2cb79f46f0395efcae595d2b1e26cadead3bda # no releases — HEAD as of 2026-07-11tags can be moved; commit SHAs can't. why a SHA?
Inputs
| name | description | required | default |
|---|---|---|---|
| python-version | Python version for consistent benchmarks | no | 3.14 |
| benchmark-run-command | Full shell command to run benchmarks (must write --benchmark-json output) | yes | — |
| setup-command | Shell command to install dependencies before running | no | "" |
| pre-benchmark-command | Optional warm-up command run after deps install | no | "" |
| benchmark-results-file | Path to the JSON output from pytest-benchmark | no | benchmark-results.json |
| cross-branch-tolerance | % increase allowed vs main branch baseline before failing | no | 20 |
| update-tolerance | % change threshold that triggers a baseline update commit | no | 5 |
| baselines-dir | Repo-relative path where baseline JSONs are stored | no | .benchmarks/baselines |
| github-token | GitHub token with contents:write and pull-requests:write | yes | — |
| threshold-map | JSON string mapping test name substrings to max-seconds thresholds. e.g. {"e2e_create": 30.0, "help": 0.5}. Empty string defaults all tests to 1.0s. | no | "" |
| enforce-same-node | How to treat a hardware mismatch between a baseline and the current run. Comparability is judged on the CPU/system fingerprint (cpu.brand_raw + arch + core count + system), NOT the hostname — so the same machine compares cleanly even when its runner node name changes between jobs. On a genuine hardware mismatch: "true" hard-fails the job (correct for stable/self-hosted runners); "false" (default) skips that comparison with a ::warning:: log line and the comparison-skipped output instead of failing. Either way the action never emits a cross-machine comparison. | no | false |
| override-label | Pull-request label that waives benchmark regressions for that one PR. When the triggering PR carries this label, a detected regression is still shown in the PR comment but does NOT fail the job (regression-overridden=true). Scoped per-PR and self-clearing — remove the label to re-enforce. Only applies on pull_request events. | no | benchmark-override |
Outputs
| name | description |
|---|---|
| regression-detected | "true" if any regression was detected |
| baseline-updated | "true" if the baseline was updated and committed |
| node | Hostname extracted from machine_info.node in the results JSON |
| comparison-skipped | "true" if a baseline comparison was skipped because it ran on a different runner node (enforce-same-node: false) |
| regression-overridden | "true" if a regression was detected but waived by the override label on the PR |