getsentry/vitest-evals GitHub Reporter

Publish vitest-evals results to GitHub Actions summaries, annotations, and checks

Trust Signals

name	description	required	default
results	Vitest JSON result files to report. Supports paths, * and ** globs, and newline-separated entries.	no	vitest-results.json
publish-summary	Write a GitHub Actions job summary.	no	true
publish-annotations	Emit GitHub workflow annotations for failed evals.	no	true
publish-check	Publish a GitHub Check Run for the combined report.	no	false
check-name	Name of the GitHub Check Run.	no	vitest-evals
github-token	GitHub token used for Check Run publishing.	no	${{ github.token }}
fail-on-failures	Fail the action when the combined eval report failed.	no	false
max-annotations	Maximum number of failure annotations to publish. Check Run annotations are capped at 50.	no	—
max-failures	Maximum number of detailed failures to include in summaries and checks.	no	—

name	description
status	Combined report status: passed or failed.
results-count	Number of Vitest JSON result files included.
evals-total	Total eval cases included in the report.
evals-failed	Failed eval cases included in the report.
score-average	Average eval score across the combined report.
check-url	URL of the published GitHub Check Run, when available.