hidai25/EvalView - AI Agent Testing
Catch agent regressions in CI. Detects tool changes, output drift, cost spikes. Works with LangGraph, CrewAI, Claude, GPT.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Activelast commit Jul 3, 2026
- License
- Apache 2.0
Pinned Snippet
uses: hidai25/eval-view@c996a89324800c0d68b6d9629dd46adb0637c91b # v0.8.0tags can be moved; commit SHAs can't. why a SHA?
Inputs
| name | description | required | default |
|---|---|---|---|
| mode | Testing mode: "check" for regression detection (default), "run" for full evaluation | no | check |
| command | Alias for "mode" — accepts "check" or "run". If both are set, "command" takes precedence. | no | — |
| openai-api-key | OpenAI API key for LLM-as-judge evaluation | no | — |
| anthropic-api-key | Anthropic API key (optional alternative to OpenAI) | no | — |
| test-path | Path to test cases directory | no | tests |
| config-path | Path to EvalView config file | no | .evalview/config.yaml |
| filter | Filter tests by name pattern | no | — |
| max-workers | Maximum parallel workers | no | 4 |
| max-retries | Maximum retries for failed tests | no | 2 |
| fail-on-error | Fail the workflow if any test fails or regressions detected | no | true |
| fail-on | Comma-separated statuses that fail CI (REGRESSION, TOOLS_CHANGED, OUTPUT_CHANGED, CONTRACT_DRIFT) | no | REGRESSION |
| strict | Fail on any change (REGRESSION, TOOLS_CHANGED, OUTPUT_CHANGED) | no | false |
| diff | Compare against golden baselines in "run" mode (ignored in "check" mode which always diffs) | no | false |
| contracts | Check MCP contracts for interface drift before running tests | no | false |
| generate-report | Generate HTML report | no | true |
| post-comment | Automatically post results as PR comment (requires GITHUB_TOKEN) | no | true |
| upload-artifact | Upload results and report as workflow artifacts | no | true |
| evalview-version | EvalView version to install (e.g. "0.5.3", default: latest) | no | "" |
| python-version | Python version to use | no | 3.11 |
Outputs
| name | description |
|---|---|
| results-file | Path to JSON results file |
| report-file | Path to HTML report file |
| total-tests | Total number of tests run |
| passed-tests | Number of passed/unchanged tests |
| failed-tests | Number of failed/regressed tests |
| regressions | Number of regressions detected (check mode) |
| tools-changed | Number of tests with tool changes (check mode) |
| pass-rate | Pass rate percentage |
| status | Overall status: passed, regression, tools_changed, output_changed |