siddhant-k-code/agent-trace eval
Run agent-trace evals in CI, post results to the step summary, and fail on regression.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| config | Path to eval config file | — | .agent-evals.yaml |
| baseline | Path to baseline scores file (optional) | — | "" |
| save-baseline | Save scores as new baseline after run | — | false |
| tolerance | Regression tolerance (0.0–1.0, default 0.05) | — | 0.05 |
| trace-dir | Directory where agent-trace sessions are stored | — | .agent-traces |
| python-version | Python version to use | — | 3.12 |
| install-extras | Comma-separated optional extras to install (e.g. openai,anthropic) | — | "" |
Outputs
| name | description |
|---|---|
| passed | true if all scorers passed, false otherwise |
| summary-path | Path to the written eval summary markdown file |