alepot55/Agentrial CI
Statistical evaluation for AI agents - test your agents with confidence intervals and failure attribution
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| trials | Number of trials per test case | no | 10 |
| threshold | Minimum pass rate (0-1) to pass CI | no | 0.9 |
| config | Path to agentrial.yml config file | no | agentrial.yml |
| test-path | Path to test files or directory | no | tests/ |
| python-version | Python version to use | no | 3.11 |
| post-comment | Post results as PR comment | no | true |
Outputs
| name | description |
|---|---|
| pass-rate | Overall pass rate |
| passed | Whether the evaluation passed the threshold |
| results-file | Path to JSON results file |