felmonon/Agent Reliability Harness
Validate agent tool-use traces against a reliability policy and (optionally) compare against a saved baseline to gate the workflow on real regressions. Runs fully locally: no telemetry, no network calls, no secrets required.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| policy | Path to the policy JSON file | yes | — |
| traces | Space-separated trace file paths or globs | yes | — |
| format | Trace format: auto | arh | openai-chat | anthropic-messages | no | auto |
| baseline | Optional path to a baseline JSON report (from a previous 'arh validate --json-out'). When set, the action runs 'arh compare' and gates on regressions instead of absolute failures. | no | "" |
| fail-on | Gate mode for compare: regressions | failures | never | no | regressions |
| max-score-drop | Optional per-trace score-drop limit for the regressions gate | no | "" |
| fail-under | Minimum passing score for validate (0-100) | no | 70.0 |
| output-dir | Directory for report artifacts | no | arh-reports |
Outputs
| name | description |
|---|---|
| json-report | Path to the JSON validation report (next baseline) |
| sarif-report | Path to the SARIF report (upload with codeql-action/upload-sarif) |
| junit-report | Path to the JUnit XML report |
| compare-report | Path to the comparison JSON (only when baseline was given) |