kochrisdev/vapviz-eval
Run a vapviz eval suite against traced agent runs and fail the build on regressions. Point it at a SQLite store, a single exported run file, or a directory of exported runs.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| suite | Path to the eval suite file (YAML or JSON). | yes | — |
| db | SQLite store to load runs from (mutually exclusive with run / runs-dir). | no | "" |
| run | A single exported run JSON file (GET /runs/{id}/export format). | no | "" |
| runs-dir | A directory of exported run JSON files (*.json). | no | "" |
| run-id | Only evaluate this run id (with db). | no | "" |
| label | Only evaluate runs with this label (with db). | no | "" |
| tag | Only evaluate runs carrying this tag (with db). | no | "" |
| latest | Only evaluate the most recent matching run (with db). | no | false |
| version | vapviz version spec to install, e.g. "1.2.0" or ">=1.2,<2". Empty installs the latest release. | no | "" |
| python-version | Python version to set up. | no | 3.12 |
Outputs
no outputs