bihanikeshav/promptry eval
Run promptry eval suites in CI. Catches LLM regressions and comments on PRs.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| suite | Eval suite to run | yes | — |
| module | Python module containing the eval suite | yes | — |
| compare | Baseline to compare against (e.g. prod) | no | "" |
| python-version | Python version to use | no | 3.12 |
| extras | pip extras to install with promptry | no | semantic |
| pr-comment | Post a comment on the PR with results (default: true) | no | true |
| github-token | GitHub token for posting PR comments | no | ${{ github.token }} |
Outputs
no outputs