yashchimata/judgegate
Statistical trust gate for LLM judges. Fails CI when the judge does not agree with human labels well enough to gate.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| labels | Path to the human-labeled examples file. | yes | — |
| config | Path to the judge.yaml configuration. | yes | — |
| min-kappa | Trust threshold override. | no | "" |
| alpha | Significance level override. | no | "" |
| no-probes | Skip the probe battery. | no | false |
| comment | Post the report as a sticky pull request comment. | no | true |
| fail-on-inconclusive | Fail the check when there are too few labels to decide. | no | false |
| github-token | Token used to post the pull request comment. | no | ${{ github.token }} |
| python-version | Python version used to run judgegate. | no | 3.12 |
Outputs
| name | description |
|---|---|
| verdict | TRUSTED, UNTRUSTED, INCONCLUSIVE, or ERROR. |
| exit-code | Raw judgegate exit code (0, 1, 2, or 3). |