madhvansh/Preflight IR evaluation run
Preflight TREC-style qrels/runs for malformed data, coverage differences, rank diagnostics, and task mismatches.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| qrels | Path to a TREC qrels file in the calling repository. | yes | — |
| run | Path to a TREC run file in the calling repository. | yes | — |
| expected-task | If set, require both '# task: ...' headers to match this value. | no | "" |
| require-query-coverage | Fail instead of warn when qrels and run query sets differ. | no | false |
| require-judged | Fail instead of warn when a run document is absent from qrels. | no | false |
| strict-ranks | Fail instead of warn on advisory rank-column anomalies. | no | false |
Outputs
no outputs