skill-bench/Skill Eval
Evaluate Claude Code skills against YAML test cases with automated grading and PR reporting
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| skill-name | Name of the skill to evaluate | yes | — |
| skill-path | Path to the skill directory (containing SKILL.md and evals/) | yes | — |
| anthropic-api-key | Anthropic API key for claude CLI | yes | — |
| pass-threshold | Minimum pass rate (0-100) to succeed | no | 80 |
| timeout | Timeout per eval case in seconds | no | 120 |
| allowed-tools | Comma/space-separated tool allow-list granted to the skill under test (e.g. "Bash(kubectl get:*),Bash(gh api:*),Read"). Forwarded to claude --allowedTools. Per-case allowed_tools overrides this. | no | "" |
| permission-mode | Permission mode for the skill under test (default, acceptEdits, plan, bypassPermissions). Forwarded to claude --permission-mode. Per-case permission_mode overrides this. | no | "" |
| post-comment | Post results as a PR comment | no | true |
| github-token | GitHub token for posting PR comments | no | ${{ github.token }} |
| upload-viewer | Upload eval-viewer HTML as an artifact | no | true |
| node-version | Node.js version for claude CLI installation | no | 22 |
| max-retries | Max retry attempts per API call on timeout/error | no | 3 |
| retry-delay | Base delay between retries in seconds (multiplied by attempt number) | no | 10 |
Outputs
| name | description |
|---|---|
| pass-rate | Overall pass rate as percentage (0-100) |
| passed | Total criteria passed |
| total | Total criteria evaluated |
| cases-run | Number of eval cases executed |