popey/Tessl Skill Eval

Run Tessl evals against tiles with scenarios when SKILL.md files change in a PR

Trust Signals

name	description	required	default
enabled	Enable eval runs. Set to false to disable entirely.	—	true
skip-label	PR label that skips eval even when enabled. Set empty to disable.	—	skip-eval
path	Root path to search for SKILL.md files	—	.
comment	Whether to post results as a PR comment	—	true
eval-workspace	Tessl workspace name. Optional when tiles have workspace set in tile.json.	—	""
eval-agent	Agent:model pair for evals (e.g. claude:claude-sonnet-4-6)	—	claude:claude-sonnet-4-6
eval-timeout	Max minutes to wait for each eval run to complete	—	45
eval-fail-on-regression	Fail the check if any scenario scores worse with context than baseline	—	true
eval-generate-scenarios	Generate fresh scenarios before running evals	—	false
eval-scenario-count	Number of scenarios to generate per tile	—	3
eval-commit-scenarios	Commit generated scenarios back to the PR branch (requires contents: write)	—	false
tessl-token	Tessl API token for authentication. Pass via secrets.	yes	—

no outputs