safeintelligence/Run Spec27 Eval
Start a Spec27 eval run, poll until it finishes, and write the final result JSON.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| base_url | Base URL for the Spec27 deployment. | no | https://dashboard.spec27.ai |
| token | Project-scoped Spec27 API token. | yes | — |
| eval_id | ID of the eval to run. | yes | — |
| poll_interval_seconds | Seconds to wait between status polls. | no | 10 |
| timeout_seconds | Maximum seconds to wait for the eval run to reach a terminal status. | no | 1800 |
| results_path | Path to write the detailed eval-run JSON results file. Defaults to a unique file in the runner temp directory so the workspace stays clean. | no | "" |
Outputs
| name | description |
|---|---|
| eval_run_id | ID of the created eval run. |
| status | Final eval-run status. |
| clean_accuracy | Clean accuracy from the final eval-run response, when present. |
| robust_accuracy | Robust accuracy from the final eval-run response, when present. |
| result_url | URL for the eval-run result in Spec27, when present. |
| summary_json | Compact eval-run summary as a JSON string, when present. |
| results_file | Absolute path to the detailed eval-run JSON results file written by the action. |