jarjarbeatyourattitude/Evalt performance gate

Find or verify the lowest-cost AI route that clears your frozen quality contract.

View on GitHub

Trust Signals

Scorecard Score
not yet scored
Maintenance Recency
Stale
License
None
namedescriptionrequireddefault
suitePath to the versioned Evalt suite.noevalt.json
resultPath where the machine-readable Evalt result is written or read.noevalt-result.json
baselineOptional earlier result from the identical frozen suite. Enables fail-closed regression gating without provider calls.no""
library-rootOptional private local evidence-library directory used by @name suite or baseline references.no""
optimizeRun the budget-capped tournament before applying the gate. Set false to gate an existing result without provider calls.notrue
evalt-versionExact PyPI version to install. Mutable aliases such as latest are rejected.no0.11.0
python-versionPython version used by the action.no3.12
min-pass-rateRequired final-test pass rate from 0 through 1.no0.95
max-cost-per-successOptional maximum USD cost per successful production call.no""
require-complete-coverageFail when the tournament did not finish all decision-relevant coverage.notrue
max-regressionsMaximum previously passing frozen cases allowed to fail when baseline is set.no0
max-quality-drop-ppMaximum aggregate final-test quality drop in percentage points when baseline is set.no0
max-cost-increase-pctOptional maximum production cost increase versus baseline, in percent.no""
max-p90-increase-msOptional maximum p90 latency increase versus baseline, in milliseconds.no""
fixed-promptDisable prompt rewrites and few-shot search while still comparing model routes.nofalse
max-parallel-modelsOptional override for parallel model lanes.no""
max-parallel-scenariosOptional override for parallel scenario executions per model.no""
request-timeout-secondsOptional wall-clock deadline for one provider response, not the tournament.no""
custom-scorer-idOptional trusted local scorer ID. The suite cannot select executable code.no""
custom-scorer-versionVersion of the trusted local scorer registered for this run.no""
custom-scorer-executableExecutable for the trusted repository-local scorer; launched directly without a shell.no""
custom-scorer-arguments-jsonJSON array of literal argv items passed to the scorer executable.no[]
custom-scorer-timeout-secondsPer-case local scorer timeout in seconds.no10
custom-scorer-max-input-bytesMaximum serialized scorer request size per case.no8388608
custom-scorer-max-output-bytesMaximum scorer stdout or stderr size per case.no65536
webhook-urlOptional explicit HTTPS destination for one signed aggregate CI decision event.no""
webhook-secretSigning secret. Pass a GitHub Actions secret; it is never written to Evalt artifacts or audit files.no""
webhook-destination-idOpaque destination label retained in the local delivery audit.nogithub
webhook-auditLocal aggregate webhook delivery audit JSONL.no.evalt/webhook-deliveries.jsonl
webhook-timeout-secondsPer-attempt HTTPS timeout, from greater than zero through 30 seconds.no5
webhook-max-attemptsDelivery attempts from 1 through 5.no3
webhook-requiredFail the Action when the configured aggregate event cannot be delivered.nofalse
webhook-include-route-nameInclude the local route name instead of only its opaque reference.nofalse
webhook-allow-private-networkExplicitly allow a private/local destination address. Disabled by default.nofalse
workspace-tokenOptional evc_ publisher capability used to synchronize aggregate route progress. Pass a GitHub Actions secret, never an owner capability.no""
workspace-api-urlHTTPS Evalt workspace API receiving privacy-bounded route metadata.nohttps://evalt.onrender.com
html-reportPath for the self-contained HTML evidence report.noevalt-report.html
junit-reportPath for the case-level JUnit report.noevalt-junit.xml
namedescription
statusPASS, FAIL, or ERROR.
selected-modelModel selected in the saved result, if one qualified.
route-versionQualified route package ID when the result contains one; otherwise empty.
final-test-pass-rateMeasured final-test pass rate.
case-regressionsCount of previously passing frozen cases that failed, when a baseline was set.
quality-delta-ppCandidate quality change versus baseline in percentage points.
cost-per-1000-successful-calls-usdMeasured production cost per 1,000 successful calls.
resultPath to the machine-readable result.
html-reportPath to the self-contained HTML report.
junit-reportPath to the case-level JUnit report.
webhook-deliveredtrue or false when a webhook is configured; empty otherwise.
webhook-event-idStable event and idempotency ID when a webhook event was built.