attenlabs/hotato conversation QA

Gate CI on your voice agent's conversations: run hotato suites, tests, or contracts offline.

Trust Signals

Scorecard Score: not yet scored
Maintenance Recency: Stale
License: None

Inputs

name	description	required	default
suite	Workspace-relative path to a committed hotato suite file. Exactly one of suite, test, contracts is required.	no	""
test	Workspace-relative path to a committed conversation-test file. Exactly one of suite, test, contracts is required.	no	""
contracts	Workspace-relative directory of hotato contract bundles to verify. Exactly one of suite, test, contracts is required.	no	""
agent	Agent id under test (required for suite and test runs); a safe identifier recorded on the results.	no	""
release	Release id the runs are recorded under; defaults to the commit SHA of the triggering workflow.	no	""
output	Workspace-relative results directory.	no	.hotato/results
parallel	Max worker threads for a suite run (1 to 64); the default is hotato's own CPU-based cap. The worker count never changes the result bytes.	no	""
gate-advisory	Set true on a test run to pass hotato's --gate-judge flag, opting the exit code into the model-judged rubric lane. Default false; the advisory lane then reports without gating.	no	false
transcript	Workspace-relative transcript JSON used as evidence for a test run, or as embedded-assertion context for a contracts run.	no	""
trace	Workspace-relative voice-trace JSONL used as evidence for a test run.	no	""
state	Workspace-relative state-adapter sandbox used as evidence for a test run.	no	""
hotato-version	What to install. "action" (default) installs the pinned Action revision itself with zero package-index egress (PYTHONPATH); "preinstalled" skips installation; an exact version such as "1.3.3" installs that pin from PyPI with --no-deps. Ranges and "latest" are refused.	no	action
render-records	Render one share-safe Failure Record per non-passing unit. Set false to disable local artifact generation.	no	true
record-limit	Resource-safety cap on how many Failure Records a single run renders (the first N non-passing units in source order); an integer from 1 to 500. When a run exceeds it the index reports the total and marks the set truncated -- omission is never silent.	no	100
python-version	Python version for actions/setup-python.	no	3.12
comment-pr	Opt in to posting the share-safe contract-verify PR-comment block onto one sticky pull-request comment (applies to a contracts run). Default false: the Action then reads no token and posts nothing. When true, the comment step is fail-open and token-gated -- it updates one comment in place and can never change the gate exit code.	no	false
github-token	Token used ONLY by the opt-in PR-comment step (comment-pr: true); the workflow must also grant pull-requests: write. Unused when comment-pr is false.	no	${{ github.token }}

Outputs

name	description
output	The validated workspace-relative results directory.
suite-result	Workspace-relative path of the machine result JSON (suite-run.json, test-run.json, or contract-verify.json).
summary	Workspace-relative path of the rendered Markdown summary.
records	Workspace-relative Failure Record directory when produced, else empty.
records-index	Workspace-relative path of the record-set index.json when records were produced, else empty.
records-count	The number of Failure Records rendered, as a string.
records-total	The total number of non-passing units the source contained, as a string (a rendered count below this means the set was truncated by record-limit).
exit-code	The hotato process exit code, as a string.
status	pass, fail, inconclusive, or error, read from the machine result (presentation only; the exit code owns the gate).
hotato-version	The executed hotato package version.
pr-comment	Workspace-relative path of the rendered share-safe PR-comment block on a contracts run (else empty). Presentation only; posting it is the opt-in comment step.