skyramp/Skyramp Testbot

Github Action for using Skyramp's Testbot for AI-Powered Deterministic Testing

View on GitHub

Trust Signals

Scorecard Score
not yet scored
Maintenance Recency
Stale
License
None
namedescriptionrequireddefault
githubTokenGitHub token for PR comments and API access. Defaults to the automatic GITHUB_TOKEN.no${{ github.token }}
githubAppIdGitHub App ID used to mint fresh installation tokens during the run. GitHub App installation tokens expire after 1 hour, so on longer runs a workflow-minted githubToken is already dead by the time tests are delivered — with githubAppId + githubAppPrivateKey set, the action mints a fresh token right before git pushes and API calls instead. Pair with the same App the workflow uses for its token step. The App installation must cover this repository (and the test repository, if testRepoPath is used — same owner only).no
githubAppPrivateKeyPrivate key (PEM, PKCS#1 or PKCS#8) of the GitHub App named by githubAppId (store in GitHub Secrets). Required for on-demand token minting on runs exceeding the 1-hour installation-token lifetime.no
skyrampLicenseFileSkyramp license file content (store in GitHub Secrets)yes
cursorApiKeyCursor API key for AI agent access (store in GitHub Secrets)no
copilotApiKeyCopilot API key for AI agent access. Requires a GitHub PAT with "Copilot Requests" permission.no
anthropicApiKeyAnthropic API key for Claude Code agent access (store in GitHub Secrets)no
allowedAuthorsNewline-separated list of GitHub usernames whose PRs Testbot will act on. When empty (default), all authors are allowed.no""
targetSetupCommandCommand to start services before test maintenance. Can optionally emit a JSON object as its last stdout line to override workspace baseUrl at runtime (e.g., {"baseUrl": "http://remote-host:8000"} for single-service, or {"services": {"backend": {"baseUrl": "..."}, "frontend": {"baseUrl": "..."}}} for multi-service). Sets SKYRAMP_TEST_BASE_URL (single) or SKYRAMP_TEST_SERVICE_URL_<NAME> (multi) env vars for test execution.nodocker compose up -d
authTokenCommandShell command to generate an authentication token. Runs after services start; stdout is captured and set as SKYRAMP_TEST_TOKEN for test execution.no""
uiCredentialsBrowser login credentials for apps that require authentication before UI test recording. Format: "username:password" (one credential per line for multiple users). Store in GitHub Secrets.no""
targetTeardownCommandCommand to tear down services after test execution (runs in post step, guaranteed even on failure/cancellation)no""
skipTargetTeardownSkip running service teardown commandnofalse
skyrampExecutorVersionSkyramp Executor Docker image version. Falls back to workspace.yml metadata, then "v1.3.34".no
skyrampMcpVersionSkyramp MCP package version. Falls back to workspace.yml metadata, then "latest".no
skyrampMcpSourceSource for Skyramp MCP package: "npm" or "github"nonpm
skyrampMcpGithubTokenGitHub PAT with read access to letsramp/mcp (required when source is "github")no
skyrampMcpGithubRefGit ref (branch/tag/commit) for letsramp/mcp (used when source is "github")nomain
nodeVersionNode.js version to usenolts/*
skipTargetSetupSkip running target setup commandnofalse
targetReadyCheckCommandShell command to verify services are ready (retried until success or timeout). When empty (default), auto-generates curl health checks against service base URLs from workspace config.no""
targetReadyCheckTimeoutMaximum seconds to wait for ready check command to succeed. Default accommodates `docker compose --build --wait` cold builds; the loop exits on first success so fast services pay no extra cost.no1800
targetReadyCheckIntervalSeconds between ready check poll attemptsno30
targetReadyCheckDiagnosticsCommandShell command to collect diagnostics on ready check timeout. Runs when targetReadyCheckCommand times out.nodocker ps -a --format 'table {{.Names}} {{.Status}} {{.Ports}}' && docker ps -a --format '{{.Names}}' | while read name; do [ -n "$name" ] && echo "--- logs $name (last 30) ---" && docker logs --tail 30 "$name"; done
workingDirectoryWorking directory for the actionno.
autoCommitAutomatically commit test changesnotrue
commitMessageCommit message for test changesnoSkyramp Testbot: test maintenance suggestions
generatedTestsModeWhere generated/maintained tests are delivered. "separate-branch" (default) opens a new PR/MR from a bot-owned branch into the feature branch, keeping the feature PR clean. "same-branch" pushes a commit directly onto the feature branch (legacy behavior). Note: for downstream CI (lint/tests/etc.) to auto-run on the side PR, the `githubToken` input must be a GitHub App installation token or PAT — the default `GITHUB_TOKEN` will not trigger other workflows.noseparate-branch
testRepoPathCross-repo test delivery: local filesystem path (relative to `$GITHUB_WORKSPACE`) where the workflow has checked out a separate test repository (via `actions/checkout` with `path:`, on its default branch, with a token that has `contents: write` + `pull-requests: write` on it). `persist-credentials` is not required — Testbot injects credentials from `githubToken` ephemerally for its git operations (push, and side-PR fetch/ls-remote), never written to `.git/config`. When set, generated tests are committed and a side PR is opened in that repo instead of the app repo — Testbot derives its `owner/repo` from the checkout and reuses the action's `githubToken` for the push and side-PR API. Leave empty to commit tests in the app repo.no
relatedRepoPathsMulti-repo analysis: newline-separated local filesystem paths (relative to `$GITHUB_WORKSPACE`) of related repositories the workflow has already checked out as sibling directories (via `actions/checkout` with `path:`). Each is analyzed read-only as shared context alongside the primary repo, with cross-repo changes correlated before tests are generated. The primary checkout repo still receives the report and test commits. Maximum 5 entries.no""
postPrCommentPost summary as PR commentnotrue
maxRecommendationsTotal number of test recommendations to produce (generated + additional). Defaults to 20.no20
maxGenerateNumber of tests to generate and execute in this run. The remainder are listed as additional recommendations. Defaults to 3.no3
testbotMaxRetriesMaximum number of retries for transient agent CLI errors (e.g., Cursor "Connection stalled")no3
testbotRetryDelayDelay in seconds between agent retry attemptsno10
testExecutionTimeoutTimeout in seconds for individual MCP tool calls (e.g., test execution). Increase if tests take longer than 5 minutes.no300
testbotTimeoutTimeout in minutes for the agent execution (safety net; does not kill the child process)no60
targetSetupRetriesNumber of retries for targetSetupCommand on failure (e.g. transient DockerHub 502 errors)no3
targetSetupRetryDelayDelay in seconds between targetSetupCommand retriesno10
authTokenRetriesTotal attempts for authTokenCommand (e.g. 6 = 1 initial + 5 retries). Tolerates SUTs whose auth surface comes up after the HTTP port (async post-start data seeding). Set to 1 to disable retries.no6
authTokenRetryDelayDelay in seconds between authTokenCommand retriesno10
reportCollapsedWrap report sections in collapsible <details> blocks with a summary linenotrue
enableDebugEnable debug loggingnotrue
sutSetupModeSUT setup mode. When not "none", Testbot enters bootstrap mode to generate SUT files. After validation succeeds or bootstrap times out, sets to "none". Supported: none | adapt_workflow | docker_compose.nonone
sutSourceWorkflowFilePath to an existing workflow file to adapt (for adapt_workflow mode). Example: .github/workflows/e2e-tests.ymlno""
sutSourceDockerComposeFilePath to an existing docker-compose file to reuse (for docker_compose mode). Example: docker-compose.ymlno""
namedescription
test_summaryFull summary of test maintenance actions
tests_modifiedNumber of tests modified
tests_createdNumber of tests created
tests_executedNumber of tests executed
skipped_self_triggerWhether execution was skipped due to detecting own commit
commit_shaSHA of the commit made by testbot (empty if no commit)
side_pr_urlURL of the side PR opened by testbot (empty unless generatedTestsMode=separate-pr and changes were produced)
side_pr_numberNumber of the side PR opened by testbot (empty unless generatedTestsMode=separate-pr and changes were produced)
duration_setupDuration of the setup step in seconds
duration_analyzingDuration of the analyzing step in seconds
duration_generatingDuration of the generating step in seconds
duration_executingDuration of the executing step in seconds
duration_maintainingDuration of the maintaining step in seconds
duration_reportingDuration of the reporting step in seconds
duration_totalTotal duration of all steps in seconds