sacharified/playwright-healer

Auto-heal flaky Playwright tests via an LLM agent + Playwright MCP

View on GitHub

Trust Signals

Scorecard Score
not yet scored
Maintenance Recency
Stale
License
None
namedescriptionrequireddefault
modeRun mode: ingest | heal | dry-runyes
commit_shaCommit SHA to checkout for heal mode (HEA-01). Defaults to empty (uses HEAD).no""
api_keyInference-provider API key. Distinct from healer_token / github_token (those are PR-creation and runner tokens). For provider=github, pass a GitHub PAT with the `models:read` scope (a different PAT than healer_token, which needs `repo`). For Ollama localhost it may be empty. Example: api_key: secrets.GH_MODELS_TOKEN # for provider: github no
healer_tokenPAT or GitHub App token required for PR creation and workflow_dispatchyes
github_tokenGitHub token; defaults to the built-in action tokenno${{ github.token }}
setup_commandCommand to set up the application under test (healer mode)no""
start_commandCommand to start the application under test (healer mode)no""
test_commandCommand to run Playwright testsno""
base_urlBase URL of the application under testno""
working_directorySubdirectory of the consumer's repo where the application + tests live, relative to GITHUB_WORKSPACE. Set this when your test_command starts with `cd <subdir> && ...` (e.g. monorepos with the app under `frontend/`). The healer reads test files, runs validation reruns, and applies fixes from this directory. setup_command / start_command / test_command keep their existing semantics — they still run from GITHUB_WORKSPACE and can use `cd` themselves. Default: '' (workspace root). no""
bot_emailGit author email used for heal-PR commits. Some deploy gates (Vercel, Netlify, GitLab CI/CD checks) verify that the commit author email maps to a known GitHub account before allowing the deployment to proceed. Default is the canonical github-actions[bot] noreply form, which maps to a real bot account and is recognized by these systems. Override with your own bot account or PAT-owner email if you want different attribution. no41898282+github-actions[bot]@users.noreply.github.com
bot_nameGit author name used for heal-PR commits. Pairs with bot_email above. Default matches the github-actions[bot] convention. nogithub-actions[bot]
providerInference provider: openrouter | github | ollama. openrouter routes to Anthropic, Google, OpenAI, Meta, and others through one OpenAI-compatible endpoint. github uses the GitHub Models free tier (PAT auth). ollama is for self-hosted localhost.noopenrouter
modelModel id. For openrouter use slash-prefixed slugs (e.g. anthropic/claude-sonnet-4.6, google/gemini-2.5-pro, openai/gpt-4.1). Empty string means use provider default.no""
api_endpointCustom provider endpoint. Useful for self-hosted Ollama (http://localhost:11434) or to override the OpenRouter endpoint (default https://openrouter.ai/api/v1) or GitHub Models endpoint (default https://models.github.ai/inference). Empty string uses provider default.no""
report_pathGlob or path to Playwright JSON report (ingest mode)notest-results/results.json
flake_rate_thresholdFlake rate (0.0–1.0) above which a test is a flake candidate. Default: 0.2 (20%)no0.2
flake_window_daysRolling window in days for flake-rate and slow-regression computation. Default: 7no7
slow_regression_pctp95 duration growth ratio that triggers slow-regression detection. Default: 1.5 (50% slower)no1.5
min_runs_for_detectionMinimum number of unique commits a test must run on (within flake_window_days) before threshold evaluation can flag it. Tests below this gate are listed in the step summary as "waiting for sample size" but are never dispatched. Lower values surface flakes faster but raise the false-positive rate; the default reflects Phase 02 research on minimum sample size for stable rolling-window flake-rate estimates. Default: 10 no10
rerun_countNumber of validation re-runs after a fix (Phase 3+). Default: 10no10
rerun_pass_rateRequired pass rate (0.0–1.0) for a fix to be accepted (Phase 3+). Default: 0.9no0.9
max_budget_usdMax USD per healing run (Phase 3+). Default: 2.00no2.00
max_turnsMax agent turns per healing run (Phase 3+). Default: 30no30
retention_daysDays to retain state branch records before GC. Default: 90. Set to 0 to disable GC.no90
max_heals_per_test_per_weekPer-test heal attempt cap per 7-day window (circuit breaker). Default: 3no3
enable_selector_fixesEnable selector-class fixes (Phase 3+). Default: truenotrue
enable_wait_fixesEnable wait/timing-class fixes (Phase 3+). Default: truenotrue
enable_assertion_fixesEnable assertion-class fixes (Phase 4+). Default: true (toggle-only in Phase 3)notrue
enable_slow_fixesEnable slow-test-class fixes (Phase 4+). Default: true (toggle-only in Phase 3)notrue
startup_timeout_secondsMax seconds to wait for start_command app to become ready (heal mode). Default: 120no120
enable_auto_dispatchEnable live workflow_dispatch on threshold breach (Phase 4+). Opt-in safe-default. Default: falsenofalse
healer_workflow_fileFilename of the healer workflow under .github/workflows/ that ingest dispatches (Phase 4+). Override for multi-workflow consumers. Default: playwright-healer.ymlnoplaywright-healer.yml
enable_auto_mergeEnable auto-merge for eligible healer PRs (Phase 5+). Opt-in safe-default. Default: false. See README §auto-merge-prerequisites for branch-protection requirements.nofalse
auto_merge_pass_rateMinimum validation pass rate (0..1) for auto-merge eligibility. Stricter than rerun_pass_rate (0.9 default). Default: 1.0 (10/10).no1.0
auto_merge_fix_classesComma-separated fix classes eligible for auto-merge. Conservative default — only proven classes. Default: selectorsnoselectors
skip_deterministic_checkSkip the deterministic-failure gate (demo mode only). Set true to let the healer attempt a fix on an always-failing test. Default: falsenofalse
skip_post_fix_validationSkip post-fix Playwright validation reruns (demo mode only). fixture-ci.yml on the opened PR is the truth source. Default: falsenofalse
skip_diff_lintSkip diff-lint forbidden-pattern check (demo mode only). Re-enable in Phase 04. Default: falsenofalse
namedescription
dry_run_summaryRendered redacted dry-run markdown summary. Populated only when mode=dry-run. Self-redacted via core.setSecret on api_key/healer_token/github_token before emission — safe to log. Empty for ingest/heal modes. Diagnostic / test-only — content not stable across versions. Do not parse from consumer workflows for production decisions.