nikolay-e/AutoQA
Automated post-deploy QA: Playwright crawler + axe accessibility + Schemathesis API fuzzing + OWASP ZAP security scan + Mozilla Observatory + visual regression + AuthZ matrix + baseline diff
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Activelast commit Jul 9, 2026
- License
- MIT
Pinned Snippet
uses: nikolay-e/autoqa@9c2f6c04bfd7c6d4313f940b4d3b7adb8c0bb14c # v1.0.1tags can be moved; commit SHAs can't. why a SHA?
Inputs
| name | description | required | default |
|---|---|---|---|
| url | Base production URL | yes | — |
| auth-url | API auth endpoint path for Bearer token (e.g., /api/Users/AuthenticateByName) | — | "" |
| auth-body | JSON body for auth request | — | "" |
| auth-mode | How to acquire the auth token. 'curl' (default): POST to auth-url with auth-body. 'playwright': render auth-login-page in a real browser (set to bypass Cloudflare/Akamai bot challenges that filter curl). | — | curl |
| auth-login-page | Login page path used when auth-mode=playwright (e.g., /login). Falls back to crawler-login-url if empty. | — | "" |
| http-basic-username | Username for proxy-level HTTP Basic Auth (e.g., Traefik basicAuth middleware in front of the whole site). Applies to crawler + monkey (Playwright httpCredentials) and Schemathesis (Authorization: Basic; a Bearer token from auth-url wins when both are set — one Authorization header). Not supported for Observatory (scans from Mozilla's servers) or ZAP. Empty = off. Ref: issue #26. | — | "" |
| http-basic-password | Password for proxy-level HTTP Basic Auth (set as a secret in the downstream workflow). | — | "" |
| crawler-enabled | Run Playwright crawler | — | true |
| crawler-username | Form login username (empty = skip form login) | — | "" |
| crawler-password | Form login password | — | "" |
| crawler-login-url | Login page path | — | /login |
| crawler-login-selector-username | CSS selector for username input | — | input[type="text"], input[type="email"] |
| crawler-login-selector-password | CSS selector for password input | — | input[type="password"] |
| crawler-login-selector-submit | CSS selector for submit button | — | button[type="submit"], button |
| crawler-seed-pages | Comma-separated seed paths to start crawling from | — | / |
| crawler-max-pages | Maximum pages to crawl | — | 50 |
| crawler-wait-ms | Wait time between pages in milliseconds | — | 2000 |
| crawler-exclude-urls | Comma-separated URL patterns to ignore in error reporting | — | "" |
| crawler-console-ignore | Comma-separated substrings matched against console message text or source script URL; matching messages are dropped from error reporting (third-party noise such as CDN bot-challenge probes) | — | "" |
| crawler-decorative-paths | Comma-separated URL substrings (e.g. /Images/) for optional decorative resources. 4xx network errors / broken links whose URL matches are downgraded by the final gate from a blocking finding to non-blocking info — for assets that are legitimately absent (404 is the correct API answer and the UI renders a placeholder) and whose per-item ids keep them from settling into the baseline. Empty = off. | — | "" |
| crawler-fail-on-violations | Fail the composite action when the crawler reports violations (broken links, JS errors, critical axe, CSP, mixed content, network errors). Honored by the final gate step. Ignored when baseline-enabled=true (baseline diff governs instead). | — | true |
| baseline-enabled | Diff crawler findings against cached baseline; fail PRs only on NEW findings (push to main updates baseline) | — | true |
| baseline-fail-on-new | Make NEW crawler findings blocking even on push to main/master. The baseline still updates in the same run, so a regression fails exactly one run and is absorbed for the next (alarm-once). For consumers whose only autoqa runs are post-deploy on main — without this the baseline gate can never block there. First run with no cached baseline never blocks. | — | false |
| mechanical-fail-on-violations | Fail the composite action when Phase 1.5 mechanical checks (M1–M6) report any P0 finding. P1 findings are always informational. Default off because mechanical checks are advisory. | — | false |
| schemathesis-enabled | Run Schemathesis API fuzzing | — | false |
| schemathesis-fail-on-violations | Fail the composite action when Schemathesis reports failed/errored cases. | — | true |
| openapi-url | OpenAPI spec path (e.g., /api/openapi.json) | — | "" |
| schemathesis-base-url | Override Schemathesis --url (defaults to `url`). Use when the API mounts under a path prefix that differs from the crawler entry, e.g., the PWA serves `/` and the backend is at `/api/*` — set this to `https://host/api` so Schemathesis hits the backend, not the SPA shell. | — | "" |
| schemathesis-exclude-paths | Comma-separated API paths to exclude from fuzzing | — | "" |
| schemathesis-exclude-checks | Comma-separated Schemathesis checks to exclude | — | ignored_auth,unsupported_method |
| zap-enabled | Run OWASP ZAP security scan | — | false |
| zap-fail-on-violations | Fail the composite action when ZAP reports HIGH-risk alerts. | — | true |
| zap-rate-limited-paths | Comma-separated URL substrings for endpoints fronted by a rate limiter / auth gate (e.g. /api/auth/login). A HIGH ZAP alert (typically boolean-based SQLi) on such a path is a false positive — the differential ZAP measured is the limiter's 429/403, not a DB-level injection; the attack never reached the database. The traditional -J report carries no per-instance HTTP status, so the gate cannot see the 429 directly; instead a HIGH alert is downgraded by the final gate from blocking to non-blocking info only when EVERY one of its instances targets a declared path (an alert that also fires on an un-gated path still blocks). Empty = off. Ref: issue #7. | — | "" |
| observatory-enabled | Run MDN HTTP Observatory security-headers scan | — | false |
| observatory-fail-grade | Fail if Observatory grade is worse than this (A+ A A- B+ B B- C+ C C- D+ D D- F) | — | B |
| observatory-fail-on-violations | Fail the composite action when Observatory grade is worse than observatory-fail-grade. | — | true |
| argos-enabled | Capture screenshots at 1440x900 and 375x667 viewports and upload to Argos for visual regression | — | false |
| argos-token | Argos CI repository token (set as a secret in the downstream workflow) | — | "" |
| authz-enabled | Run AuthZ two-token matrix (BOLA / OWASP API1:2023) against resource paths owned by user A | — | false |
| authz-user-a-body | JSON body for user A login (sent to auth-url) | — | "" |
| authz-user-b-body | JSON body for user B login (sent to auth-url) | — | "" |
| authz-resource-paths | Comma-separated resource paths owned by user A (e.g., /api/notes/1,/api/notes/2). For each: user A must get 2xx, user B must get 401/403/404, no-auth must get 401/403/404. | — | "" |
| authz-fail-on-violations | Fail the composite action when the AuthZ matrix reports BOLA / auth-bypass. | — | true |
| monkey-enabled | Run the monkey / chaos UI tester — seeded random Playwright interactions (clicks, fuzz input, keys, navigation) for a time budget, hunting crashes, uncaught JS errors and 5xx responses. Opt-in. Reuses crawler-username/password/login selectors to log in first. | — | false |
| monkey-duration-ms | Monkey time budget in milliseconds (default 300000 = 5 minutes). A hard-kill backstop fires 30s after this. | — | 300000 |
| monkey-seed | Seed for the monkey's PRNG. Same seed + same app replays the same action sequence; the seed is logged in the report so a crash can be reproduced. | — | 1337 |
| monkey-seed-pages | Comma-separated paths the monkey starts from and resets to when it gets stuck (default '/'). | — | / |
| monkey-avoid-text | Comma-separated lowercased text / aria-label substrings the monkey will NOT click, so it does not log itself out or delete data and end the session early. Set empty to allow destructive controls. | — | logout,log out,sign out,signout,delete account,deactivate,remove account |
| monkey-exclude-urls | Comma-separated URL substrings to ignore in monkey error reporting (third-party noise). | — | "" |
| monkey-console-ignore | Comma-separated substrings matched against monkey console-error text or source script URL; matching messages are dropped from error reporting. Falls back to crawler-console-ignore when empty. | — | "" |
| monkey-fail-on-violations | Fail the composite action when the monkey finds a serious issue (crash / uncaught JS error / 5xx). Default off — the monkey is advisory because exploration is inherently variable; console errors and failed requests are always non-blocking info. | — | false |
| extra-findings-path | Path to a JSON array of custom StandardFindings (fingerprint/severity/category/tool/title/url/…) from your own checks. Merged into the normalized findings.json and the COMPLETE report without forking autoqa. Invalid entries are dropped (warned), never fatal. | — | "" |
Outputs
| name | description |
|---|---|
| report-path | Directory containing all reports |
| complete-report-md | Path to the always-on human-readable COMPLETE QA report |
| findings-json | Path to the normalized StandardFinding[] (machine-readable) |
| pages-visited | Number of pages crawled |
| js-errors | Number of JS errors found |
| axe-violations | Number of accessibility violations found |
| broken-links | Number of broken links found |
| csp-violations | Number of CSP violations captured |
| mixed-content | Number of mixed-content requests captured |
| monkey-actions | Number of monkey actions performed |
| monkey-serious | Number of serious monkey findings (crashes / uncaught errors / 5xx) |