nscc-itc-assessment/GrillMyCode

Analyses code changes submitted via GitHub and uses AI to generate targeted comprehension questions for spoken or written assessments. Created to promote code comprehension and critical thinking in NSCC's ITC courses, but adaptable to various educational contexts.

View on GitHub

Trust Signals

Scorecard Score
not yet scored
Maintenance Recency
Stale
License
None
namedescriptionrequireddefault
github_tokenGitHub token used for API access and as the credential for GitHub Models. Defaults to the built-in GITHUB_TOKEN. yes${{ github.token }}
ai_providerAI provider to use for question generation. Supported values: github-models | openrouter nogithub-models
ai_modelModel identifier for the chosen provider. GitHub Models : gpt-4.1 OpenRouter : provider/model-name format (e.g. deepseek/deepseek-v4-flash) nogpt-4.1
ai_retry_max_attemptsTotal number of attempts (initial attempt plus retries) when calling the AI provider. Retries are triggered by transient errors: 429 (rate limit), 500, 502, 503, 504, and network-level failures. A 429 response with a Retry-After header is honoured; all other retryable errors use exponential backoff with full jitter. Values below 1 are clamped to 1. no5
ai_temperatureControls the randomness of the AI's output (0.0 = fully deterministic, 1.0 = most random). Lower values produce more predictable, consistent questions; higher values produce more varied output. Most users should leave this at the default. no0.5
api_keyAPI key for the chosen AI provider. For github-models, leave empty to use github_token automatically, or supply a Personal Access Token (e.g. an instructor token with an alternative license) to use that instead. no""
num_questionsNumber of comprehension questions to generate (minimum 1, maximum 50). Supplied values above 50 are automatically capped to 50.no20
include_answersWhen true, each question is immediately followed by its answer, labelled "Answer:", in the student-facing report. This means the student sees the answers — defeating the purpose of the assessment, which is for the student to research and determine the answers themselves. Leave this false in almost all cases. The instructor repository (when instructor_repo_token is configured) always includes answers regardless of this setting. nofalse
exclude_pattern_overridesComma-separated entries that allow specific files through the default exclude list. Each entry can be either an exact pattern from the default list (e.g. **/*.md — re-includes all Markdown files) or a specific file path that would be excluded by a default pattern (e.g. README.md — only that file is re-included while **/*.md still excludes everything else). no""
additional_exclude_patternsComma-separated glob patterns for extra files to exclude from the assessment, on top of the patterns auto-detected from the repository's language and framework stack. Use this to exclude files specific to your assignment (e.g. provided starter files, test fixtures, data files) that would not normally be excluded by the auto-detected patterns. no""
instructor_repo_tokenPersonal Access Token (PAT) with repo scope and permission to create repositories in the same organisation as the student repositories. When provided, the action writes a private instructor-only assessment file (questions and answers) to a repository named {assignment-name}-grillmycode-instructor in the same organisation. The repository is created automatically on first run if it does not already exist. Leave empty to disable instructor repository delivery. no""
assignment_contextComma-separated file glob(s) whose contents are read from the repository and injected into the AI prompt as assignment context. Useful for automatically including README files, assignment briefs, or style guides. Supported file types: plain text and source files (any UTF-8 text), PDF (.pdf — text layer only, images ignored), and Microsoft Word (.doc/.docx — text content only, images ignored). Globs are matched against the full relative path from the repository root, so subdirectory paths and wildcards work (e.g. "docs/assignment.md", "**/*.md"). If no files match, a workflow warning is emitted and the action continues without assignment context. SECURITY: point this only at instructor-controlled paths. These globs are matched against the student's checked-out working tree, so a glob like "README.md" or "**/*.md" can pick up files the student can edit. Assignment context is treated as untrusted reference data that only steers question topics — it cannot override the rubric or reveal answers — but to keep students from influencing question focus, prefer paths they do not control (e.g. an instructor-managed docs/ directory or a file committed only by the instructor). Use instructor_context for instructions that must take effect. Example: "docs/assignment.md, instructor/rubric.md" no""
assignment_context_max_charsMaximum total characters read from all assignment_context files combined. Prevents extremely large context files from flooding the AI prompt. Values below 1 are clamped to 1. no20000
instructor_contextFree-text context about the assignment or topic being assessed. Helps the AI generate more relevant, targeted questions. Example: "Assignment 3 — Python list comprehensions and lambda functions" no""
include_initial_commitWhen false (the default), the diff base is pinned to the repository's very first commit so that starter/template files are excluded from assessment. When true, the base is set to the empty tree regardless of the event type, so the initial commit's eligible files are included in the assessed diff. nofalse
keep_commentsWhen false (the default), inline and block comments are stripped from the submitted code before it is sent to the AI. Set to "true" to preserve comments exactly as written. nofalse
skip_committersComma-separated list of bot account logins (e.g. github-actions[bot]). Any consecutive commits at the start of the assessed range (immediately after the resolved base SHA) that belong to one of these accounts are excluded from the diff. Matching is case-insensitive. This handles the automated commits that GitHub Classroom applies to a student repository when the assignment is accepted (e.g. feedback PR setup, autograder configuration) so they are never included in the assessed diff. Only consecutive leading commits are skipped — if a bot commit appears after a student commit it is included as normal. SECURITY: a leading commit is skipped only when its GitHub-verified account login matches one of these values. The local commit author name/email is attacker-controlled, so it is used only as a pre-filter, never as the sole basis for skipping — a student cannot hide their own commits by setting their git author name to a bot's. Consequently these values must be GitHub account logins, not arbitrary display names or emails. Set to an empty string ('') to disable bot-commit skipping entirely. nogithub-classroom[bot],github-actions[bot]
base_shaOverride the base commit SHA for the diff. When provided this takes precedence over include_initial_commit. Auto-detected when not provided.no""
head_shaOverride the head commit SHA for the diff. Only applied when base_sha is also provided — both must be set together for the manual override to take effect. Auto-detected when not provided.no""
namedescription
questionsThe raw generated questions as a string.
code_before_stripThe full code content of all assessed files before comment stripping.
code_after_stripThe full code content of all assessed files after comment stripping.
issue_urlURL of the created or updated assessment issue.
issue_numberNumber of the created or updated assessment issue.
pdf_urlDownload URL of the assessment PDF release asset. Empty if PDF generation failed.