notadestroyer/vLLM Benchmark
Run vLLM performance benchmarks and post results as PR comments
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| vllm-url | vLLM server URL | no | http://localhost:8000 |
| preset | Benchmark preset: quick, standard, or thorough | no | quick |
| context-lengths | Comma-separated context lengths (overrides preset) | no | — |
| concurrency | Comma-separated concurrency levels (overrides preset) | no | — |
| output-tokens | Max output tokens per request | no | 500 |
| prompt-type | Prompt type: classic, deterministic, madlib, random, all | no | classic |
| compare-with | Path to previous benchmark JSON for regression detection | no | — |
| fail-on-regression | Fail the workflow if major regressions detected | no | false |
| python-version | Python version to use | no | 3.12 |
Outputs
| name | description |
|---|---|
| score | Composite vLLM benchmark score (0-10000) |
| grade | Benchmark grade (S/A/B/C/D/F) |
| peak-throughput | Peak throughput in tokens/sec |
| results-json | Path to results JSON file |