| folder_with_bench_data | The file serves as the repository for both historical benchmark data
and the location where new benchmark data will be appended.
Currently, only .js files are supported.
| no | bench_data |
| file_with_bench_data | The name of the file that contains the benchmark data.
Currently, only .js files are supported.
| no | data.json |
| folder_with_current_benchmark_results | A path to the folder where benchmark results are stored.
Can be a single file or multiple files.
| yes | — |
| bench_group_name | The name of a specific benchmark group. This name will be used to identify the benchmark ... | no | Benchmark |
| result_files_merge_strategy_for_each_metric | The strategy to use when merging the the benchmark results from the downloaded folder.
This should be a comma separated list of strategies. Each strategy corresponds to a metric.
The possible values are: sum, average, max, min, median.
| no | — |
| metrics_to_evaluate | A comma-separated list of metrics to evaluate for the current benchmark.
This must be a subset of metrics from the file pointed to by current_bench_res_file.
If not specified, all metrics will be evaluated.
| no | — |
| github_token | GitHub token to pull/push to repo, add benchmark info to the repo, etc. | no | — |
| comment_to_commit | Add commit comment with the detailed benchmark info.
Possible values are on, off, if_failed.
| no | if_failed |
| alert_users_if_bench_failed | Commas-separated list of users to alert if the benchmark failed.
| no | — |
| action_page_job_summary | Leave a short summary on the workflow page. Possible values are on, off, if_failed | no | on |
| save_curr_bench_res | Save the benchmark data pointed to by current_bench_res_file
to a file located in benchmark_data_file'
| no | true |
| evaluation_method | It refers to a reference point by which we determine whether the benchmark is successful or not.
The possible values are 'previous', 'previous_successful', 'threshold', 'jump_detection', 'trend_detection_moving_ave',
'trend_detection_deltas'
| yes | — |
| benchmark_group_to_compare | The name of the benchmark to compare to.
This input is only used when the reference is set to 'previous' or 'previous_successful.'
If not specified, the benchmark with the same name as the current benchmark is used.
| no | — |
| threshold_values | Comma-separated list of threshold values for comparison. | no | — |
| comparison_operators | Comma-separated list of operators for threshold comparison. | no | — |
| comparison_margins | Describes by how much percentage a given value can deviate from the desired
value (threshold or prev) to still qualify the benchmark as successful.
If you have selected range as one of the modes in the above set of modes,
and it is not the only function, please provide values of -1 for the remaining functions.
For example, if you previously entered: smaller, smaller, range, and your range is 20%,
this input should look as follows: "-1, -1, 20"
| — | — |
| threshold_upper | Comma-separated list of upper threshold values for range comparison. | no | — |
| threshold_lower | Comma-separated list of lower threshold values for range comparison. | no | — |
| jump_detection_thresholds | The threshold values for jump detection. | no | — |
| trend_thresholds | The threshold value for both deltas trend detection and moving average. | no | — |
| moving_ave_window_size | The window size for moving average trend detection. | no | — |
| trend_det_no_sufficient_data_strategy | The strategy to use when there is not enough data to perform trend detection.
The possible values are use_available and fail.
| no | use_available |
| trend_det_successful_release_branch | The name of the branch that contains the successful benchmark result
in a release branch.
If not specified, main branch will be used.
| no | main |
| failing_condition | Specifies when the benchmark should fail.
The possible values are any, all, and none.
| no | any |
| link_to_templated_gh_page_with_results | The link to the GitHub page with the benchmark results.
The link should be templated with the benchmark name.
For example, if the benchmark name is 'Benchmark', the link should look as follows:
https://dawidniezgodka.github.io/Bench1-Final/
| no | — |
| github_context | The GitHub context. This is used to get the current branch name.
If not specified, the default GitHub context is used.
| no | — |