clearml/ClearML detect model degradation
Search ClearML for a task corresponding to the current PR and compare its performance to the previous best task.
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| CLEARML_API_ACCESS_KEY | ClearML api access key (can be generated in your profile page) | yes | — |
| CLEARML_API_SECRET_KEY | ClearML api secret key (can be generated in your profile page) | yes | — |
| CLEARML_API_HOST | ClearML server api host | yes | — |
| CLEARML_PROJECT | Which project to search in for the task. | yes | — |
| CLEARML_TASK_NAME | Which task name to search for. | yes | — |
| CLEARML_SCALAR_TITLE | Which scalar to use for comparison. Title of the scalar. | yes | — |
| CLEARML_SCALAR_SERIES | Which scalar to use for comparison. Series to use within given plot. | yes | — |
| CLEARML_SCALAR_MIN_MAX | Whether smaller is better (MIN) or larger is better (MAX). (default: "MAX") | — | MAX |
| CLEARML_BEST_TAGNAME | The name of tag to be given to the best task. Every task that is checked and is equal or better than the previous best will get this tag. (default: "Best Performance") | — | Best Performance |
| CLEARML_SCALAR_THRESHOLD | The threshold for the difference between the previous best and the current commit. This should be a percentage of the previous best. E.g. CLEARML_SCALAR_THRESHOLD=3 -> if previous best PR is metric value 100 and CLEARML_SCALAR_MIN_MAX is max, then current PR can have a minimum metric value of 97 while still passing. (default: 0) | — | 0 |
Outputs
no outputs