microsoft/Azure AI Agent Evaluation
Evaluate the performance, quality and safety of AI agents using the Azure AI Evaluation SDK
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| azure-ai-project-endpoint | The endpoint for the Azure AI Project. This is used to connect to Azure AI Foundry to simulate conversations with each agent, and to connect to the Azure AI Evaluation SDK to perform the evaluation. | yes | — |
| deployment-name | The name of the Azure AI model deployment to use for evaluation. | yes | — |
| data-path | Path to the input data file containing the conversation starters. Each conversation starter will be sent to each agent for a pairwise comparison of evaluation results. If using the GitHub Action workflow, you can override this with a custom data file through the workflow_dispatch input parameter. | yes | — |
| agent-ids | Comma-separated list of agent IDs to evaluate. | yes | — |
| baseline-agent-id | Agent ID of the baseline agent to compare against when evaluating multiple agents. If not provided, the first agent is used. | no | — |
| api-version | The API version to use when connecting to model deployment. | no | "" |
Outputs
no outputs