components generation_safety_quality_signal_monitor - Azure/azureml-assets GitHub Wiki
Computes the content generation safety metrics over LLM outputs.
Version: 0.5.21
View in Studio: https://ml.azure.com/registries/azureml/components/generation_safety_quality_signal_monitor/version/0.5.21
| Name | Description | Type | Default | Optional | Enum |
|---|---|---|---|---|---|
| monitor_name | string | ||||
| signal_name | string | ||||
| production_data | mltable | ||||
| metric_names | a comma-separated list of metric names to compute | string | False | ||
| model_deployment_name | model name, e.g., name of the deployment for Azure OpenAI | string | False | ||
| sample_rate | sample rate for the input dataset, should be greater than 0 and at most 1 | number | True | ||
| prompt_column_name | string | prompt | True | ||
| completion_column_name | string | completion | True | ||
| ground_truth_column_name | string | ground_truth | True | ||
| context_column_name | string | context | True | ||
| groundedness_rating_threshold | integer | 4 | True | ||
| similarity_rating_threshold | integer | 4 | True | ||
| relevance_rating_threshold | integer | 4 | True | ||
| fluency_rating_threshold | integer | 4 | True | ||
| coherence_rating_threshold | integer | 4 | True | ||
| thresholds | a comma-separated dictionary of threshold:value for each metric | string | True | ||
| groundedness_passrate_threshold | number | 0.7 | True | ||
| similarity_passrate_threshold | number | 0.7 | True | ||
| relevance_passrate_threshold | number | 0.7 | True | ||
| fluency_passrate_threshold | number | 0.7 | True | ||
| coherence_passrate_threshold | number | 0.7 | True | ||
| workspace_connection_arm_id | string | False | |||
| monitor_current_time | string | ||||
| notification_emails | string | True | |||
| instance_type | string | standard_e4s_v3 | True |
| Name | Description | Type |
|---|---|---|
| signal_output | uri_folder | |
| signal_scored_data | mltable |