components model_prediction_with_container - Azure/azureml-assets GitHub Wiki
Optimized Distributed inference component for LLMs.
Version: 0.0.4
type : evaluation sub_type : subgraph Preview
View in Studio: https://ml.azure.com/registries/azureml/components/model_prediction_with_container/version/0.0.4
model prediction
| Name | Description | Type | Default | Optional | Enum |
|---|---|---|---|---|---|
| task | Task type | string | text-classification | ['text-classification', 'text-classification-multilabel', 'text-named-entity-recognition', 'text-summarization', 'question-answering', 'text-translation', 'text-generation', 'fill-mask', 'chat-completion'] | |
| test_data | Test Data | uri_folder | False | ||
| mlflow_model | Mlflow model | mlflow_model | False | ||
| label_column_name | Label column name in provided test dataset (Ex: label) | string | |||
| input_column_names | Input column names in provided test dataset (Ex : column1). Add comma delimited values in case of multiple input columns (Ex : column1,column2) | string | True | ||
| mini_batch_size | integer | 100 | True | ||
| evaluation_config_params | JSON Serialized string of evaluation_config | string | True |
| Name | Description | Type |
|---|---|---|
| predictions | uri_file | |
| ground_truth | uri_file | |
| prediction_probabilities | uri_file |
TODO: Not implemented yet in this component
| Name | Description | Type |
|---|---|---|
| performance_metadata | Json File containing metadata for perf output | uri_file |
azureml://registries/azureml/environments/foundation-model-inference/labels/latest