Component for benchmarking an embedding model via MTEB.

Version: 0.0.2

View in Studio: https://ml.azure.com/registries/azureml/components/benchmark_embedding_model/version/0.0.2

Name	Description	Type	Default	Optional	Enum
endpoint_url	For AOAI, the base endpoint url. For OAI, this will be ignored. For OSS_MaaS, the target url.	string		True
deployment_type	The deployment type.	string		False	['AOAI', 'OAI', 'OSS_MaaS']
deployment_name	For AOAI, the deployment name. For OAI, the model name. For OSS_MaaS, this wil be ignored.	string		True
connections_name	Used for authenticating endpoint.	string		False
tasks	Comma separated string denoting the tasks to benchmark the model on.	string		True
task_types	Comma separated string denoting the task type to benchmark the model on. Choose from the following task types: BitextMining, Classification, Clustering, PairClassification, Reranking, Retrieval, STS, Summarization.	string		True
task_langs	Comma separated string denoting the task languages to benchmark the model on.	string		True
preset	Choose from one of the presets for benchmarking.	string	None	False	['None', 'mteb_main_en']

Name	Description	Type
metrics	Directory where the benchmark metrics will be saved.	uri_folder

azureml://registries/azureml/environments/model-evaluation/labels/latest