batch_benchmark_config_generator

Generates the config for the batch score component.

batch_benchmark_inference

Components for batch endpoint inference

batch_benchmark_inference_claude

Components for batch endpoint inference

batch_benchmark_inference_with_inference_compute

Components for batch endpoint inference with inference compute support.

batch_benchmark_score

batch_inference_preparer

Prepare the jsonl file and endpoint for batch inference component.

batch_output_formatter

Output Formatter for batch inference output

batch_resource_manager

Resource Manager for batch inference.

benchmark_embedding_model

Component for benchmarking an embedding model via MTEB.

benchmark_result_aggregator

Aggregate quality metrics, performance metrics and all of the metadata from the pipeline. Also add them to the root run.

compute_performance_metrics

Performs performance metric post processing using data from a model inference run.

dataset_downloader

Downloads the dataset onto blob store.

dataset_preprocessor

Dataset Preprocessor

dataset_sampler

Samples a dataset containing JSONL file(s).

inference_postprocessor

Inference Postprocessor

prompt_crafter

This component is used to create prompts from a given dataset. From a given jinja prompt template, it will generate prompts. It can also create few-shot prompts given a few-shot dataset and the number of shots.