Customize Experiments - vhive-serverless/STeLLAR GitHub Wiki

Command Line Parameters

  • -o outputPathFlag (default "latency-samples"): The directory path where latency samples should be written.
  • -c configPathFlag (default "experiments/tests/aws/data-transfer.json"): Configuration file with experiment details.
  • -g endpointsDirectoryPathFlag (default "endpoints"): Directory containing provider endpoints to be used.
  • -r specificExperimentFlag (default -1): Only run this particular experiment.
  • -l logLevelFlag (default "info"): Select logging level.

JSON Configuration File Details

You can find examples of valid experiment configurations in the folder experiments. Below are a table and a further discussion about the main elements of a configuration.

Experiment settings:

  • Sequential (default false) Boolean specifying whether to run the sub-experiments in parallel or sequentially.
  • Provider (default aws) String representing the provider to be benchmarked (aws, misc. hostname).

Sub-experiment array settings:

  • Title Name of the directory created for the experiment.
  • Bursts Number of bursts (groups of simultaneous requests) which the latency profiler will trigger.
  • BurstSizes Number of requests to be sent in a burst. This is an array, e.g., [1 2 3] will send bursts as such: 1, 2, 3, 1, 2, 3, etc.
  • IATType (default stochastic) Whether the inter-arrival time should be deterministic, a step function or stochastic (Gaussian).
  • PayloadLengthBytes Length of the payload generated by the serverless function(s).
  • IATSeconds Seconds to wait for in-between bursts.
  • Function (default producer-consumer) Instructs vHive-bench on which function image to use when deploying.
  • PackageType Can be Zip (essential for image size experiments) or Image.
  • DesiredServiceTimes Service times for the serverless function(s) to busy spin on.
  • Parallelism (default 1) Integer representing how many endpoints to use from the endpoints file for this sub-experiment.
  • Visualization (default cdf) The type of visualization to create (histogram, cdf, bar, all, none).
  • FunctionMemoryMB (default 128) How much memory should the benchmarked function allocate. Note: does not do anything with vHive
  • DataTransferChainLength (default 1) Chain length to use for this data transfer experiment. If this is 1, this will be a burstiness experiment.
  • StorageTransfer (default false) Should the data transfer experiment use storage (e.g., S3 or minio) for the transmission?

Tool Output

Each object in the SubExperiments array of a JSON configuration file will create its own directory. Along with the title, further information appended at the end includes memory allocated, the IAT used, and the transfer payload (if applicable).

For example, an experiment with the title 2chain will create a directory 2chain-128MB-IAT10s-10KBpayload.