Executing concurrent tests - dmwm/WMCore GitHub Wiki

Concurrent tests consists of sending many requests concurrently and evaluating the throughput of your web service. To be performed in scenarios like (not limited to though):

• update of the web framework
• non-minor changes to the REST model
• change of web framework, etc

For these concurrency tests, we've used a Go tool compiled and made available by Valentin, called hey. In order to see how to use it, check out the help function, e.g.:

/afs/cern.ch/user/v/valya/public/hey_linux --help

Once you have the service you want to test up & running (do NOT run it against a production environment!!!), you need to provide a list of API/URLs that we want to call, e.g.:

amaltaro@vocms0192:/data/amaltaro/HG1906_Validation $ cat url.txt 
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?status=new
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?status=assigned
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?status=staging
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?request_type=ReDigi
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?request_type=StepChain
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?request_type=TaskChain&detail=True
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?outputdataset=/a/b/c
https://alancc7-cloud2.cern.ch/reqmgr2/data/request?requestor=amaltaro

Then we can execute the following command:

/afs/cern.ch/user/v/valya/public/hey_linux -n 500 -c 15 -t 300 -m GET -U cloud2.txt -A "application/json"

where:

• -n: total number of requests to make
• -c: number of requests to run concurrently
• -t: timeout for each request, in seconds
• -m: the HTTP method to use
• -U: text file containing URLs to be requested
• -A: HTTP accept header

The tool gives us a nice report, e.g.:

amaltaro@vocms0192:/data/amaltaro/HG1906_Validation $ /afs/cern.ch/user/v/valya/public/hey_linux -n 1000 -c 50 -t 300 -m GET -U cloud2.txt -A "application/json"
Summary:
  Total:	29.9914 secs
  Slowest:	6.9225 secs
  Fastest:	0.0353 secs
  Average:	1.4460 secs
  Requests/sec:	33.3429
  Total data:	16000 bytes
  Size/request:	16 bytes

Status code distribution:
  [200]	1000 responses

Response time histogram:
  0.035 [1]	|
  0.724 [51]	|∎∎∎
  1.413 [605]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  2.101 [286]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  2.790 [7]	|
  3.479 [0]	|
  4.168 [0]	|
  4.856 [1]	|
  5.545 [15]	|∎
  6.234 [16]	|∎
  6.923 [18]	|∎

Latency distribution:
  10% in 0.8263 secs
  25% in 0.9514 secs
  50% in 1.1995 secs
  75% in 1.5345 secs
  90% in 1.8035 secs
  95% in 4.7520 secs
  99% in 6.3769 secs

For further information on the hey tool, please refer to this document hey_http_group