Multiprocessing Test of (u,v)-advanced

Compared with single process, we make the following experiment to check how good the multiprocessing acceleration can achive.

Multiprocessing Designs

The script, Func uv advanced.py, can provide the all-year round (u, v) plots, all-sky (u, v) plots, and multiple-source (u,v) plots. These are very time-consuming. Since the acceleration of multi-threading in Python is strictly limited by the Global Interpreter Lock (GIL), we adopt the multiprocessing to optimize the computing-intensive task in VNSIM. To reduce the overhead of creating sub-processes, a process pool is pre-created and the number of sub-processes can be user-defined (or adopting the CPU core number by default). Each sub-process can deal with a complete combination of parameter settings, and the inter-communication among sub-processes is performed via the asynchronous queue objects. Eventually, all the running results are gathered and parsed at the main process.

Experiment configuration

We first create the configuration file,

[obs_time] start = 2020/01/01/00/00/00 end = 2020/01/02/00/00/00 step = 00/00/05/00

[bs_type] bs_flag_gg = 1 bs_flag_gs = 0 bs_flag_ss = 0

[obs_mode] obs_freq = 22e9 bandwidth = 3.2e7 cutoff_angle = 10.0 precession_mode = 0 unit_flag = km

[station] pos_source = 0316+413, 0202+319, 0529+483, 1030+415, 1128+385, 1418+546, 1823+568, 1828+487, 1928+738, 1954+513 pos_vlbi = Tianma, Urumqi, KVNTN, KVNUS, KVNYS, VERAIR, VERAIS, VERAMZ, VERAOG pos_telemetry = pos_satellite =

As you can see, we set 9 stations and 10 radio sources. We then run the Func_uv_advanced.py with all-sky-uvplot and 10 sources, i.e., 40 uvplots need to be generated.

We use time python module to track how much time is consumed by calculations (excluding the image-plotting).

By default, the maximum subporcesses will be created (the number of CPU effective cores)

Run experiments

With the above configuration, we run the experiments 10 times for each and average running time is obtained.

If you wanna run the test by yourself, you can modify the script Func_uv_advanced.py

if __name__ == "__main__":
    # run_uv_advanced()  # by default
    test_accelerate_result()

python Func_uv_advanced.py -t src+sky

Then repeat the running 10 times and record the running information.

The information about my computer is

iMac (Retina 5K, 27-inch, 2017)

Processor 4.2 GHz Intel Core i7 (4 cores).

Memory 16 GB 2400 MHz DDR4

Results Analysis

Uvplot results

Single Process and Multiple Processes should generate the same uvplots.

Running Time

The 10 times running information is recorded:

	Single Process Run (second)	Multiprocessing Run (second)
1	11.506165027618408	3.587195873260498
2	11.742285966873169	3.318614959716797
3	11.819091796875	3.3208959102630615
4	11.901732683181763	3.392956256866455
5	11.446985960006714	3.3106188774108887
6	11.289961099624634	3.4709770679473877
7	11.491694211959839	3.4852287769317627
8	11.342378854751587	3.305959939956665
9	11.408003091812134	3.3841350078582764
10	11.650150060653687	3.5008509159088135
Average	11.559844875335694	3.4077433586120605

$$\frac{11.559844875335694}{3.4077433586120605} = 3.3922287152644914 < 4 $$

We can make the conclussion that the The computing speed is roughly proportional to the core number. Since the inter-processing communication is also time-consuming, the accelaration ratio is slightly smaller the number of CPU cores. (In this case, 4 cores.)