Example Scenarios 3 - nps-ros2/ns3_testbed GitHub Wiki
This page examines latency reported by the Testbed and compares it against data captured by Wireshark.
Contents:
- Setup
- Wifi testbed and Wireshark measurements
- Direct measurements without Wifi and without ns-3
- Direct measurements without Wifi but with ns-3
- Direct testbed measurements without Wifi
Comments:
From Wireshark analysis, it looks like packets are broadcast and are not repeatedly transmitted point-to-point as the testbed results suggest. Why does Wireshark show microsecond latency while the testbed shows millisecond latency? What is causing this latency?
Setup
Setup consists of Robot R1 talking to robots R2, R3, R4, R5.
example_keep_last_5.csv:
Publish,,,,,,,
Node,Subscription,Frequency,Size,History,Depth,Reliability,Durability
R1,odometry,2,10,keep_last,5,best_effort,volatile
,,,,,,,
Subscribe,,,,,,,
Node,Subscription,History,Depth,Reliability,Durability,,
R2-5,odometry,keep_last,5,best_effort,volatile,,
The QoS setting of best_effort means the recipient does not return an Ack.
ns3_defaults.csv for Wifi mobility:
mobility,,,
Node,X,Y,Walks
R1,0,0,false
R2,1,0,false
R3,0,1,false
R4,-1,0,false
R5,0,-1,false
The stationary mobility at 1 meter means all antennas are well within range.
Run
These steps run the scenario:
-
Start Wireshark:
sudo wireshark -
Start the Testbed Monitor, specifying to write output to file
1_to_4.csv:cd ~/gits/ns3_testbed/testbed_monitor ./testbed_monitor.py -o 1_to_4.csv -
Start the ns-3 simulator program, providing setup file
ns3_defaults.csv:cd ~/gits/ns3_testbed/ns3_programs/build ./ns3_mobility -s ns3_defaults.csv -c 5 -
Start the robots, providing setup file
example_keep_last_5.csv:sudo /bin/bash cd ~/gits/ns3_testbed/cpp_testbed_runner/build/cpp_testbed_runner ./testbed_runner -c 5 -i example_keep_last_5.csv -n -p -
Allow the simulation to run for a few minutes, then stop the started programs.
Testbed and Wireshark Measurements using ns-3 Wifi
Testbed Measurements
We post-process 1_to_4.csv to obtain graphs:
cd ~/gits/ns3_testbed/testbed_monitor
./plot_analytics.py 1_to_4.csv "1_to_4 1 meter spacing", -m 30 -w 1_to_4
Here are the results:
Latency point plot:
Latency:
Wireshark Measurements
Here are seconds 10 and 11 of captured data. Times are the time the packets come into Wireshark. We do not have start times for calculating latency.
No. Time Source Destination Protocol Length Info
808 10.043423218 10.0.0.1 10.0.0.13 RTPS 130 INFO_TS, DATA
809 10.043443176 10.0.0.1 10.0.0.7 RTPS 130 INFO_TS, DATA
810 10.043452752 10.0.0.1 10.0.0.10 RTPS 130 INFO_TS, DATA
811 10.043461369 10.0.0.1 10.0.0.4 RTPS 130 INFO_TS, DATA
812 10.543419829 10.0.0.1 10.0.0.13 RTPS 130 INFO_TS, DATA
813 10.543439775 10.0.0.1 10.0.0.7 RTPS 130 INFO_TS, DATA
814 10.543449671 10.0.0.1 10.0.0.10 RTPS 130 INFO_TS, DATA
815 10.543458485 10.0.0.1 10.0.0.4 RTPS 130 INFO_TS, DATA
816 11.043413694 10.0.0.1 10.0.0.13 RTPS 130 INFO_TS, DATA
817 11.043437247 10.0.0.1 10.0.0.7 RTPS 130 INFO_TS, DATA
818 11.043446936 10.0.0.1 10.0.0.10 RTPS 130 INFO_TS, DATA
819 11.043455734 10.0.0.1 10.0.0.4 RTPS 130 INFO_TS, DATA
820 11.543419550 10.0.0.1 10.0.0.13 RTPS 130 INFO_TS, DATA
821 11.543440390 10.0.0.1 10.0.0.7 RTPS 130 INFO_TS, DATA
822 11.543450768 10.0.0.1 10.0.0.10 RTPS 130 INFO_TS, DATA
823 11.543459791 10.0.0.1 10.0.0.4 RTPS 130 INFO_TS, DATA
What is significant here is that a set of four messages reach their destination addresses over a period of about 40 microseconds. This suggests that the message is broadcast to all four listeners at once.
Testbed Measurements
Here are some timestamp values taken from the Testbed timestamp log, where:
-
Transmit lines contain: from, to, subscription, tx_count, timestamp
-
Receive lines contain: from, to, subscription, tx_count, rx_count, size, timestamp
R1,R2,odometry,195,1572456550507227634 R1,R3,odometry,195,1572456550507227634 R1,R4,odometry,195,1572456550507227634 R1,R5,odometry,195,1572456550507227634 R1,R3,odometry,195,194,10,1572456550508610099 R1,R4,odometry,195,194,10,1572456550510463885 R1,R5,odometry,195,194,10,1572456550512080628 R1,R2,odometry,195,193,10,1572456550513262992 R1,R2,odometry,196,1572456551007223181 R1,R3,odometry,196,1572456551007223181 R1,R4,odometry,196,1572456551007223181 R1,R5,odometry,196,1572456551007223181 R1,R3,odometry,196,195,10,1572456551008754335 R1,R4,odometry,196,195,10,1572456551010446435 R1,R5,odometry,196,195,10,1572456551012089624 R1,R2,odometry,196,194,10,1572456551013336412
Note that the transmit time is the same, while the receive times progressively lag behind. Transmit times are the same because the ROS2 publisher issues single publications for subscriptions. Receipt times vary because ROS2 delivers data from source to destination individually. For odometry transmit number 195, receive times are 1.38 ms, 2.24 ms, 4.85 ms, and 6.04 ms.
Measurements are similar when the pipe logger is disabled and stdout is used instead, indicating that the pipe logger is not interfering with timing.
Without Wifi and Without ns-3
Here we examine our testbed performance without network namespaces or ns-3. We keep robots within the host network by not setting the network namespaces for the robot processes.
Run
-
Start the Testbed Monitor, specifying to write output to file
1_to_4_direct.csv:cd ~/gits/ns3_testbed/testbed_monitor ./testbed_monitor.py -o 1_to_4_direct.csv -
Start the robots outside of root (we only use root to change network namespaces), providing setup file
example_keep_last_5.csv, not selecting to use network namespaces, and selecting to use pipes:cd ~/gits/ns3_testbed/cpp_testbed_runner/build/cpp_testbed_runner ./testbed_runner -c 5 -i example_keep_last_5.csv -p -
Allow the simulation to run for a few minutes, then stop the started programs.
Analysis
Post-process 1_to_4_direct.csv:
cd ~/gits/ns3_testbed/testbed_monitor
./plot_analytics.py 1_to_4_direct.csv "1_to_4 direct", -m 5 -w 1_to_4_direct
Latency point plot:
Latency:
Without Wifi and with ns-3 in CSMA mode
Here we examine our testbed performance with ns-3 in wired CSMA mode.
Run
-
Start the Testbed Monitor, specifying to write output to file
1_to_4_direct.csv:cd ~/gits/ns3_testbed/testbed_monitor ./testbed_monitor.py -o 1_to_4_csma.csv -
Start the ns-3 CSMA simulator program, providing setup file
ns3_defaults.csv:cd ~/gits/ns3_testbed/ns3_programs/build ./ns3_csma -s ns3_defaults.csv -c 5 -
Start the robots, providing setup file
example_keep_last_5.csv:sudo /bin/bash cd ~/gits/ns3_testbed/cpp_testbed_runner/build/cpp_testbed_runner ./testbed_runner -c 5 -i example_keep_last_5.csv -p -
Allow the simulation to run for a few minutes, then stop the started programs.
Analysis
Post-process 1_to_4_direct.csv:
cd ~/gits/ns3_testbed/testbed_monitor
./plot_analytics.py 1_to_4_csma.csv "1_to_4 csma", -m 3 -w 1_to_4_csma
Latency point plot:
Latency:
Reversed Flow
This section describes latency when flows are reversed: 4 robots communicate to 1 robot using Wifi. These results seem consistent with what is expected.
Transmit Simultaneously
Four robots attempt to transmit at the same time, showing that collisions cause staggered latency:
Latency point plot:
Latency:
Transmit Staggered
Four robots transmit at staggered intervals, showing an independent variation in latency:
Latency point plot:
Latency:
