UltraGrid Network Check - CESNET/UltraGrid GitHub Wiki
At this page you can find several tips how to check your connectivity.
The Myrinet card uses 8 lanes in PCIe. To check, whether the port it is connected to provides all 8, enter the command:
lspci -vv | fgrep -A 50 Myri | egrep '(LnkCap|LnkSta)'
The output should look like this:
LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited
LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive + BWMgmt- ABWMgmt-
Check the MSI interrupts:
lspci -vv | fgrep -A 50 Myri | egrep '(MSI)'
Look for the output
Subsystem: MYRICOM Inc. 10G-PCIE-8A (MSI-X firmware)
Capabilities: [44] MSI: Enable+ Count=1/1 Maskable- 64bit+
Throughput of your DMA must be at least 1.3GBps (note that the output is in Bytes, not bits):
ethtool -S eth2 | fgrep dma
read_dma_bw_MBs: 1757
write_dma_bw_MBs: 1736
read_write_dma_bw_MBs: 3436
It is a good idea to have the write-combining set, although for fast processors it is not a critical point
How to do so you can find at this page.
Check whether the path between sender and receiver supports jumbo frames.
ping -M do -s 8500 <target-ip>
- you may try to fiddle with flow control (myri10ge_flow_control during load of module via
modprobe), althrough usually the correct setting is the default one (i.e. flow control is on) - check whether the buffers are set correctly with
sysctl. The directions to set them can be found at the appropriate setup page - To measure the network, use
Iperf 2.0.2with patches by Andrew Gallatin. For more informations about the iperf glitches, follow this link.