DTN - shawfdong/hyades GitHub Wiki
dtn.ucsc.edu is UCSC's Data Transfer Node. It is a Dell PowerEdge R510, equipped with two (2x) quad-core Intel Westmere Xeon E5620 processors at 2.40GHz, and 24GB memory.
The RAID controller of dtn is a PERC H700 Integrated[1], as reported by:
# omreport storage controller
but is actually a rebranded LSI MegaRAID SAS 2108:
# lspci | grep RAID 02:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] (rev 05)
Two virtual disks are configured on the RAID controller. One is a RAID-1 volume (/dev/sda), comprised of two 146 GB SAS drives; and the other is a RAID-6 volume (/dev/sdb), comprised of twelve (12x) 1TB SAS drives. The former is used for the OS. On the latter is created an XFS file system, which is mounted at /data.
$ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda6 117G 1.6G 109G 2% / tmpfs 12G 0 12G 0% /dev/shm /dev/sda1 248M 101M 135M 43% /boot /dev/sda5 3.9G 72M 3.6G 2% /tmp /dev/sda2 9.7G 485M 8.7G 6% /var /dev/sdb1 9.1T 11G 9.1T 1% /data
For more information, see dtn Benchmarks.
There are 2 on-board Broadcom GbE interfaces. 10 GbE connections are provided by a dual-port Intel 82599ES 10 Gigabit Ethernet Controller.
$ lspci | grep Ethernet 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20) 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20) 03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 03:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Officially, Intel doesn't sanction other brands of SFP+ optical modules to work with Intel 10 Gigabit Ethernet adapters[2]. When a non-Intel branded SFP+ optical module, the Linux kernel module ixgbe (driver for Intel GbE adapters) will fail to load and it will post a message into dmesg like the following:
ixgbe 0000:03:00.0: failed to load because an unsupported SFP+ module type was detected.
Fortunately, the Linux kernel module ixgbe permits a workaround:
# modinfo ixgbe | grep parm parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599-based adapters (uint)
To manually unload the ixgbe module and then reload the module with the parameter allow_unsupported_sfp, run
# modprobe -r ixgbe # modprobe ixgbe allow_unsupported_sfp=1
If it works, you can make it permanent (to survive reboots), by running:
# echo "options ixgbe allow_unsupported_sfp=1" > /etc/modprobe.d/ixgbe-options.conf # depmod -a
It is only befitting to have Globus Toolkit installed on the Data Transfer Node. The Globus endpoint name for dtn is ucsc#dtn. For more details, see Globus on dtn.