Huawei UDSN - shawfdong/hyades GitHub Wiki
Unified Distributed Storage Nodes (UDSN) are 2U enclosures of SoD nodes of the the Huawei Universal Distributed Storage. Each UDSN holds two (2x) independent UM680 blade servers, which are installed in slot 1 and slot 2 located in the front of the chassis; and each UM680 blade carries a Gigabit Ethernet switch and eight (8x) SoD nodes.
The Baseboard Management Controller (BMC) on UDSN can be accessed via SSH or telnet, e.g.:
# ssh UDSN_0101 # telnet -l root UDSN_0101
The BMCs, STM-spear300, appear to be made by STMicroelectronics. And they run Wind River Linux release 3.0.
The CPU on each BMC appears to be a ARM926EJ-S processor at 166MHz.
There is 128MB memory on each BMC.
There is 64MB of NOR flash on each BMC.
# cat /proc/mtd dev: size erasesize name mtd0: 00020000 00020000 "xloader" mtd1: 00040000 00020000 "u-boot" mtd2: 00020000 00020000 "Parameter" mtd3: 01800000 00020000 "Filesystem1" mtd4: 01800000 00020000 "Filesystem2" mtd5: 00f80000 00020000 "Appcfgdata"
/dev/mtdblock4 is the root partition:
Kernel command line: root=/dev/mtdblock4 rw rootfstype=cramfs console=ttyS0,115200 init=/sbin/init ip=192.168.1.10:192.168.1.1:192.168.1.1:255.255.255.0::eth0 panic=1
and /dev/mtdblock5, a JFFS2 (Journaling Flash File System version 2), is mounted at /data[1].
Besides an IP address in the Storage Plane, every BMC on UDSN has the following same set of IP addresses:
| Interface | IP Address | Netmask | VLAN |
|---|---|---|---|
| eth0.4092 | 192.168.2.17 | 255.255.255.0 | 4092 |
| eth0.4094 | 192.168.0.17 | 255.255.255.0 | 4094 |
| eth1.4092 | 192.168.3.17 | 255.255.255.0 | 4092 |
| eth1.4094 | 192.168.1.17 | 255.255.255.0 | 4094 |
| eth2 | 192.168.6.17 | 255.255.255.0 |
eth0 is connected to the switch of the UM680 blade in slot 1; and eth1 in slot 2. Other than that, I am not sure what purpose those network configurations serve.
There are two (2x) Huawei Quidway S5300 gigabit switches inside each UDSN enclosure, one on each UM680 blade. The IP addresses of the 2 switches are 192.168.0.18 & 192.168.1.18, respectively. The switches can be accessed from the BMC on a UDSN, via telnet:
root@BMC:/#telnet 192.168.0.18and
root@BMC:/#telnet 192.168.1.18
There are 24x GbE ports, and 2x 10GbE ports, on each switch:
<Quidway>display interface brief Interface PHY Protocol InUti OutUti inErrors outErrors Eth-Trunk1 up up 0.20% 0.01% 0 0 XGigabitEthernet0/1/1 up up 0.20% 0.01% 0 0 XGigabitEthernet0/1/2 down down(e) 0% 0% 0 0 GigabitEthernet0/0/1 up up 0% 0.01% 0 0 GigabitEthernet0/0/2 up up 0% 0.01% 0 0 GigabitEthernet0/0/3 up up 0.01% 0.25% 0 0 GigabitEthernet0/0/4 up up 0.01% 0.23% 0 0 GigabitEthernet0/0/5 up up 0.01% 0.26% 0 0 GigabitEthernet0/0/6 up up 0.01% 0.24% 0 0 GigabitEthernet0/0/7 up up 0.01% 0.27% 0 0 GigabitEthernet0/0/8 up up 0.01% 0.26% 0 0 GigabitEthernet0/0/9 up up 0.01% 0.27% 0 0 GigabitEthernet0/0/10 up up 0.01% 0.24% 0 0 GigabitEthernet0/0/11 up up 0% 0.01% 0 0 GigabitEthernet0/0/12 up up 0% 0.01% 0 0 GigabitEthernet0/0/13 up up 0% 0.01% 0 0 GigabitEthernet0/0/14 up up 0% 0.01% 0 0 GigabitEthernet0/0/15 up up 0% 0.01% 0 0 GigabitEthernet0/0/16 up up 0% 0.01% 0 0 GigabitEthernet0/0/17 up up 0.01% 0.01% 0 0 GigabitEthernet0/0/18 down down 0% 0% 0 0 GigabitEthernet0/0/19 down down 0% 0% 0 0 GigabitEthernet0/0/20 down down 0% 0% 0 0 GigabitEthernet0/0/21 down down 0% 0% 0 0 GigabitEthernet0/0/22 down down 0% 0% 0 0 GigabitEthernet0/0/23 down down 0% 0% 0 0 GigabitEthernet0/0/24 down down 0% 0% 0 0 MEth0/0/1 down down 0% 0% 0 0 NULL0 up up(s) 0% 0% 0 0 Vlanif1 up down -- -- 0 0 Vlanif4094 up up -- -- 0 0The 10GbE ports are connected to the Huawei S6700 Switches; and the GbE ports are connected to the SoD nodes. Note that each switch is connected to all 16 SoD nodes in a UDSN enclosure, providing redundant paths to every SoD node.
#netstat -a Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 localhost:3009 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:ftp 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:telnet 0.0.0.0:* LISTEN tcp 0 0 localhost:3001 0.0.0.0:* LISTEN tcp 0 0 localhost:3007 0.0.0.0:* LISTEN udp 0 0 0.0.0.0:664 0.0.0.0:* udp 0 0 0.0.0.0:snmp 0.0.0.0:* udp 0 0 localhost:2001 0.0.0.0:* udp 0 0 localhost:2007 0.0.0.0:* udp 0 0 localhost:2008 0.0.0.0:* udp 0 0 localhost:2009 0.0.0.0:* udp 0 0 0.0.0.0:65501 0.0.0.0:* udp 0 0 0.0.0.0:623 0.0.0.0:*
Running are services for SSH, telnet and FTP, along with some obscure ones at /ipmc:
PID USER VSZ STAT COMMAND 353 root 3020 S /bin/bash /ipmc/heart_to_cpld.sh 355 root 2624 D insmod /ipmc/heart_cpld.ko 359 root 19888 S /ipmc/fsync 365 root 5044 S /ipmc/uusync 367 root 192m S /ipmc/bmcipmi.out 406 root 60532 S /ipmc/cooling 425 root 45196 S /ipmc/ipmgnt 509 root 38040 S /ipmc/snmpd 516 root 1604 S /ipmc/nsupdate
Baseboard Management Controller (BMC) is the main controller of an IPMI sub-system. The BMC connects to satellite controllers or another BMC in another chassis via the Intelligent Platform Management Controller (IPMC) bus or bridge.
#ipmctool Ipmctool v1.04 WARRNING: this tool only for developing purpose! 0 Fpga graceful load. e.g. ipmctool 0 <filename> 1 Check i2c device reading. e.g. ipmctool 1 2 Read fans duty. e.g. ipmctool 2 3 Read fan speed. e.g. ipmctool 3 4 Set fan duty. e.g. ipmctool 4 <fan> <duty> 5 Set fpga reg. e.g. ipmctool 5 <reg> <value> 6 Get fpga reg. e.g. ipmctool 6 <reg> 7 Force load cpld. e.g. ipmctool 7 <filename> 8 Dump power supply info e.g. ipmctool 8 9 Force load fpga. e.g. ipmctool 9 <filename> 10 Short push button. e.g. ipmctool 10 11 Long push button. e.g. ipmctool 11 12 Read 64 bytes bios data. e.g. ipmctool 12 <addr> 13 Erase bios. e.g. ipmctool 13 14 Upgrade bios. e.g. ipmctool 14 <filename> 15 Download bios. e.g. ipmctool 15 <filename> 16 Do nothing. e.g. ipmctool 16 17 Cpld rand read write test. e.g. ipmctool 17 <reg> <delay> 18 Write file to block device. e.g. ipmctool 18 <device> <filename> 19 Do nothing. e.g. ipmctool 19 20 Read port80 and history. e.g. ipmctool 20 21 Get Env Variable. e.g. ipmctool 21 <env> 22 Set Env Variable. e.g. ipmctool 22 <env> <string> 23 Dump Env Variable. e.g. ipmctool 23 24 Read eeprom to file. e.g. ipmctool 24 [filename] 25 Erase & read eeprom to file. e.g. ipmctool 25 26 Read from raid. e.g. ipmctool 26 Uasge:ipmctool <option> <param...>