DroboFS dmesg - droboports/droboports.github.io GitHub Wiki
Analysis of dmesg output
This is what I could gather from the output of dmesg so far:
Linux version 2.6.22.18 (root@jwinfield01) (gcc version 4.2.0 20070413 (prerelease)) #1 Wed May 12 18:34:06 PDT 2010
CPU: ARM926EJ-S [41159260] revision 0 (ARMv5TE), cr=00053977
Machine: Feroceon-MV78XX0
Using UBoot passing parameters structure
Memory policy: ECC disabled, Data cache writeback
On node 0 totalpages: 131072
DMA zone: 1024 pages used for memmap
DMA zone: 0 pages reserved
free_area_init_core: realsize: 48128, size: 131072, memmap_pages: 1024
DMA zone: 48128 pages, LIFO batch:15
Normal zone: 0 pages used for memmap
free_area_init_core: realsize: 0, size: 0, memmap_pages: 0
CPU0: D VIVT write-back cache
CPU0: I cache: 32768 bytes, associativity 4, 32 byte lines, 256 sets
CPU0: D cache: 32768 bytes, associativity 4, 32 byte lines, 256 sets
Built 1 zonelists. Total pages: 48128
This is just standard uBoot stuff. Nothing unusual.
Kernel command line: console=ttyS0,115200 mtdparts=cfi_flash_0:3m@30464k(kernel),21m@33536k(root_fs),2m@59m(var) root=/dev/mtdblock1 rw cpu0=eth0,eth2,eth3,pcie0,pcie1,sata,nand,spi,usb0,usb1,usb2,tdm cpu1=eth1,nor,xor ip=169.254.1.0:169.254.123.234:::DB78xx0:eth0:none
Here is where things start becoming interesting. First, we see the split of the flash memory in 3 areas: a 3 MB "kernel," a 21 MB "root_fs," and a 2 MB "var." Notice that the kernel area only starts at 30 MB, meaning that there is something else between 0 and 30 MB in the flash memory. We also see the separation of the devices between the cores. CPU0 gets everything except one network interface and the NOR/XOR engines. The ip address given is explained further down.
IRQ initialize for core 1
PID hash table entries: 1024 (order: 10, 4096 bytes)
Console: colour dummy device 80x30
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 512MB 0MB 0MB 0MB = 512MB total
Memory: 188800KB available (2984K code, 255K data, 108K init)
More standard stuff with one very interesting bit: we are only initializing core 1. Also notice that although the FS ships with 512 MB of RAM, only 180 MB are available to the Linux OS. I assume that the other core reserves the rest of the RAM for itself.
Calibrating delay loop... 799.53 BogoMIPS (lpj=3997696)
This is the only indication of the processor speed from the OS. /proc/cpuinfo does not report the FS's processor speed.
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
NET: Registered protocol family 16
CPU 1, CPU Interface
-------------
SDRAM_CS0.... ....base 00000000, size 512MB
SDRAM_CS1.... ....disable
SDRAM_CS2.... ....disable
SDRAM_CS3.... ....disable
DEVICE_CS0... ....no such
DEVICE_CS1... ....base fe000000, size 1MB
DEVICE_CS2... ....no such
DEVICE_CS3... ....base fd000000, size 1MB
DEV_BOOCS.... ....base f8000000, size 64MB
DEVICE_SPI... ....no such
PEX0_IO...... ....base f2000000, size 8MB
PEX0_MEM0.... ....base c8000000, size 64MB
PEX1_IO...... ....base f3000000, size 8MB
PEX1_MEM0.... ....base cc000000, size 64MB
PEX2_IO...... ....no such
PEX2_MEM0.... ....no such
PEX3_IO...... ....base f5000000, size 8MB
PEX3_MEM0.... ....base d4000000, size 64MB
PEX4_IO...... ....base f6000000, size 8MB
PEX4_MEM0.... ....base d8000000, size 64MB
PEX5_IO...... ....no such
PEX5_MEM0.... ....no such
PEX6_IO...... ....no such
PEX6_MEM0.... ....no such
PEX7_IO...... ....no such
PEX7_MEM0.... ....no such
CRYPT_ENG..... ....base f7000000, size 2MB
INTER_REGS... ....base f1000000, size 1MB
Marvell Development Board (LSP Version 2.0.2_Patch1_MV78XX0)-- DB-MV78200-A-BP Soc: MV78200 LE
Detected Tclk 200000000 and SysClk 400000000
pci_preinit(): PCI-E 0 not mapped to this CPU.
pci_preinit(): PCI-E 1 not mapped to this CPU.
pci_preinit(): PCI-E 2 not mapped to this CPU.
pci_preinit(): PCI-E 3 not mapped to this CPU.
pci_preinit(): PCI-E 4 not mapped to this CPU.
pci_preinit(): PCI-E 5 not mapped to this CPU.
pci_preinit(): PCI-E 6 not mapped to this CPU.
pci_preinit(): PCI-E 7 not mapped to this CPU.
mv_pci_setup(): PCI-E 0 not mapped to this CPU.
mv_pci_setup(): PCI-E 1 not mapped to this CPU.
mv_pci_setup(): PCI-E 2 not mapped to this CPU.
mv_pci_setup(): PCI-E 3 not mapped to this CPU.
mv_pci_setup(): PCI-E 4 not mapped to this CPU.
mv_pci_setup(): PCI-E 5 not mapped to this CPU.
mv_pci_setup(): PCI-E 6 not mapped to this CPU.
mv_pci_setup(): PCI-E 7 not mapped to this CPU.
SCSI subsystem initialized
NET: Registered protocol family 2
Time: orion_clocksource clocksource has been installed.
IP route cache hash table entries: 8192 (order: 3, 32768 bytes)
TCP established hash table entries: 32768 (order: 6, 262144 bytes)
TCP bind hash table entries: 32768 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 32768 bind 32768)
TCP reno registered
RTC registered.
SHARED_MEM: INIT: Shared Memory mapped into Kernel space at Addr: 0x0 for size 0xc000000
SHARED_MEM: INIT: Shared Memory Header in Kernel space at Addr: 0xa0800000
shared_mem_module_init: Location of shm_dev: 93c14000
Use the XOR engines (acceleration) for enhancing the following functions:
o RAID 5 Xor calculation
o kernel memcpy
o kenrel memzero
o copy user to/from kernel buffers
Number of XOR engines to use: 1
Use IDMA channels for enhancing the following function:
Enable iDMA idma_chan0 interrupt IRQ 4 for iDMA 0
Enable iDMA idma_chan1 interrupt IRQ 5 for iDMA 1
Enable iDMA idma_chan2 interrupt IRQ 6 for iDMA 2
Enable iDMA idma_chan3 interrupt IRQ 7 for iDMA 3
mv_dma_init: iDMA channel 1 settings, state 0x00000000, cause 0x00000000 [0x00000000], mask 0x000004f0 [0x00000000], idmaCause 0x00000000, idmaMask 0x00000018
JFFS2 version 2.2. (NAND) ?ëß 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler anticipatory registered (default)
These are just messages from the initialization of the devices assigned to the core running Linux. Notice that no PCI device is found, since they are all assigned to the other core. This means that Linux is running on CPU1.
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0xf1012100 (irq = 13) is a 16550A
There is a serial port available on the FS. The pins have been found out. See https://github.com/droboports/droboports.github.io/wiki/DroboFS-hardware for more information.
Load Marvell Ethernet Driver
o Cached descriptors in DRAM
o DRAM SW cache-coherency
o Single RX Queue support - ETH_DEF_RXQ=0
o Single TX Queue support - ETH_DEF_TXQ=0
o TCP segmentation offload enabled
o Receive checksum offload enabled
o Transmit checksum offload enabled
o Driver statistics enabled
o Proc tool API enabled
o Rx descripors: q0=128
o Tx descripors: q0=532
o Loading network interface:
Giga 0 is not mapped to this CPU
eth0 Giga 2 is not mapped to this CPU
Giga 3 is not mapped to this CPU
Network interface initialization. It is good to see that the FS uses the hardware off-loading provided by the chip. Again we have confirmation that Linux runs on CPU1, since interfaces 0, 2, and 3 are not mapped to the Linux core.
wait_for_init_shared_mem: Entering Mutex for shared memory init
SHARED_MEM: CPU1: READY TO WAIT FOR INITIALIZATION OF SHARED MEMORY
At this point the Linux core is ready to start communication with the VxWorks core.
NFTL driver: nftlcore.c $Revision: 1.98 $, nftlmount.c $Revision: 1.41 $
cfi_flash_0: Found 1 x16 devices at 0x0 in 16-bit bank
Amd/Fujitsu Extended Query Table at 0x0040
cfi_flash_0: CFI does not contain boot bank location. Assuming top.
number of CFI chips: 1
cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness.
3 cmdlinepart partitions found on MTD device cfi_flash_0
Creating 3 MTD partitions on "cfi_flash_0":
0x01dc0000-0x020c0000 : "kernel"
0x020c0000-0x035c0000 : "root_fs"
0x03b00000-0x03d00000 : "var"
While the shared memory is not yet initialized, the Linux core creates the pseudo-partitions on the flash memory.
mice: PS/2 mouse device common for all mice
oprofile: using timer interrupt.
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
VFP support v0.3: implementor 41 architecture 1 part 10 variant 9 rev 0
eth0: link down
eth0: started
IP-Config: Guessing netmask 255.255.0.0
IP-Config: Complete:
device=eth0, addr=169.254.1.0, mask=255.255.0.0, gw=255.255.255.255,
host=DB78xx0, domain=, nis-domain=(none),
bootserver=169.254.123.234, rootserver=169.254.123.234, rootpath=
eth0: link up, full duplex, speed 1 Gbps
VFS: Mounted root (jffs2 filesystem).
Freeing init memory: 108K
More device initialization. It seems that the FS's ethernet interface is configured to boot from a specific ip address: 169.254.123.234. It is not clear to me whether this means the FS can netboot or not.
SHARED_MEM: CPU1: INITIALIZATION STARTED BY CPU0. WAITING ON INIT DONE
SHARED_MEM: CPU1: INITIALIZATION OF SHARED MEMORY BY CPU0 DONE DETECTED
wait_for_init_shared_mem: Exiting Mutex for shared memory init
wait_for_init_shared_mem: Entering Mutex for shared memory init
wait_for_init_shared_mem: Exiting Mutex for shared memory init
Shared Memory Partition Map Addresses:
-----------------------------------------------------------------------------
SHJ1: Start: 0xa6000000, End: 0xaa000000, Phys: 0x1bfff000
HLRC: Start: 0xab000000, End: 0xb1000000, Phys: 0x15dff000
SGLD: Start: 0xa0a00000, End: 0xa0b00000, Phys: 0x158ff000
SHLX: Start: 0xa5400000, End: 0xa5600000, Phys: 0x15bff000
SHVX: Start: 0xa5800000, End: 0xa5a00000, Phys: 0x159ff000
-----------------------------------------------------------------------------
SHARED_MEM: shm_queue_sync_cores: Mapped SHMQ partition at phys addr: 0x1bdff000 in kernel mem for size 0x200000
SHARED_MEM: CPU1: shm_queue_sync_cores: Ready to wait for init of SHMQ
SHARED_MEM: CPU1: shm_queue_sync_cores: Queue Init Started CPU0. Waiting on Init Done
SHARED_MEM: CPU1: shm_queue_sync_cores: Init of SHMQ done
This is the best evidence for the VxWorks/Linux split design. Here CPU1 (Linux) waits for confirmation from CPU0 (VxWorks) that the shared memory has been initialized. Once this happens, we get the memory mapping for the shared area.
handle_intercore_msg: Got Time Valid message
handle_intercore_msg: Got a NET_INFO request
about to lock the queue
about to look for elements
about to copy the message
about to lock the queue
about to look for elements
about to unlock the queue
about to wait for completion
handle_intercore_msg: Got ISCSI_ENABLE message
init_write_buffer: dev: 93da7400, FreeList: a6000200, FreeBlocks: 131071
handle_intercore_msg: QUEUING ISCSI_ENABLE to sysfs
scsi0 : dri_dnas, version 0.1.0 [20090720], Commands: 0, Aborts: 0, Device Resets: 0
about to copy the message
scsi 0:0:0:0: Direct-Access DROBO DroboPro 1.00 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] Very big device. Trying to use READ CAPACITY(16).
sd 0:0:0:0: [sda] 34359738368 512-byte hardware sectors (17592186 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 1b 00 10 00
sd 0:0:0:0: [sda] Got wrong page
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Very big device. Trying to use READ CAPACITY(16).
sd 0:0:0:0: [sda] 34359738368 512-byte hardware sectors (17592186 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 1b 00 10 00
sd 0:0:0:0: [sda] Got wrong page
sd 0:0:0:0: [sda] Assuming drive cache: write through
sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
sd 0:0:0:0: Attached scsi generic sg0 type 0
Once the shared area is made available, the two cores start exchanging messages to start the SCSI wrapper. In other words, CPU0 tells CPU1 that it can tell the Linux kernel that a new SCSI device is available. This device is the Drobo storage pool (16 TB worth of 512-byte sectors). Notice that there is an "sda2" partition, but that one is never mounted.
process_get_msg:NR trying to free elt: 937d9438, cache: 93cd06e0
handle_intercore_msg: Got Time Valid message
process_get_msg:NR trying to free elt: 937d9400, cache: 93cd06e0
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
Adding 262136k swap on /mnt/DroboFS/swapfile. Priority:-1 extents:67 across:266812k
Once the SCSI device is attached, the filesystem kernel subsystem takes over, and mounts the EXT3 filesystem that resides on the Drobo storage pool. The file /mnt/DroboFS/swapfile is used for swap memory.
There is no known boot log of the other core. However, it seems plausible that the code executed on CPU0 is on the flash memory in the area between 0 MB and 30 MB.
Summary: one of the cores runs Linux, and has access to only one of the network interfaces. That is the network interface we see at the back of the FS. That core does not have direct access to the hard disks, SATA controller, or even the PCI interface. The other core runs (supposedly) VxWorks, and implements the BeyondRAID functionality. The two cores talk to each other using shared memory areas defined at boot time. The Linux core uses a virtual SCSI device that forwards the command to the VxWorks core using shared memory.