Intel® Server Board S2600CP Family - jonmason/ntb GitHub Wiki

NTB support was added to the BIOS version 01.08.0001. If BIOS is not at that version (or newer), update the BIOS to the latest version (download from here).

***WARNING: The S2600CP BIOS does not support NTB BAR size greater than 1M. This is a BIOS restriction. It effectively makes NTB not usable on this platform. There are no future plans to adding proper support on this particular platform. Stay tuned for additional updates. (6/24/2015)

To get NTB (in the NTB-NTB configuration) working, you will need to complete the following steps:

Plug in a External PCI Express Cable to PCIE Adapter Card into Slot #2 of both systems.

Enable NTB in the BIOS

  1. Type F2 to enter the BIOS during boot.
  2. Under the "Advanced" tab, select "PCI Configuration".
  3. Then modify the value of "NTB PCIe Port on CPU Socket 1" to be "NTB to NTB".
  • This will then make 2 new options available "Enable NTB BARs" and "Crosslink control Override".
  • Enable the NTB BARs and configure one of the 2 systems to be "USD/DSP" and the other to be "DSD/USP"

Boot Operating System

You can verify the presence of the NTB device in the hardware via lspci. You should see something like:

00:03.0 Bridge: Intel Corporation Sandy Bridge Non-Transparent Bridge (rev 06)

The NTB device driver was included in the 3.9 Linux kernel. If the driver is already present in your kernel, Linux will detect the NTB device and automatically probe the driver. When both systems have booted and both drivers are probed, you should see the link negotiation in the dmesg output. Something like:

ntb 0000:00:03.0: Link Width 8, Link Speed 2

Congrats, you have connected your systems via NTB!

To get NTB (in the NTB-RP configuration) working, you will need to complete the following steps:

In the "standard" configuration, you have this setup:

System A: Socket 1 NTB = Transparent
    Socket 2 NTB = Transparent
System B: Socket 1 NTB = NTB-RP
    Socket 2, N/A

There is an issue in the BIOS, and to get this working use the following workaround:

System A: Socket 1 NTB = Transparent
    Socket 2 NTB = NTB-RP (doesn't matter if there is a CPU present).
System B: Socket 1 NTB = NTB-RP
    Socket 2, N/A