ROACH2 - david-macmahon/wiki_convert_test GitHub Wiki

LATEST REVISION HERE -> ROACH-2 Revision 2

ROACH-2 is a stand-alone FPGA board and is the successor to the original ROACH board. ROACH stands for Reconfigurable Open Architecture Computing Hardware.

ROACH-2 was designed as the sequel to ROACH 1 using the new Xilinx Virtex-6 series of FPGAs. It maintains all the aspects that made ROACH 1 a success, but increase the overall performance in terms of processing power, IO throughput and memory bandwidth. It uses the same PowerPC 440EPx present on the ROACH 1 but adds a unified JTAG interface provided through an FTDI FT4232H IC.

The ROACH-2 design includes the following notable features:

• Virtex-6 SX475T FPGA (XC6VSX475T-1FFG1759C)
• PowerPC 440EPx stand-alone processor to provide control functions
• 2 x Multi-gigabit transceiver break out card slots, supporting up to 8x10Ge links which may be CX4 or SFP+
• 4 x 36 * 2M QDR II+ SRAMs connected to the FPGA
• A single 72-bit DDR3 RDIMM slot connected to the FPGA
• 2 x ZDOKs
• An FTDI FT4232H USB to JTAG, serial and IIC

The current block diagram is shown below...

The first prototypes have been assembled and are being brought up. Both the PowerPC and the FTDI debug subsystem have been brought up and the FPGA is successfully being programmed via JTAG.

The svn repository for ROACH2 is hosted on a KAT server at the moment. Certain files will be loaded over onto http://casper.berkeley.edu/svn/roach2. If you wish to access the kat svnROACH2 repository you can contact Philip Gibbs or Francois Kapp.

You can find final revision 0 (the first prototype) schematics at: http://casper.berkeley.edu/svn/roach2/elec/roach2/release/rev0/A/PDF_SCHEMATICS/roach2_sch.pdf

You can find possibly/probably? final Roach2 rev1 schematics at; https://casper.berkeley.edu/wiki/images/a/a5/Roach2_rev1_schematics.pdf

You can find all the design files for the first prototype, including gerbers, at: http://casper.berkeley.edu/svn/roach2/elec/roach2/release/rev0/A/

The roach2.PcbDoc and various schematics files like qdr0_1.SchDoc may be viewed on Windows computers using Altium's free viewer found at http://www.altium.com/community/downloads/en/downloads_home.cfm specifically the 131 MB file at http://downloads.altium.com/altiumdesigner/AltiumDesignerViewerBuild9.3.0.19153.zip No registration is required and install is simple.

ROACH-2 was primarily specified for the packetised correlator, which was considered to be the most demanding application. A number of configurations were considered, including PAPER, ATA and MeerKAT's upcoming requirements. The calculations for memory sizing, datarates and bus widths can be found here.

ROACH-2 introduces slots for two multi-gigabit transceiver mezzanine cards. This add flexibility to choose whichever 10Ge physical layer is most convenient. It also provides an opportunity to research high-speed ADCs.

A CX4 version of this card has been designed and tested. The card supports three CX4 ports, a fourth set of XAUI signals is taken to two headers. There are an additional 6 x 1.5V CMOS IOs taken to a gpio connector. The CX4 port support active cables. More information is here: CX4 (3 x CX4 Mezzanine board)

A quad SFP+ version is in progress. See SFP+ (Quad SFP+ Mezzanine board).

The slide show for the design review can be found at: http://casper.berkeley.edu/svn/roach2/doc/review/ROACH%20II%20Review.pdf

Prototype run of two boards completed with Tellumat Jan 2011.

Pre-production run of 5-10 boards planned with Tellumat May-June 2011.

Full production still TBD.

Currently the MMSGE toolflow will only support 11.4-5 ISE/EDK tools. The board support package supports any ISE version beyond 11.4. In the long term we need to get the MSSGE tools working with newer tools.

media:roach2_knight_rider.mpg

A preliminary datasheet can be found here: http://casper.berkeley.edu/svn/roach2/doc/pcb_review/doc/DataSheet/ROACHII%20Datasheet%20Rev1A%20Draft.pdf

We have now tested enough that we are happy to start working on the next rev. Only the dram remains untested.

The following works:

• PowerPC subsystem: including DRAM/USB/1Ge/Serial/External Bus
  • Uboot boots cleanly, as does Linux

• FTDI subsystem: including JTAG, UARTs (but not including I2C)

• FPGA
  • SelectMAP programming
  • PPC External Bus
  • SGMII 1Ge
  • ZDOKs
  • Configuration via JTAG
  • QDRs
  • 10Ge (with CX4 mezzanine card)

• I2C sensors

Will not test:

• FPGA
  • DDR3: The IO hasn't been allocated in a way that the Xilinx controller likes. Will wait until next revision to fix. The alternative of writing a custom controller is too much work.

Issues unresolved:

• Xilinx Impact cant program the cpld via the FTDI, only the platform usb programmer works.
• The MAX16071 IC published datasheet had mistakes. After much hassle the actual pinout was found and

the chip function with a couple of easy board changes. Existing boards have been/will be reworked and the production version of the board will incorporate the revisions.

Derived from Philip Gibbs' email of 2011sep12

• 1 for these very initial steps there need not be a fan on U1, the large FPGA, as that part won't be programmed and thus will remain at or just above the ambient temperature.
• 2 connect a current-limiting DC power supply unit (PSU) to the 5Vaux input at P1, 5x2 0.1" pitch receptacle, which is adjacent to J1 (the ATX PSU connector). Pin 2 is 5VAUX and pin 1 is GND.
• 3 Set the PSU current limit to minimum setting.
• 4 Turn on the PSU and slowly increase the current. You'll notice straight away whether you have a short or not.
• 5 Once the current limit is set at no more than 0.06 Amp look for the AUX PWR LED DS25 to be lit. DS25 is located between the SD card slot and the ZDOK 0 connector P10.
• 6 Check the output levels of the various DC-DC Modules to make sure there is no back powering. At one point this was an issue but should now be completely resolved. To do this check: R21 (1.5V), R23 (5V), R27 (1.6V), R24 (12V), R11 (1.8V), R4 (2.5V), R28 (3.3V)
• 7 Once you're happy that nothing is "heating up" you can remove the 5Vaux PSU and switch to the standard ATX PSU.
• 8 Using the standard ATX PSU the board is switched on by pressing the POWER button on the top left of the PCB (S4). S4 is about 2" above the FPGAs DIMM.
• 9 To power down, press and hold the POWER button for approximately 4 seconds.
• 10 In preparation for further testing a fan needs to be mounted to U1 if not already done. Using the Radian Heatsinks F142.5+Y+T725 part as an example, this is done when the board is powered off by performing these steps
  • Remove the non-stick backing from the adhesive
  • Mount the fan to U1 with the fins aligned from ZDok to Mezzanine board connectors to allow the best air flow in from the front of the chassis and out via the back.
  • Hook the fan assembly's yellow clips under the FPGA. Space for the clips is provided by the package's solder balls.
  • Gently lever the clips into position using the a screwdriver in the slots in the yellow clips.
  • Connect the fans 3 line cable to the 1x3 header J2 labeled "FGPA".

Below are a collection of pictures of board 010101 which is one of the first pre-production boards produced at DigiCom.

• Roach2 S/N 010101 top
• Roach2 S/N 010101 bottom
• Roach2 S/N 010101 top left
• Roach2 S/N 010101 top middle
• Roach2 S/N 010101 top right
• Roach2 S/N 010101 middle left
• Roach2 S/N 010101 middle middle
• Roach2 S/N 010101 middle right
• Roach2 S/N 010101 bottom left
• Roach2 S/N 010101 bottom middle
• Roach2 S/N 010101 bottom right

Here are a couple of pictures highlighting the rework required to compensate for the errors in the datasheet Maxim published for the MAX16071 current and voltage monitoring IC.

• Roach2 S/N 010101 U23
• Roach2 S/N 010101 U16

• Getting Started with ROACH2.
• ROACH2 DDR3 Modules.
• Sync inputs
  • For Roach2 rev1, and Roach2 rev2, there are at least two possible sync inputs:
    • via the vertical mount SMA receptacle connector J10. This signal is terminated into 50 ohms and then turned into the LVDS differential pair "AUX_SYNCI_P" and "AUX_SYNCI_N" via an Analog Devices ADCMP605BCPZ comparator and enters the FPGA at pins BB16 and BB17.
    • via the vertical mount SMA receptacle connector J9. This signal is terminated into 50 ohms and then turned into the LVDS differential pair "AUX_CKL_P" and "AUX_CLK_N" via an Analog Devices ADCMP605BCPZ comparator and enters the FPGA at pins AV16 and AW16. These pins are designated as single region clock capable (SRCC) and thus are apt to be better suited to drive signals with stricter timing requirements (high speed, numerous loads, ...)
    • These pins are in the standard Roach1 design libraries but the updated GPIO block in the ADC16 gateware repository may be required for the Roach2 rev 2.