This page details the bringup procedure for a new SNAP board. It includes configuration of the SNAP power controllers, jasper toolflow, and raspberry pi infrastructure.

Currently (i.e. for the 4 prototype SNAP boards which currently exist) SNAP ships with unprogrammed power controllers, and thus won't do anything until these have been programmed. Raspberry Pi's are not provided, and need obtaining and programming by the user.

For configuring the SNAP hardware, you will need:

• A SNAP Power cable and 12V power supply.
• A windows machine with the Fusion Power Controller GUI installed
• A Texus Instruments GPIO-TO-USB Programmer and cable (10 pin 0.1" pitch ribbon)

For configuring a Raspberry Pi:

• an SD card writer
• a binary image writing tool (on Linux: dd, on Windows: don't use Windows)
• a 40 pin 0.1" pitch ribbon cable (for connecting the Pi to the SNAP)

For building firmware

• Vivado (I've been using 2014.4 2016.4)
• MATLAB (I've been using 2013a 2016b)
• The JASPER toolflow

Grab a copy of the Fusion power controller software from ti.com and install it on a windows PC. Download the SNAP configuration files from github. If you want, clone the entire github repository. Otherwise you only need the two .hex files.

Connect your Texas instruments programmer to your PC (via USB) and to the SNAP (using a 10 pin ribbon connector). The SNAP-side connector is labelled J1 on the board. It is the enclosed, gray 5x2 pin header. After connecting, Windows will probably install some drivers. Let it finish before continuing. Apply 12V power to your SNAP. Open the Fusion Digital Power Designer tool. With any luck, it should auto-detect the two power controller chips on the SNAP board. In the top right of the GUI, a drop down menu will allow you to select between the power controller chips.

To program, do the following:

• Use the drop down menu to select a rail at address 52d (there may be a few at address 52, and a few at address 53. Pick any of the 52's).
• Select File->import->Data Flash
• Select the file "UCD9248 5.8.0.11400 Address 52 Data Flash_gpio.hex" downloaded from github
• Click Next
• Select "WRITE program checksum"
• Click Next
• It should take a few seconds to program, after which you can continue through the prompt. The software will refresh.

Repeat for the other power controller. That is:

• Use the drop down menu to select a rail at address 53d (not 52!) (there may be a few at address 52, and a few at address 53. Pick any of the 53's).
• Select File->import->Data Flash
• Select the file "UCD9248 5.8.0.11400 Address 53 Data Flash_gpio.hex" downloaded from github (note this is not the same file as for the chip at address 52!)
• Click Next
• Select "WRITE program checksum"
• Click Next
• It should take a few seconds to program, after which you can continue through the prompt. The software will refresh.

You can now power cycle the SNAP, after which the various rails on the board should come up in order. If all is well, a SNAP board with idle FPGA should draw about 0.8A at the 12V input (0.9A if a Raspberry Pi is also being powered).

The SNAP is designed to interface with a Raspberry Pi B+ (or Pi 2) which has a 40 pin GPIO header. You can interface with a model B(no plus) or A, but you'll need to cludge the connector.

First, you'll need to download an appropriate Raspberry Pi image.

• For a Pi B+, download a copy of rpi_snap.img.tar.gz from here (drive.google.com).
• For a Pi 2 use the image here (drive.google.com).
• For a Pi 3 use the image here (drive.google.com)

Unzip the file (the resulting image is about 4 GB). Note that this image for the Pi 2/3 is designed to be used with the latest version of casperfpga. Specifically, when programming the SNAP you must use casperfpga's upload_to_ram_and_program method, with a valid .fpg file. "progdev" and boffiles are *no longer supported*.

 tar -xzvf rpi_snap.img.tar.gz

Now you just need to write the image to an SD card. There are plenty of guides to do this online, but the basic steps are:

• Insert a (micro, for a Pi B+/2/3) SD card into an appropriate writer.
• Figure out which device it is on your machine (e.g., sdc, sde, sdX, etc.) ("dmesg | tail" should give you some helpful feedback when you plug the card in.)
• If you're not sure which device your SD card is, don't do anything! (Else you may wipe your main hard drive!)
• When you know which device your SD card is, write the image to it (replace sdX with the correct identifier):

 dd if=/path/to/rpi_snap.img of=/dev/sdX bs=1M

This will take a few minutes to run (if you're bored -- read this which will tell you how to get status info from dd.)

When complete, put the SD card in the Pi, and plug the Pi into the SNAP with a ribbon cable (do this with the power off, you're liable to short something if you misalign the ribbon!). Here's a photo of the SNAP and Pi connected -- make sure you get the orientation right!

Once connected, power the SNAP up. The Pi should switch on (the total current draw at the 12V PSU should be <1A). The image you've loaded on the Pi has it obtain an IP address via DHCP on boot. So make sure a network cable is plugged in, and (preferably) that you have access to the DHCP server which will give it an IP address. If you can log into the machine serving DHCP (simech1 and asa2 do this for the casper.pvt and serendip.pvt networks in the NCH 425 lab) looking at the dhcp lease list should tell you which address the pi has obtained. Eg, on simech1, the command

 less /var/lib/misc/dnsmasq.leases

returns

 1431494293 b8:27:eb:57:a5:20 10.0.1.168 raspberrypi *

i.e., the raspberry pi has address 10.0.1.168.

TODO: Probably should do something with hostnames to support multiple boards... We could support netbooting...

Once you have a Pi connected, and you know it's IP/hostname, you can talk to it using your favourite katcp library (e.g., python's corr, etc.).

Before setting up the JASPER toolflow, you'll need to obtain copies of MATLAB and Vivado. I've been using MATLAB 2013a and Vivado 2014.4 MATLAB 2016b and Vivado 2016.4, both of which are installed on simech1 acme1 in Berkeley.

Next, check out the jasper github repo. Note that the master branch does not include the jasper flow, only the jasper_devel branch. Exactly how you perform the checkout is up to you, but you can clone a copy of the jasper_devel branch with

 git clone https://github.com/jack-h/mlib_devel.git --branch jasper_devel

Next customise your environment variable to suit your local setup. For simech1 acme1, create a file called, vivado_config.local containing the following. For other machines, customize the first five lines to suite your setup. A template can be found in vivado_config.sh, which is in the repository. My config file currently (as of March 2017) reads:

 export XILINX_PATH=/data/opt/Xilinx/Vivado/2016.4  export USE_VIVADO_RUNTIME_FOR_MATLAB=1  export USE_VIVADO=1  export MATLAB_PATH=/usr/local/MATLAB/R2016b  export MLIB_DEVEL_PATH=/home/jackh/casper/mlib_devel  export PLATFORM=lin64  export XILINXD_LICENSE_FILE=/home/jackh/.Xilinx/Xilinx.lic

 export SYSGEN_SCRIPT=$MLIB_DEVEL_PATH/startsg  export MATLAB=$MATLAB_PATH  export CASPER_BASE_PATH=$MLIB_DEVEL_PATH  export HDL_ROOT=$CASPER_BASE_PATH/jasper_library/hdl_sources  export XPS_BASE_PATH=$MLIB_DEVEL_PATH/xps_base  export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/lib64:/lib  source $XILINX_PATH/settings64.sh

When running simulink, the startsg command will automatically source vivado_config.local (if it exists, else it will source vivado_config.sh). If running the backend tools separately to finish a compile without using matlab, you'll need to manually source vivado_config.local before running exec_flow.py (see below).

You'll also need a Xilinx license file. You can either set this by having a XILINXD_LICENSE_FILE=/some/path/Xilinx.lic line in your environment setup file, or by putting a license file somewhere Xilinx looks by default (eg. ~you/.Xilinx/Xilinx.lic)

Having run this, you can fire up MATLAB to get the familiar simulink interface by running (assuming you are in the mlib_devel root directory)

 ./startsg

In theory, from here you can build a design as normal. Currently supported yellow blocks are:

• SNAP yellow block. The XSG yellow block is deprecated. You should use the xps_library/platforms/SNAP block to configure your design.
• software register (not advanced, arbitrary bitwidth options)
• shared bram (all width options should work)
• snap_adc (in the xps_library/adcs blockset)
• ten_gbe_v2 (using sfp ports 0 and 1)
• GPIO (both gpio pins and leds) -- use ROACH2:led and ROACH2:gpio for these (the toolflow ignores the platform, so actually you can use any XXX:gpio XXX:led)
• IP yellow block, which is akin to the "PCORE" block of old. Instructions for this block pending.

Once you have a design you're happy with, make sure your active diagram is the top-level (i.e., you're on the page with the XSG / sysgen blocks) and run, from the MATLAB prompt:

 jasper

This runs (for the interested)

• update diagram
• parse yellow blocks
• generate a .yaml configuration file containing details of yellow blocks
• run system generator to generate HDL code for your simulink design

In principle, the whole compile could run inside MATLAB. But the jasper command will only perform sections of the compile which require MATLAB (i.e., anything directly involving simulink). Once this is complete, a message will invite you to finish the compile from a bash prompt, and tell you the command to do this. If you're going to run this command in a new terminal, remember that you'll have to source the appropriate configuration file first. TODO: exec_flow could be added to PATH

The compile command will be something of the form (assuming you are in the jasper_library directory):

 python ./exec_flow.py --middleware --backend --software -m /path/to/my_model.slx

In principle if you include the --frontend flag you can run exec_flow on a model, and complete an entire compile from a shell, with MATLAB instances spawned as necessary to do the simulink-side stuff. In practice a) it takes an irritating amount of time to spawn a MATLAB instance, and b) this probably doesn't work anymore anyway.

The exec_flow command will build a verilog-based vivado project, and compile it. You'll see a whirlwind of Vivado messages as it does this. Hopefully, without errors. If all has gone to plan, a boffile will be delivered to the build directory <my_model_name>/outputs/_.bof.

You can also find a bitstream and (for SNAP at least) a PROM file, at <my_model_name>/myproj/myproj.runs/impl_1/top.[bit|mcs]. If you wish to open up the Vivado project itself, either to see what the toolflow has actually generated, or to hack things, you can find the project in <my_model_name>/myproj/myproj.xpr

Once you have a boffile, copy it to the /boffiles directory of your SNAP's raspberry pi, as you would with a ROACH. I.e.

 scp my_boffiles.bof pi@raspberrypi:/boffiles/a_new_name_for_the_bof.bof

The raspberry pi username is 'pi' and the password is 'raspberry', as per the default raspbian image.

Once copied, you should be able to program and interact with the firmware as with any other CASPER board. Eg. from ipython:

 import corr  fpga = corr.katcp_wrapper.FpgaClient('raspberrypi')  fpga.progdev('some_boffile.bof') #this will take 10 seconds :(  fpga.listdev()  fpga.write_int('my_sw_reg', 12)  fpga.read_int('my_sw_reg')  # etc.

• When programmed via the Raspberry Pi, the SNAP FPGA is loaded via a bit banging JTAG interface. This is quite slow (~10s for a SNAP FPGA)
• The software interface between Pi and SNAP is SPI, running at 4 MHz. Factoring in messaging overheads, this gives a theoretical maximum throughput of ~1Mb/s see here which is much slower than ROACH. 1000 4-byte register reads (initiated locally, not from a network socket) were timed at 200 ms. (~0.16 Mb/s). In burst reads of 4 kB, the interface reached ~0.5 Mb/s (60 ms per read). Note that these bursts represent single calls to the linux SPI driver for multiple SPI transactions but they are still implemented on the Pi<->FPGA interface as single-word transactions.
• tgtap does not exist on SNAP. Therefore SNAP will not send or respond to ARP messages. If you want to use the 10GbE interface, you must therefore configure the SNAP's ARP table manually. The katcp command 'config_10gbe_core' should provide this functionality (though I didn't know this existed, so I've always configured the 10GbE core by manually writing to the ARP table locations with the blind_write command).

 config_10gbe_core(self, device_name, mac, ip, port, arp_table, gateway=1)

• To configure the SNAP's ADC, you must use a slightly tweaked, SNAP-specific version of the casper_adc16 ruby code. It is available on github. Note you must use the snap branch!

It is now possible to interact with a SNAP board without a Raspberry Pi. Huzzah! To do this, designs must incorporate a Microblaze softcore CPU, which facilitates reading and writing registers in a design, and programming the SNAP board's flash memory with a bootable bitstream.

In order to set up Microblaze control, first you need to burn your flash with a "Golden image". This is a minimal image which the FPGA will boot from on power-up. Once loaded, the golden image facilitates rebooting the FPGA from a different image stored in another flash location. In order to use the SNAP's flash, you must first set switch S1 so that switches 2 and 5 are set to on. (In the on position, the switches are moved towards the edge of the PCB). The other switches on S1 should be off.

If you have a JTAG programmer compatible with Xilinx Vivado (for example, the Xilinx Platform II programmer) you can use this to burn your flash.

The flash binary is generated by the toolflow in the build directory: <model_name>/myproj/myproj.runs/impl_1/top.bin. Use Vivado to burn your SNAP prom with this image. The configuration memory part you will need to select in Vivado is n25q256-3.3v-spi-x1_x2_x4.

If you don't have a JTAG programmer, you can program any Microblaze-enabled design via a Raspberry Pi and the usual toolflow-generated .bof file. Once loaded, if your SNAP is correctly networked, you should be able to interact with the board using the casperfpga code library as below. This library includes the ability to write the Golden Image section of SNAP's flash memory.

Note that if you program a Microblaze-enabled design via Raspberry Pi, you will not be able to use the Pi to talk the designs registers. The implication of a design being Microblze-based is that it's software bus is driven by the Microblaze, not a Pi.

The steps are as follows:

• Build the golden image model with the jasper flow.
• Connect your SNAP to a 10 GbE network which provides a DHCP server.
• Program the resulting boffile on to your board with a Raspberry Pi.
• On your DHCP server, you should see your SNAP obtain an IP address.
• Use this address, and the instructions below, to use casperfpga to update the golden image stored in flash. The binary file you will need is in the build directory -- ''/myproj/myproj.runs/impl_1/top.bin
• Once the flash has been programmed, disconnect the Raspberry Pi and power cycle the board. It should now be accessible via casperfpga.

 from casperfpga import CasperFpga    myfpga = CasperFpga(host='hostname', port=69)

 # Program a new Golden Image. Be careful, if the new image doesn't include the Microblaze interface  # you won't be able to talk to it without first writing a good Golden Image with a Raspberry Pi or JTAG cable  myfpga.transport._program_new_golden_image'/path/to/golden/bin/file.bin'

Support for the SNAP microblaze (or, hypothetically, any other board running similar Microblaze code) has been baked in to the casperfpga package, so it is (at least somewhat) backwards compatible with existing code.

• Download and install casperfpga. Make sure you have the non-standard tapcp library referenced in casperfpga's requirements.txt file.

Now you're ready to go. To use the library:

 from casperfpga import CasperFpga    myfpga = CasperFpga(host='hostname', port=69)  myfpga.estimate_fpga_clock() # returns FPGA clock rate in MHz  myfpga.write_int('sys_scratchpad', 0xdeadbeef)  myfpga.read_uint('sys_scratchpad') # returns 0xdeadbeef

 # program a different bit stream  myfpga.upload_to_ram_and_program('/path/to/bin/file') # will take a couple of minutes to return

 # reboot from Golden Image  myfpga.transport.progdev()

 # reboot from User Image (previously written with upload_to_ram_and_program)  myfpga.transport.progdev(addr=0x800000)

You can also overwrite the golden image with the below commands. Be very sure you want to do this before running the below code.

 # Program a new Golden Image. Be careful, if the new image doesn't include the Microblaze interface  # you won't be able to talk to it without first writing a good Golden Image with a Raspberry Pi or JTAG cable  myfpga.transport._program_new_golden_image'/path/to/golden/bin/file.bin'

This code will probably neaten over time, and will hopefully be expanded to support all of casperfpga's .fpg-file-based features.

Note the following:

• Reprogramming (either via upload_to_ram_and_program or progdev) will kill the Microblaze you are talking to, and hence will throw timeout errors after the programming call. After a few seconds, when the FPGA has rebooted, it should be back up and ready to communicate. In the future, these unnecessary errors will be suppressed. For now, they can be safely ignored.
• After reprogramming, you do not need to reintantiate the CasperFpga object. The connection to the boards is stateless (based on TFTP) and will continue to work as soon as the FPGA has reacquired its IP address.

• If trying to port an old model from non-Vivado version of the CASPER toolflow (basically any of the ROACHX flows) to SNAP, be sure to update_casper_blocks(bdroot) on your model.
• You may have to update non-CASPER-library Xilinx blocks too.
• You will definitely need to update the System Generator block!