Setup Guide - farhanrahman/riffa GitHub Wiki

The drivers provided by RIFFA needs to be installed on your (riffa compatible) Linux distribution but before that the hardware needs to be synthesised and ported onto your FPGA development board. For that you need the listed CAD tools on the Home page. I faced a few problems trying to setup the Xilinx tools on Ubuntu therefore I advise the user to go through the Xilinx tools setup page to get your software running properly.

Programming FPGA board

Once you have setup the Xilinx tools, you can choose to do ONE of the following things to port the design onto your FPGA board:

Use SystemAce provided in the repository

The repository includes the contents of a Compact Flash drive in the directory flash/RiffaInt.

  • Copy the contents onto your Compact Flash card. I generally empty the flash card before copying but as long as there is a single 'xilinx.sys' file in the root directory and the corresponding directory containing the .ace file in it, it should be okay.

  • You then need to configure your board dip switches so that it programs itself from the .ace file when the system boots up. The configuration bits I had to use was the following:

//Need to include the configuration here

OR

Generating Bitstream from XPS project provided

If you have setup Xilinx tools according to the instructions in Xilinx tools setup then the following instructions should work.

Open up Xilinx Platform Studio:

  • Run the command xps
  • Select Open Project
  • Go to the directory project/base_systems/ml505 and select the system.xmp file.
  • If you are on a newer version of XPS (version used for this project: 14.1) then it will migrate to the new version creating a backup file for your project. Once open XPS will ask for the files Central Notifier. Specify the directory riffa/pcores/central_notifier_v2_00_a for the missing file. *You can then click Generate Bitstream on the left hand side on the Navigator menu *After the process completes the system.bit file will be generated in the implementation folder.

OR

Creating your own XPS Design from scratch / Integrate RIFFA into existing design

Information about creating your own XPS project from scratch is given in the Getting Started section on the RIFFA website. I will still include the instructions here:

Assuming that the above Xilinx tools setup instructions were followed to install the Xilinx tools, start up Xilinx XPS by typing in shell xps in your command line.

Xilinx instructions for PLB based systems:

  1. Add the RIFFA distribution pcores into either your XPS global pcore repository (recommended, I used it as well so it works that way) or into the pcores directory of your base system directory for example in this case the pcores directory is project/base_systems/ml505/pcores directory. Then you can refresh XPS by re-scanning the IP cores library in XPS. There should be a button to do that no the top left side of the pcores library screen (will post picture if required?). Alternatively restart XPS if the first one does not work.

  2. Add a plbv46_pcie Xilinx PCIe Bridge core and xps_central_dma Xilinx DMA core to your design.

  3. Connect the master and slave ports of the xps_central_dma and plbv46_pcie to your PLB system bus. Configure the xps_central_dma with:

  4. The maximum FIFO depth (currently 48).

  5. The maximum read & write burst size (currently 16).

  6. Make sure it has a valid address on the system bus. This address will be used in step 9.

  7. Configure the plbv46_pcie with:

  8. A vendor id = 10EE (for Xilinx) and a device id appropriate to your FPGA (the ML505 = 0505). I used a device id 0509. It doesn't really matter as long as the drivers are configured correctly. However it is good to stick to the device id recognised by everyone and not just a made up device id. I just took the values from the PCI database. These values will be used by the driver installation described later.

  9. The completion timeout should be checked.

  10. The PCIe capabilities register slot implemented should be checked.

  11. Configure 1 PCIe BAR, of size 213 = 8 KB. The address should be 0x80000000. This value will be used in step 8.

  12. Configure 6 IPIF BARs, of size 222 = 4 MB. The address of the first IPIF BAR should be 0xA0000000. All the bars should follow and be adjacent (e.g. the second IPIF BAR starts at 0xA0400000). The remote translation address can be left at 0x00000000. These values will be assigned at boot time by the PC. The starting address and size of IPIF BARs is used by the driver.

  13. Make sure the plbv46_pcie has a valid address on the system bus. Note, this is separate from the BAR addresses. In the Address tab, this shows as the C_BASEADDR address for the plbv46_pcie. This value will be used in step 9.

  14. Add a central_notifier, a simbpus_mst_plbv46_adapter, and a simpbus_slv_plbv46_adapter to the design.

  15. Connect both adapter cores to the system bus and connect the SIMPBUS interfaces on both adapter cores to the central_notifier.

  16. Configure the simpbus_slv_plbv46_adapter to have an address of 0x80000000 and size of 8 KB. This must be the same address and size as in step 4. In truth, the address can be anything, as long as it doesn't conflict with any other core on the system bus and is consistent with the value of the PCIe BAR.

  17. Configure the central_notifier with:

  18. Set the number of channels as desired. This enables the same number of buses on the Bus Interfaces tab.

  19. Check the init bus box if you want to enable and connect an IP core to perform system initialization using the SIMPBUS interface. Note, do not check this box unless you do connect an IP core that will perform initialization.

  20. Set the DMA address to the value assigned in step 4.3.

  21. Set the PCIe Bridge address to the value assigned in step 5.6. This is the C_BASEADDR value of the plbv46_pcie component.

  22. Set the IPIF BAR length to 4 MB.

  23. Configure the SIMPBUS data width to the PLB data width.

  24. Connect the remaining central_notifier ports:

  25. SYS_CLK should be connected to the clock used for the PLB bus.

  26. SYS_RST should be connected to the system reset.

  27. INTR_PCI should be connected to plbv46_pcie's MSI_request port.

  28. INTR_DMA should be connected to the xps_central_dma's IP2INTC_Irpt port.

Sometimes Xilinx Platform Studio (XPS) will not allow you to make some of the necessary port connections. In that case you can alter the .mhs file, which is more of a hacky way of getting things done I guess (but I still prefer it that way).

Look into the project system.mhs file for more details/reference for how things should be connected in your own design to make use of riffa.

#Programming the device

Once you have generated the bitstream file, you have to program the FPGA device. Have a look at this guide to see how to produce a SystemAce file for the FPGA device.

Installing the drivers

Information about installing the drivers is also available in the Getting Started section of the RIFFA website. So I will just copy the contents here:

The Linux driver must be installed before applications can access the FPGA. The driver is located in the riffa/pcores/central_notifier_v2_00_a/sw/linux/driver directory. To build and install the driver, you'll need root/sudo privileges. The steps are below:

Ubuntu/Debian and Fedora instructions:

  1. In a terminal, move into the riffa/pcores/central_notifier_v2_00_a/sw/linux/driver directory.

  2. Execute:

sudo make setup

This will ensure that your Linux system has the kernel headers that correspond to the current version of the kernel you're running. If you don't have them installed, this command will attempt to install them, typically into /usr/src. You only ever need to run this once to ensure you have the kernel headers. If you know you have them, you can skip this step.

  1. Execute:
make

This will compile the driver against your kernel. You will probably receive an error message indicating that you must specify variable value for VENDOR_ID and DEVICE_ID. You may update the Makefile to includes these values (see the top of the Makefile). Or you can pass them in via the command line (e.g. make VENDOR_ID=10EE DEVICE_ID=0509). These values correspond to the PCIe header values configured in your PCIe endpoint (see above). They must match or the OS will not load the driver when it detects the FPGA's PCIe endpoint.

  1. Execute:
sudo make install

This will install the driver into the kernel to be automatically loaded at boot time. You can uninstall the driver by executing:

sudo make uninstall

This will remove everything that was installed. If you want to manually load the driver you can always execute:

sudo make load

Since you need to reboot the computer every time you download a new image to your FPGA, installing it is the best choice.

Reboot

Once you have done the above things, you need to shut down your computer and boot up again. The way to tell the installation worked is by opening the terminal and typing in the following command:

lspci | grep "Xilinx"

OR just

lspci

OR

Check for evidence that the driver loaded by executing:

dmesg

If the driver loaded correctly you should see something like the following:

[   12.759709] FPGA PCIe endpoint name: 0000:02:00.00
[   12.759735] BAR 0 address: d0000000
[   12.759736] BAR 0 length: 8192
[   12.759791] fpga 0000:02:00.0: irq 28 for MSI/MSI-X
[   12.759819] MSI setup on irq 28
[   12.761444] gDMABuffer 0: ffff880037400000 -> 0000000037400000
[   12.763685] gDMABuffer 1: ffff8800bf800000 -> 00000000bf800000
[   12.765908] gDMABuffer 2: ffff8800bf400000 -> 00000000bf400000
[   12.768100] gDMABuffer 3: ffff8800bf000000 -> 00000000bf000000
[   12.770268] gDMABuffer 4: ffff8800bec00000 -> 00000000bec00000
[   12.772382] gDMABuffer 5: ffff8800be800000 -> 00000000be800000

If you don't see something like that, grep your /var/log/syslog for the term: FPGA. Double check that you've set the same VENDOR_ID and DEVICE_ID in your driver as you have in your PCIe endpoint configuration. You may also check the /dev directory to see if the fpga device was created. It should be listed as /dev/fpga.

You will see an information about the board with the Vendor ID that you specified while installing the drivers and when setting up the XPS project to generate the bitstream.

#Reference

Most of the above information is available on the RIFFA Getting Started section.