LEAP provides the management utility leap-fpga-ctrl as a means of sharing and coordinating FPGAs among users.
leap-fpga-ctrl makes use of several user provided configuration scripts to manage the specific FPGAs within the user system. run invokes leap-fpga-ctrl to execute hybrid FPGA programs.

leap-fpga-ctrl manages the state of each FPGA in the system. FPGAs can be in the following states:

• FREE – unowned by a program
• RESERVED – owned by a particular program
• PROGRAM – being programmed
• ACTIVE – executing a program
• INACTIVE – transient state used in multi-fpga programs. FPGA is owned and programmed, but held in reset.
• RESET

A typical programming cycle involves a run script actuating the FPGA through several steps:

FREE → RESERVED → PROGRAM → INACTIVE → ACTIVE → → FREE

As leap-fpga-ctrl transitions between states, it touches several metadata files in the system and performs actions on behalf of the program.

The default state for FPGAs. The FPGA may retain a previously programmed bitfile, which reduces the need for reprogramming. To retain this information, leap-fpga-ctrl
maintains a lockfile at PREFIX/var/lock/leap/PHYSICAL_ID, which contains metadata about specific physical devices, including user, state, and the programming information.

[keflemin-vs-fpga-2 leap-platforms(203)] cat /usr/var/lock/leap/0000:04:00.0
11300365 1 hello_hybrid_vc707_vivado-4ab9615c13a2a9d730024f000d488bf0

The FPGA has been reserved by a user for exclusive use. This exclusivity is enforced by the lockfile described above. When a programmer reserves an FPGA leap-fpga-ctrl supplies a logical device id to the programmer. This id can be viewed as handle on which the programmer can invoke the various functionalities of leap-fpga-ctrl. Device ids are assigned lexically based on the order of the device descriptors in /etc/leap/config.

The FPGA is transient state in which it may be programmed by the user program. At programming time, the programming script makes use of leap-fpga-ctrl to obtain information about how to program the FPGA. The config file contains a field prog_cable_id which describes the device serial number. Information on finding device serial numbers is here.

LEAP supports programmers which run on multiple platforms. Since loading programs on multiple platforms requires some form of distributed coordination at programming time. Thus, LEAP programs require some mechanism to place programmed into a known state.

Since deactivating a platform sometimes requires the support of platform device drivers, leap-fpga-ctrl provides an interface by which programmers may supply the driver interface to their specific drivers. If a platform in the config file supplies a script, leap-fpga-ctrl will inform the programmer, so that the programmer may take appropriate actions after programming the FPGA. The platform script supplies an executable interface for programming, activating, and deactivating the platform.

A platform in the active state is running an FPGA program. During the transition to the active state, leap-fpga-ctrl will invoke the platform script to perform any activities necessary for running a program on the platform, for example, installing a device driver.

A transient state in which leap-fpga-ctrl drops an existing reservation. leap-fpga-ctrl will also remove any device drivers associated with the platform by invoking the platform script.

Contains a description of the FPGAs located in the current system. Each system may have several FPGAs, and several FPGAs of the same class.

LEAP includes a sample config script which is intended to be used as a template for programmer. However, because it involves highly-system-specific parameters, like device serial numbers, the file must be modified on a per system basis.

# instance of Xilinx VC707.
[VC707.0]
class = VC707 
has_control_script = yes # control script provided at /usr/share/leap/scriptsVC707
bus_id = 0000:04:00.0 #  physical pcie address, used to disambiguate pcie clients
prog_cable_id = -target "digilent_plugin frequency=30000000 device=SN:210201234286" # part serial number used to disambiguate multiple FPGAs on same USB programming cable. 
dev = /dev/bluenoc_0000:04:00.0 # location of physical device driver. 

[ML605]
class = ML605
location = localhost
has_control_script = yes
dev = /dev/bluenoc_0000:01:00.0

# Instance of Altera DE3. Since we communicate with this device over usb, no special mechanisms are needed to program it.                                               
[DE3.0]
class = DE3
                                               

The above sample configuration file declares a single VC707-class FPGA. The class field is required. However, for single FPGA systems, the bus_id and prog_cable_id are not required. The has_control_script permits user-level run script to execute privileged operations, for example, setting permissions on a device driver.

leap-fpga-ctrl manages FPGAs by device id. Generally, programmers need not be aware of this mechanism. Device id are inferred from the lexical order on the config file. For the above config file, the single VC707 has device id 0:

[keflemin-vs-fpga-1 pm(308)] leap-fpga-ctrl --reserve=VC707
0

This directory contains user supplied platform scripts. LEAP supports many different FPGA boards and their associated device drivers. Since each driver typically has a specialized mode of interaction, leap-fpga-ctrl supplies an extensible, script-based interface by which programmers can supply platform-specific management code.

leap-fpga-ctrl looks for the programmer-supplied scripts in the directory PREFIX/usr/share/leap/scripts/. During each phase of the configuration process, leap-fpga-ctrl will invoke the user-supplied script so that the user may take actions appropriate for the step. For example, during the activate step the user may mount the FPGA device driver in a well-known location in the file system.

As part of the LEAP installation, LEAP provides several example scripts. The VC707 script is located here: source:tools/platforms/leap-fpga-ctrl/scripts/VC707.example

You can find detailed information about how to fill in the platform scripts here:

• ML605 and VC707

Platform control scripts may perform activities at program, activate, or deactivate. These scripts contain a number of system-specific variables which must be determined by the programmer for a specific system or class of systems.

Places platforms in a suitable state for programming. Since programming FPGAs may place unpredictable signals on system busses, these busses may need to be disabled prior to programming.

Removes the platform from its reset state. Typical activities include installing and configuring any necessary device drivers.

Places the platform in a known reset state. Typically this involves deactivating device drivers.

leap-fpga-ctrl manages device drivers, and therefore operates with root privileges. When writing scripts invoked by leap-fpga-ctrl users should be careful to defend their scripts against malicious use.

Xilinx and Altera require some driver setup to enable device programming. A list of programming cable help pages can be found here.

Sometimes FPGAs can be left in an unclean state, for example, due to erroneous program termination or a run-away job left by another user. In this case, leap-fpga-ctrl provides a mechanism for resetting its metadata state. The symptom of this issue is a warning that no free devices are available:

[keflemin-vs-fpga-1 pm(307)] ./run
leap-fpga-ctrl: Error - Failed to find free device matching requested class

to fix, manually free devices according to their device id:

[keflemin-vs-fpga-1 pm(308)] leap-fpga-ctrl --status
vc707.0:
  Current state: Reserved
  Current owner: keflemin
  Current signature: hello_hybrid_vc707_vivado-4ab9615c13a2a9d730024f000d488bf0

[keflemin-vs-fpga-1 pm(307)] leap-fpga-ctrl --reset --device-id 0