building_running_scif - noma/ham GitHub Wiki

Building and Running: Intel Xeon Phi (MPSS/SCIF)

The Intel Xeon Phi accelerator was the first target for HAM-Offload. However the Xeon Phi product line has been discontinued by Intel in the meantime. The SCIF backend is still supported and part of the current version, but it has not been tested due to a lack of hardware. So please give it a try and let me know whether it works or not - I'd be happy to fix it or accept pull requests.

The last tested HAM-Offload release is v2.0 which was built using Boost.Build and had different dependencies. The build Process is described in the included README.md. It should work fine.

Also, the MPI build should work, but has a bit more overhead for purely local offloads, as compared with the SCIF backend (which lacks offload over fabric on the other hand).

From the v0.2 README

With SCIF, you have to start the host process (either on a Xeon Phi, or on a host) first, followed by the other processes that are offload targets. During initialisation all the offload targets connect to the host in a server-client like fashion. There is a set of options that can be passed either via the environment (HAM_OPTIONS) or via the command-line (if it does not interfere with the application code). These are the options:

Option	Description
--ham-cpu-affinity arg	sets the CPU affinity of the process to arg (no affinity is set if omitted)
--ham-process-count arg	sets the number of processes of your application
--ham-address arg	sets an individual address for each process (between 0 and ham-process-count - 1)
--ham-host-address	optional: use a host address other than 0
--ham-host-node	optional: specifies the SCIF node (physical host or MIC) where the host process is spawned on (physical host node is always 0 = default)

Example (host to Xeon Phi offload):

# Use different terminals or start jobs in background (by appending &)
# host process 
$ <application_binary> --ham-process-count 2 --ham-address 0 
# MIC process
$ ssh mic0 `pwd`<application_binary_mic> --ham-process-count 2 --ham-address 1

The same Example using the environment to pass the options:

# host process 
$ HAM_OPTIONS="--ham-process-count 2 --ham-address 0" <application_binary> 
# MIC process
$ ssh mic0 env HAM_OPTIONS="--ham-process-count 2 --ham-address 1" `pwd`<application_binary_mic>

Example (reverse offload):
The MIC process is the logical host (address 0, and started first). The host's node address of the SCIF network must be explicitly specified this time.
MIC process:

# MIC process
$ ssh mic0 `pwd`<application_binary_mic> --ham-process-count 2 --ham-address 0 --ham-host-node 1
# host process:
$ <application_binary> --ham-process-count 2 --ham-address 0 --ham-host-node 1

The inner_product example can be started like this. The env LD_LIBRARY_PATH=$MIC_LD_LIBRARY_PATH may or may not be necessary depending on your configuration.

$ bin/intel-linux/release/inlining-on/threading-multi/inner_product_scif --ham-process-count 2 --ham-address 0 &
$ ssh mic0 env LD_LIBRARY_PATH=$MIC_LD_LIBRARY_PATH `pwd`/bin/intel-linux/release_mic/inlining-on/threading-multi/inner_product_scif --ham-process-count 2 --ham-address 1