High Level design - openucx/ucx GitHub Wiki
UCX code consists of 3 parts:
- Protocol Layer - UCP
- Transport Layer - UCT
- Services - UCS
Supports all functionality described in the API, and does not require knowledge of particular hardware. It would try to provide best "out-of-box" performance, combining different hardware mechanisms and transports. It may emulate features which are not directly supported in hardware, such as one-sided operations. In addition, it would support common software protocols which are not implemented in hardware, such as tag matching and generic active messages. More details here
Provides direct access to hardware capabilities, without decision logic which would prefer one hardware mechanism over another. Some functionality may not be supported, due to hardware limitations. The capabilities are exposed in the interface.
More details here.
Collection of generic services, data structures, debug aids, etc.
| What | Where | Why |
| Tag matching | High level | Software protocol |
| RMA/AMO emulation | High level | Software protocol |
| Fragmentation | High level | Software protocol |
| Pending queue | High level | Stateful |
| Multi-transport/channel/rail | High level | OOB optimization |
| Select inline/bcopy/zcopy | High level | optimization logic |
| Reliability (e.g UD) | Low level | Transport specific |
| DMA buffer ownership | Low level | Transport specific |
| Memory registration cache | Low level | Transport dependent |
See also: