To implement the various RDMA transport services, quite a bit of state is required per queue-pair to keep track of various aspects of the protocol. The queue pair state will need to efficiently store both queue pair configuration information and connection state information. The configuration information will be writeable via a configuration interface and readable by both the transmit and receive engines. The connection state information will primarily be shared between the transmit and receive engines. These lists are most likely not exhaustive.

Configuration information

• QP state
• CQ associations
• Destination IP address
• Protection domain
• Transport service type
• SRQ
• etc.

Connection state information

• PSN (TX)
• Expected PSN (RX)
• MSN (TX)
• Expected MSN (RX)
• ACK scheduled status (RX + TX)
• etc.

Phase 2 of the development process includes initial support for RoCEv2, storing state in on-FPGA memory. Limiting storage to only on-FPGA SRAM limits scalability, but it simplifies the implementation and keeping things self-contained can be useful for performance reasons (no cache misses if there is no cache) or in embedded applications (limited or no DRAM).

• Queue pair state storage (transport state information)

Phase 3 of the development process includes more scalable implementations of the phase 2 components, storing state in on-host or on-card DRAM and caching state in on-FPGA memory. These changes will enable support for a large number of queues (QPs/CQs/SRQs).

• DRAM-backed caching of queue pair state information