NVMe specification - animeshtrivedi/notes GitHub Wiki

Commentary on the NVMe 2.1 specification

Controller page sizes

3.1.4.1 Offset 0h: CAP – Controller Capabilities

Figure 36: Offset 0h: CAP – Controller Capabilities tells about the min/max configurable page sizes.

3.1.4.5 Offset 14h: CC – Controller Configuration

The host can set this a value between min and max controller capabilities.

Figure 41: Offset 14h: CC – Controller Configuration

you can follow this output of 3.1.4 Controller Properties. Figure 33: Property Definition shows which registers are mandatory to implement, which are optional. Below you can see the values of cap.mpsmin, cap.mpsmax and then cc.mps (MPS = memory page size). There may be slight deviation here as this controller implements 1.4 specification while I am reading 2.1.

sudo nvme show-regs /dev/nvme4 -H 
cap     : 3028033fff
	Controller Memory Buffer Supported (CMBS): The Controller Memory Buffer is Not Supported
	Persistent Memory Region Supported (PMRS): The Persistent Memory Region is Not Supported
	Memory Page Size Maximum         (MPSMAX): 4096 bytes
	Memory Page Size Minimum         (MPSMIN): 4096 bytes
	Boot Partition Support              (BPS): No
	Command Sets Supported              (CSS): NVM command set is Supported
	                                           One or more I/O Command Sets are Not Supported
	                                           I/O Command Set is Supported
	NVM Subsystem Reset Supported     (NSSRS): Yes
	Doorbell Stride                   (DSTRD): 4 bytes
	Timeout                              (TO): 20000 ms
	Arbitration Mechanism Supported     (AMS): Weighted Round Robin with Urgent Priority Class is supported
	Contiguous Queues Required          (CQR): Yes
	Maximum Queue Entries Supported    (MQES): 16384

version : 10400
	NVMe specification 1.4

cc      : 460001
	I/O Completion Queue Entry Size (IOCQES): 16 bytes
	I/O Submission Queue Entry Size (IOSQES): 64 bytes
	Shutdown Notification              (SHN): No notification; no effect
	Arbitration Mechanism Selected     (AMS): Round Robin
	Memory Page Size                   (MPS): 4096 bytes
	I/O Command Set Selected           (CSS): NVM Command Set
	Enable                              (EN): Yes

SGL requirements

Section 2.2.4, Capsules and Data Transfer

NVMe over Fabrics requires SGLs for all commands (Fabrics, Admin, and I/O). An SGL may specify the
placement of data within a capsule or the information required to transfer data using an NVMe Transport
specific data transfer mechanism (e.g., via memory transfers as in RDMA). Each NVMe Transport binding
specification defines the SGLs used by a particular NVMe Transport and any capsule SGL and data
placement restrictions.

Where is SGL support? In the NVMe identify command (NVMe identify command).

Figure 312: Identify – Identify Controller Data Structure, I/O Command Set Independent

sudo nvme id-ctrl /dev/nvme0 -H
...
sgls      : 0x70001
 [21:21]: 0	Transport SGL Data Block Descriptor Not Supported
 [20:20]: 0	Address Offsets Not Supported
 [19:19]: 0	Metadata Pointer Containing SGL Descriptor is Not Supported
 [18:18]: 0x1	SGL Length Larger than Buffer Supported
 [17:17]: 0x1	Byte-Aligned Contig. MD Buffer Supported
 [16:16]: 0x1	SGL Bit-Bucket Supported
 [15:8] : 0	SGL Descriptor Threshold
  [2:2] : 0	Keyed SGL Data Block descriptor Not Supported
  [1:0] : 0x1	Scatter-Gather Lists Supported. No Dword alignment required.