繁體中文 00.01.NVMe is an interface between a host and a storage controller. This note establishes how the host submits commands, how the controller reports results, and what namespaces, controllers, and NVM subsystems represent. These concepts provide the foundation for later commands and fields.controllerController, the entity that implements the NVMe interface, fetches commands, and reports completions.HostThe system running the operating system and issuing NVMe commands.NVMeNon-Volatile Memory Express, the specification family for a host interface to a non-volatile-memory subsystem.NVMNon-Volatile Memory, memory that retains data without power. The main ideas 01Specification responsibilities01-01Distinguish what Base, Transport, and I/O Command Set specifications define.I/OInput/Output, the class...
read more
NVMe Base 2.4 Chapters 1-2: Specification Language, PCIe Queues, and Storage Model
00.01.NVMe is an interface between a host and a storage controller. This note establishes how the host submits commands, how the controller reports results, and what namespaces, controllers, and NVM subsystems represent. These concepts provide the foundation for later commands and fields.
- controller
- Controller, the entity that implements the NVMe interface, fetches commands, and reports completions.
- Host
- The system running the operating system and issuing NVMe commands.
- NVMe
- Non-Volatile Memory Express, the specification family for a host interface to a non-volatile-memory subsystem.
- NVM
- Non-Volatile Memory, memory that retains data without power.
The main ideas
Specification responsibilities
01-01Distinguish what Base, Transport, and I/O Command Set specifications define.
- I/O
- Input/Output, the class of data operations performed on a namespace.
Command round trips
02-01Understand cooperation between host and controller through submission and completion queues.
Storage objects and paths
03-01Distinguish namespaces, controllers, and subsystems, including multiple access paths.
00.02.The host includes the operating system and driver; the controller provides the NVMe interface accessible to that host. NVMe describes host-visible behavior, which does not directly specify the SSD’s physical NAND organization.
Connecting the main ideas
00.03.Establish hosts, controllers, and namespaces, then follow command submission and completion. The specification family assigns responsibility for those relationships, while numeric conventions and units provide the tools for reading later fields.
00.04.The aim is to draw the path from host to storage, explain where commands and data reside, and distinguish multiple paths to one storage object from sharing by multiple hosts. Later reports apply these foundations to specific mechanisms.
01 Roles of Base, Command Set, and Transport
01.01.When a command, register, or data format appears, the first question is not merely where it is found, but which specification owns the definition. Base supplies the common protocol, the Transport adds the PCIe binding, and an I/O Command Set defines namespace data operations. The boxes in Figure 1 show applicability, not mandatory packet traversal through a stack.
- namespace
- Namespace, a formatted quantity of non-volatile memory accessed by a host through a controller.
- PCIe
- PCI Express, the transport and device interconnect used by an NVMe memory-based controller.
01.02.The Base Specification defines the common NVMe protocol; a Transport Specification binds it to a transport, and an I/O Command Set Specification extends commands and data structures. This is an applicability relationship, not a protocol stack.
Sources: Base 2.4 §1.1.1
Source: NVME-BASE-2.4, Rev. 2.4, §1.1.1, printed pages 1, PDF pages 27
| Specification | Content defined | Relationship to other specifications |
|---|---|---|
| Base | Common commands, queues, status, and structures | Do not assume it owns every PCIe-register detail |
| PCIe Transport | BARs, MMIO, doorbells, interrupts, and PCIe-specific behavior | It does not override Base in a conflict |
| I/O Command Set | Specific namespace I/O commands and extensions | It does not redefine the transport |
- MMIO
- Memory-Mapped I/O, access to device registers through CPU memory operations.
02 Namespaces, controllers, and access paths
02.01.A namespace is the formatted capacity actually accessed by the host, while capacity management, endurance, reclamation, and paths live at different levels. Figures 11-18 describe containment using NVM Sets or Reclaim Groups; Figures 19-22 instead show controllers, ports, paths, and PCIe Functions. The two groups answer different questions and must not be collapsed into a falsely one-to-one tree.
02.02.The storage model expresses containment through the NVM subsystem, domain, Endurance Group, NVM Set or Reclaim Group, Reclaim Unit, and namespace. A namespace is the formatted capacity a host accesses through a controller.
- Endurance Group
- Endurance Group, a group of NVM resources for isolating and reporting endurance-related state.
- NVM subsystem
- NVM subsystem, the NVMe system boundary containing controllers, ports, namespaces, and non-volatile storage resources.
- Reclaim Group
- Reclaim Group, a set of non-volatile storage resources with shared reclamation behavior.
- Reclaim Unit
- Reclaim Unit, a smaller management granularity used when a controller reclaims media.
- NVM Set
- NVM Set, a capacity grouping that associates namespaces with a managed set of NVM resources.
Sources: Base 2.4 §2.3.1
Source: NVME-BASE-2.4, Rev. 2.4, §2.3.1, printed pages 26-33, PDF pages 52-59
| Access arrangement | Hosts and storage objects involved | Problem addressed |
|---|---|---|
| Multi-path I/O | One host and one namespace with two or more independent paths | Focus: path redundancy |
| Namespace sharing | Two or more hosts access one shared namespace | Focus: host ownership and coordination |
| SR-IOV | One PCIe device exposes PFs/VFs | A PCIe Function need not be an independent subsystem |
- SR-IOV
- Single Root I/O Virtualization, a PCIe capability that exposes one PF and multiple VFs from one device.
03 Command submission and completion
03.01.The host does not write a command directly into the controller. It builds an SQE in memory and publishes a new SQ tail; the controller fetches and executes the command, then places a CQE into a CQ. The 1:1 and n:1 distinction in Figures 6 and 7 concerns whether multiple SQs share one CQ, not whether commands share one SQE.
- CQE
- Completion Queue Entry, one completion-result structure in a CQ.
- SQE
- Submission Queue Entry, one command structure in an SQ.
- CQ
- Completion Queue, the queue into which a controller posts command completions.
- SQ
- Submission Queue, the queue into which the host places commands.
- The host writes a command to the Submission Queue (SQ).
- The host updates the SQ Tail Doorbell to announce new work.
- The controller retrieves and executes the command, then writes its result to the Completion Queue (CQ).
- The host reads the CQE and updates the CQ Head Doorbell to release consumed entries.
03.02.In the PCIe memory-based model, Submission and Completion Queues reside in memory. Multiple I/O Submission Queues may share an I/O Completion Queue, while the Admin queue pair remains one-to-one.
Sources: Base 2.4 §2.1
Source: NVME-BASE-2.4, Rev. 2.4, §2.1, printed pages 21-23, PDF pages 47-49
| Queue arrangement | SQ-to-CQ relationship | Identifying a completed command |
|---|---|---|
| Admin queue pair | One Admin SQ to one Admin CQ | Initialization and management path |
| I/O 1:1 | One I/O SQ to one I/O CQ | Simple tracking and clear isolation |
| I/O n:1 | Multiple I/O SQs share one I/O CQ | Merged completion path; SQID/CID still recover the command |
- SQID
- Submission Queue Identifier, the numeric identifier of the SQ containing a command.
- CID
- Command Identifier, used with the SQ identifier to identify an outstanding command.
04 Numeric encodings and units
04.01.Before reading any raw value, establish its unit and encoding. A zero-based count of 3 may represent 4 units; an index selects a list item, while an offset measures distance from a start. They are not interchangeable.
- zero-based
- zero-based: numbering starts at zero, so raw=3 can mean the fourth item or four units; the field definition still decides which.
- raw value
- raw value: the value read directly from a field before applying zero-based, unit, or scaling rules; confirm the definition before converting it.
- offset
- offset: a displacement measured from a stated start. It answers “how far from the start,” unlike an index.
- index
- index: selects an item or format in a list. It answers “which one,” not “how far from the start.”
04.02.A value is interpreted together with its radix and units. Hexadecimal uses the h suffix, binary uses b, and decimal may omit d. Decimal and binary capacity prefixes represent different multipliers.
Sources: Base 2.4 §1.4.2
Source: NVME-BASE-2.4, Rev. 2.4, §1.4.2, printed pages 3-5, PDF pages 29-31
| Field or notation | Decoded value | Basis for the interpretation |
|---|---|---|
| 1000 | Decimal 1000 | No b/h suffix means decimal |
| 1000b | Binary value 8 | b is a radix marker, not a bit unit |
| 1000h | Hexadecimal value 4096 | Common for offsets and register values |
| NUMD=0 | One actual dword | Add one only when the field is explicitly zero-based |
- Dword
- Dword (double word): 32 bits, or 4 bytes. A word is 16 bits; for example, a zero-based dword count of 3 represents 4 Dwords, or 16 bytes.
- NUMD
- Number of Dwords, a zero-based transfer-dword count; actual bytes = (NUMD + 1) × 4.
Where to continue in the specification
05.01.Use the flow above to frame the problem, then open the corresponding sections for fields and full conditions. The Chinese tutorial also explains every in-scope figure with its takeaway, example, and details.
| Concept to explain | Specification sections |
|---|---|
| Roles of Base, Command Set, and Transport | Base 2.4 §1.1.1 · Base 2.4 §2.3.2 |
| Namespaces, controllers, and access paths | Base 2.4 §2.3.1 · Base 2.4 §2.3.3 · Base 2.4 §2.4.1 · Base 2.4 §2.4.2 |
| Command submission and completion | Base 2.4 §2.1 · Base 2.4 §2.3.3 |
| Numeric encodings and units | Base 2.4 §1.4.2 · Base 2.4 §1.4.3 |
Check your understanding
1. Why do Read semantics and delivery over PCIe require different specifications?
06.01.The Command Set defines the operation on data; the Transport defines how commands and completions cross the connection; Base supplies the shared controller, queue, and management model. Together they describe the complete operation.
Sources
Source: NVME-BASE-2.4, Rev. 2.4, §1.1.1, printed pages 1, PDF pages 27
Source: NVME-BASE-2.4, Rev. 2.4, §2.3.2, printed pages 33, PDF pages 59
2. Where is the command itself when the host writes a doorbell?
06.02.The command is already in a Submission Queue entry. The doorbell updates queue progress so the controller can determine the available work; it does not carry the entire command.
Sources
Source: NVME-BASE-2.4, Rev. 2.4, §2.1, printed pages 21-23, PDF pages 47-49
4. Why can a raw length value of 3 not immediately be interpreted as 3 bytes?
06.04.The unit and encoding must be established first. A zero-based dword count of 3 means 4 dwords, or 16 bytes. An actual byte count of 3 instead means 3 bytes.
Sources
Source: NVME-BASE-2.4, Rev. 2.4, §1.4.2, printed pages 3-5, PDF pages 29-31
Source: NVME-BASE-2.4, Rev. 2.4, §1.4.3, printed pages 5, PDF pages 31
Comments