NVM Command Set 1.3: Logical blocks, I/O commands, and data protection

32 minute read

繁體中文

00.01.The NVM Command Set defines how a host reads, writes, and manages storage in logical blocks. Its central connections are data formats, command behavior, data integrity, and resource management: a command’s applicable conditions depend on the namespace format, advertised capabilities, and settings.

namespace
Namespace, a formatted quantity of non-volatile memory accessed by a host through a controller.
Host
The system running the operating system and issuing NVMe commands.
NVM
Non-Volatile Memory, memory that retains data without power.

The main ideas

01

Storage and formats

01-01Establish namespace capacity, LBA formats, and the relationship between data and metadata.

metadata
Additional information stored with a logical block; it can contain protection information or serve other purposes.
LBA
Logical Block Address, measured in blocks of the selected format.
02

Command behavior

02-01Compare Read, Write, Compare, Verify, Copy, and space-management commands and their completion conditions.

03

Integrity and ordering

03-01Understand atomicity, command dependencies, Protection Information, and the scope of checks.

Protection Information
PI: protection fields containing a Guard and tags for checking data and its associated information.
04

Capabilities and resources

04-01Use Identify, Features, and logs to understand format selection, performance limits, and advanced resource features.

00.02.The host submits commands through a submission queue, and the controller reports results through a completion queue. The Base specification defines this shared mechanism; this note focuses on how commands act on logical blocks within a namespace.

controller
Controller, the entity that implements the NVMe interface, fetches commands, and reports completions.

Connecting the main ideas

00.03.Follow the life of data: create a namespace and select a format to establish the relationship among LBAs, data, and metadata; then read, write, or move data and determine the guarantees from ordering, atomicity, and protection information.

00.04.The later lessons place individual operations in the whole device: performance hints guide operation sizing, Rate Limiting manages shared resources, FDP/Streams describe placement, and tracking/state structures record changes. These features depend on the earlier format and capability information. The aim is to explain a command’s range, transfer size, requirements, and completion result.

FDP
Flexible Data Placement, a capability connecting data-placement hints with media-reclamation management.

01 Namespace capacity and allocation

01.01.Separate logical address space from allocated resources before considering writes and deallocation. Returned data and allocation state answer different questions.

Addressable, allocatable, and allocated are distinctExample: NSZE=1000, NCAP=800, NUSE=600. The LBA address range is distinct from current allocation.NSZE = 1000LBA 0 … 999NCAP = 800NUSE = 600
Example: NSZE=1000, NCAP=800, NUSE=600. The LBA address range is distinct from current allocation.
NCAP
Namespace Capacity, the maximum number of simultaneously allocated logical blocks.
NSZE
Namespace Size, the total number of addressable logical blocks.
NUSE
Namespace Utilization, the number of currently allocated logical blocks.

01.02.NSZE ≥ NCAP ≥ NUSE: NSZE defines the addressable range, NCAP limits simultaneously allocated blocks, and NUSE counts current allocation. THINP=0 requires NCAP=NSZE. NVMCAP is measured in bytes and is not necessarily NSZE multiplied by data size.

NVMCAP
NVM Capacity, measured in bytes; not directly comparable to NSZE/NCAP logical-block counts.
THINP
Thin Provisioning, the NSFEAT bit governing whether NCAP may be below NSZE and whether the controller tracks NUSE.
Sources: NVM Command Set 1.3 §2.1.1; 4.1.5.1

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.1; 4.1.5.1, printed pages 13-14,85-93, PDF pages 13-14,85-93

Namespace capacity and allocation
Capacity or operationRange or change describedWhat affects the interpretation
NSZEValid LBAs run from 0 to NSZE−1Out-of-range differs from capacity exhaustion
THINPRequires NUSE tracking when supportedWithout support NUSE may remain NCAP
AllocationWrite, the Copy destination, and WU allocateRead/Verify do not change deallocation state

02 Metadata transfer and PI placement

02.01.Metadata need not consist entirely of PI. Mark data, non-PI metadata, and PI separately before computing host-buffer size and CRC coverage.

CRC
Cyclic Redundancy Check: a check value computed from data bits to detect changes.
PI
Protection Information: Guard and tag fields used to check data and its associated information.
Where data and metadata travelData and metadata must correspond to the same logical blocks, whether transferred together or separately.Data 0MD 0Data 1MD 1DPTR: Data 0, Data 1MPTR: MD 0, MD 1
Data and metadata must correspond to the same logical blocks, whether transferred together or separately.
DPTR
Command data pointer: destination for Read, source for Write, descriptors for Copy/DSM.
MPTR
Separate-metadata pointer; namespace format and command fields determine metadata placement.

02.02.Formatting selects one metadata transfer mechanism per namespace: contiguous extended LBAs or a separate buffer addressed by MPTR. Metadata cannot be split between the mechanisms and must be written atomically with its associated logical block.

logical block
An addressable unit in a namespace; its data size is determined by the active format.
Sources: NVM Command Set 1.3 §2.1.6; 5.2.3; 5.3

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.6; 5.2.3; 5.3, printed pages 22,129-131, PDF pages 22,129-131

Metadata transfer and PI placement
Transfer arrangement or locationHost buffer arrangementFormat constraints
Extended LBADPTR addresses interleaved data+metadataMSET/MTELBA reflect the selection
Separate bufferDPTR addresses data; MPTR addresses metadataPRP metadata is physically contiguous; SGL metadata may be scattered
PI locationValid current formats put PI at the metadata endCRC covers preceding non-PI metadata
PRP
Physical Region Page, a pointer format describing a host-addressable data buffer in memory-page units.
SGL
Scatter Gather List, a descriptor-and-segment format for one or more data-buffer regions.

03 Read/Write data and completion

03.01.Check range, buffer, PI, and completion separately to distinguish addressing errors, format mismatches, and media failures.

03.02.Read/Write specify a contiguous range with SLBA and zero-based NLB. DPTR is a destination buffer for Read and a source buffer for Write. FUA=1 requires nonvolatile-media handling without implying ordering of other commands.

zero-based
zero-based: numbering starts at zero, so raw=3 can mean the fourth item or four units; the field definition still decides which.
SLBA
Starting LBA; Starts the command range.
FUA
Force Unit Access; requires nonvolatile-media semantics without automatically ordering other commands.
NLB
Number of Logical Blocks; this field in the report’s commands/status descriptors is zero-based. DSM LLB is separately one-based.
Sources: NVM Command Set 1.3 §3.3.4; 3.3.6

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.4; 3.3.6, printed pages 48-51,53-56, PDF pages 48-51,53-56

Read/Write data and completion
Command fieldData or operation describedSelections affecting interpretation
CDW10 / CDW11Low/high 32 bits of SLBANLB=0 still requests one block
CDW12LR, FUA, PRINFO, STC, CETYPE, NLBThe DTYPE area is reserved for Read
CDW13CETYPE selects DSM or CEV interpretationWrite also carries DTYPE/DSPEC
MPTRUsed for separate metadataDo not split metadata between both mechanisms
CETYPE
Command Extension Type; Selects the interpretation of the command extension value CEV.
PRINFO
Protection Information; The command field combining PRACT and PRCHK.
DSPEC
Directive Specific; Its contents depend on the Directive type.
DTYPE
Directive Type; Selects the Directive type for the command.
CDW
CDW (Command Dword): a 32-bit command field. In CDW10, 10 is the field index, not a byte offset.
CEV
Command Extension Value; Its contents are interpreted according to CETYPE.
DSM
Dataset Management: host hints about use and allocation of data ranges.
STC
Storage Tag Check, separate from three-bit PRCHK and ignored when STS=0.
LR
Limited Retry; selects retry behavior governed by Error Recovery policy.

04 Normal, power-fail, and multiple atomicity

04.01.Read size, starting alignment, NSABP, and MAM together. Atomicity and persistence on nonvolatile media are separate checks; FUA/Flush do not establish ordering for other commands.

NSABP
Namespace Atomic Boundary Parameters: indicates applicability of namespace atomic-write parameters.
MAM
Multiple Atomicity Mode; a crossing command is divided into independently atomic subranges.

04.02.AWUN/NAWUN and AWUPF/NAWUPF describe normal and failure-condition atomicity. Single Atomicity Mode provides no whole-command guarantee across a boundary; Multiple Atomicity Mode divides the range at boundaries into separately atomic subranges, without promising one combined outcome.

NAWUPF
Namespace Atomic Write Unit Power Fail; Namespace failure-condition atomic-write size, subject to applicability and zero-value rules.
AWUPF
Atomic Write Unit Power Fail, the zero-based atomic size for failure conditions.
NAWUN
Namespace Atomic Write Unit Normal; Namespace normal atomic-write size, subject to applicability and zero-value rules.
AWUN
Atomic Write Unit Normal, the controller’s zero-based normal atomic-write size.
Sources: NVM Command Set 1.3 §2.1.4; 4.1.3.4; 5.9

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.4; 4.1.3.4; 5.9, printed pages 15-21,66-67,165, PDF pages 15-21,66-67,165

Normal, power-fail, and multiple atomicity
Atomicity field or modeRange governedWhat it does not establish
AWUN / AWUPFSizes use zero-based encodingAWUPF does not exceed AWUN
NABO / NABSN / NABSPFBoundaries occur at offset + k × sizeRead each field and its unreported cases
MAMEach atomic subrange has its own guaranteeFused operations still use Single mode
FID 0Ah.DNDN=1 releases normal-atomicity requirementsPower-fail guarantees remain
NABSPF
Namespace Atomic Boundary Size Power Fail; The failure-condition atomic boundary size.
offset
offset: a displacement measured from a stated start. It answers “how far from the start,” unlike an index.
NABSN
Namespace Atomic Boundary Size Normal; The normal-operation atomic boundary size.
NABO
Namespace Atomic Boundary Offset; determines the first boundary location.
FID
Feature Identifier: selects the Feature to read or configure.
DN
Disable Normal in Write Atomicity Normal; does not remove power-fail atomicity.

05 Compare and Verify answer different questions

05.01.Expected-content comparison, integrity verification, and ordinary Read provide different evidence. Compare excludes PI from metadata comparison and checks PI separately as requested.

05.02.Compare compares media data with a host-supplied buffer. Verify checks stored-data integrity without returning data or metadata to the host. Both require PRACT=0; Verify and Read need not report an identical error code for a detected failure.

Sources: NVM Command Set 1.3 §3.3.1; 3.3.5

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.1; 3.3.5, printed pages 27-30,51-53, PDF pages 27-30,51-53

Compare and Verify answer different questions
Command or size fieldWhat is checkedMeaning of the result
CompareA miscompare returns Compare FailurePI can be checked on both host and media paths
VerifyNo data-buffer transferVerified data still counts toward Data Units Read
VSL / NVMVFYSThe variant selects recommended size or a hard limitNonzero VSL uses 2^n × minimum page size
VSL
Verify Size Limit; A Verify size limit, interpreted with its variant capability.

06 Copy: source descriptors, contiguous destination, partial failure

06.01.Calculate the expanded destination range before checking formats, limits, overlap, and atomicity. Copy reduces host data movement but is not an unconditional transaction.

06.02.Copy concatenates one or more source ranges in descriptor order into one contiguous destination range. Formats 0h/1h use one namespace; 2h/3h carry SNSID and require controller support and host enablement. Failure CQE DW0 is the lowest unsuccessful source index; later ranges may already have been copied.

SNSID
Source Namespace Identifier; Selects a Copy source namespace.
index
index: selects an item or format in a list. It answers “which one,” not “how far from the start.”
CQE
Completion Queue Entry, one completion-result structure in a CQ.
Sources: NVM Command Set 1.3 §3.3.2

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.2, printed pages 30-44, PDF pages 30-44

Copy: source descriptors, contiguous destination, partial failure
Descriptor or behaviorData arrangementLimits and completion meaning
NR / NLBSource count and per-range block count are zero-basedCheck MSRC, MSSRL, and MCL
FCOFormats 2h/3h can request fast copy onlyInspect DNR after Fast Copy Not Possible
OverlapFormats 2h/3h prohibit source/destination overlap within one namespaceFormats 0h/1h need their separate overlap rules
NVMCSARevision 1.3 treats the destination as one write commandMAM, size, and boundary limits still apply
DNR
Do Not Retry, a CQE-status bit indicating that retrying the same command is not expected to succeed.
FCO
Fast Copy Only; requests a fast-copy method for the applicable source.
NR
Zero-based range-count field; actual descriptor count is NR+1.

07 Combining PRACT with PRCHK/STC

07.01.First determine whether namespace PI is enabled, then choose a processing branch by command direction and metadata size. Evaluate checking bits and special disable values afterward.

07.02.PRACT controls PI transfer, insertion, stripping, or replacement. PRCHK Guard/Application/Reference bits and independent STC control checking. With PRACT=1 and MS>PI size, Read still returns all metadata; PRACT=1 does not universally mean stripping PI.

Guard
The PI check-value field; the selected format determines its width and calculation.
PRCHK
Protection Information Check bits for Guard, Application, and Reference.
Sources: NVM Command Set 1.3 §2.1.5; 5.3.2-5.3.3

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.5; 5.3.2-5.3.3, printed pages 21-22,141-152, PDF pages 21-22,141-152

Combining PRACT with PRCHK/STC
Operation or protection conditionController PI handlingEffect on checking or transfer size
Write, PRACT=1Insert PI when MS=PI; replace it when MS>PIThis generation branch ignores PRCHK/STC
Read, PRACT=1Perform requested checks; strip only when MS=PIMS>PI still returns metadata including PI
Type 1 / Type 2Reference increments per blockType1 initial value must match relevant low SLBA bits
Type 3Should not compare a computed referenceAn RTCHK rejection uses Invalid Protection Information
Disable sentinelsType 1/2 disable all PI checks when Application Tag=FFFFh; Type 3 additionally requires an all-ones Reference Tag, if definedOverrides PRCHK/STC settings
MasksA zero mask bit excludes comparisonStorage masking also obeys STMLA
RTCHK
Reference Tag Check; Requests Reference Tag checking.

08 Hard/Soft modes and token-bucket examples

08.01.Interpret results using capability, configured limits, and actual demand. A configured ratio does not force a fixed throughput ratio at every instant; internal resources and workload still matter.

08.02.Hard Limit sets a ceiling; Soft Limit can use unused bandwidth/IOPS. Resource shortfalls are shared proportionally to configured limits. Appendix A illustrates multiple token buckets without requiring one identical controller implementation.

IOPS
Input/Output Operations Per Second; an operation rate, distinct from byte throughput.
Sources: NVM Command Set 1.3 §5.10.1-5.10.2; Appendix A

Source: NVME-NVM-CS-1.3, Rev. 1.3, §5.10.1-5.10.2; Appendix A, printed pages 166-168,176-177, PDF pages 166-168,176-177

Hard/Soft modes and token-bucket examples
Mode or commandSharing or consumption of creditsLimits on interpretation
HardProportional sharing under demand and resource shortageA configured ceiling is not a performance floor
SoftMay consume unused capacityMultiple soft targets share it in proportion to limits
Write tokensTotal bytes × WRBWR; write bytes; total IOPS × WRIOPSR; one write IOPSCheck each of four buckets
Read tokensTotal bytes and one total IOPSDoes not consume write-only buckets
WRIOPSR
Write-to-Read IOPS Ratio; The write-to-read operation-count weight.
WRBWR
Write-to-Read Bandwidth Ratio; The write-to-read bandwidth weight.

Where to continue in the specification

09.01.Use the flow above to frame the problem, then open the corresponding sections for fields and full conditions. The Chinese tutorial also explains every in-scope figure with its takeaway, example, and details.

Concept to explainSpecification sections
Logical blocks, formats, and unitsNVM 1.3 §1.1-1.6; 4.1.3.9; 4.1.4.8; 4.1.5
Namespace capacity and allocationNVM 1.3 §2.1.1; 4.1.5.1
Identify: multiple structures for one namespaceNVM 1.3 §4.1.5
LBAF, ELBAF, and unique-attribute formatsNVM 1.3 §4.1.5.1; 4.1.5.3; 5.6
Format, Host Behavior, and extended LBAsNVM 1.3 §4.1.2; 4.1.3.7
Namespace creation: format, mask, granularityNVM 1.3 §4.1.6; 4.1.5.8; 5.8
Metadata transfer and PI placementNVM 1.3 §2.1.6; 5.2.3; 5.3
Capabilities, opcodes, and statusNVM 1.3 §2.2; 3.1; 3.3
Read/Write data and completionNVM 1.3 §3.3.4; 3.3.6
Ordering and Compare-and-WriteNVM 1.3 §2.1.2-2.1.3
Normal, power-fail, and multiple atomicityNVM 1.3 §2.1.4; 4.1.3.4; 5.9
Compare and Verify answer different questionsNVM 1.3 §3.3.1; 3.3.5
Copy: source descriptors, contiguous destination, partial failureNVM 1.3 §3.3.2
Copy PI compatibility and transformationNVM 1.3 §3.3.2.3-3.3.2.4; 5.3.2.5
Dataset Management and three processing limitsNVM 1.3 §3.3.3
Reading deallocated/unwritten blocksNVM 1.3 §3.3.3.2.1; 4.1.3.3
Write Uncorrectable, Write Zeroes, and whole-namespace zeroingNVM 1.3 §3.3.7-3.3.8
16/32/64b Guard and qualified PINVM 1.3 §5.3.1; 4.1.5.3
CRC parameters, bit order, and known vectorsNVM 1.3 §5.3.1.1-5.3.1.3
Packing Storage/Reference Tags into DwordsNVM 1.3 §5.3.1.4
Combining PRACT with PRCHK/STCNVM 1.3 §2.1.5; 5.3.2-5.3.3
Basic Feature scopes and exceptionsNVM 1.3 §4.1.3.1-4.1.3.4
NVM extensions to AER, SMART, and error recordsNVM 1.3 §1.4.2; 4.1.1; 4.1.3.5; 4.1.4.1-4.1.4.4
LBA Status: notices, scans, and recoveryNVM 1.3 §4.1.3.6; 4.1.4.5; 4.2.1; 5.2.1
NVM rules for Sanitize and Media VerificationNVM 1.3 §4.1.7; 5.12
Alignment, granularity, and performance hintsNVM 1.3 §5.2.2
Performance Characteristics attributesNVM 1.3 §4.1.3.8
Rate Limiting configuration fieldsNVM 1.3 §4.1.3.9; 4.1.5.4; 5.10
Hard/Soft modes and token-bucket examplesNVM 1.3 §5.10.1-5.10.2; Appendix A
The Rate Limiting log is a capability graphNVM 1.3 §4.1.4.8; 5.10.3
FDP: placement, RUHs, and observable dataNVM 1.3 §3.2.1; 4.1.4.6-4.1.4.7; 4.1.6.3
NVM units and priorities for StreamsNVM 1.3 §5.2.2.3; 5.13
NVM behavior under ANA and ReservationsNVM 1.3 §5.1; 5.11
NVM alignment constraints for Key Per I/ONVM 1.3 §5.5; 4.1.5
LBA Migration Queue and change trackingNVM 1.3 §4.1.8; 5.7
Memory-based resource export templateNVM 1.3 §5.4.1-5.4.1.1
Exported-state length and consistencyNVM 1.3 §5.4.1.2
ELBAF
Extended LBA Format: pairs with LBAF at the same index and adds PI format and Storage Tag size.
LBAF
LBA Format: a description of a logical-block format, including data and metadata sizes.
AER
Advanced Error Reporting, the PCIe capability for classifying, masking, and logging link or transaction errors.
ANA
Asymmetric Namespace Access: the state of access to a namespace through different controllers.
I/O
Input/Output, the class of data operations performed on a namespace.
Open the complete Chinese tutorial and figure explanations →

Check your understanding

1. With NSZE=1000, NCAP=800, and NUSE=600, what question does each value answer?

10.01.NSZE gives the addressable range, LBA 0 through 999; NCAP allows up to 800 allocated logical blocks; NUSE reports 600 currently allocated blocks. These are logical-block counts, so conversion to bytes also needs the data size.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.1; 4.1.5.1, printed pages 13-14,85-93, PDF pages 13-14,85-93

2. Does placing Compare and Write next to each other in an SQ prevent intervening modifications?

10.02.Adjacency alone is insufficient. Use a supported fused operation, mark its first and second commands correctly, and satisfy pairing conditions such as matching ranges. Ordinary submission order does not provide that guarantee.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.2-2.1.3, printed pages 14-15, PDF pages 14-15

3. Does satisfying the atomic-write size guarantee that the data survives a power loss?

10.03.Atomicity concerns whether a partial update can be observed in the specified circumstances; persistence concerns storage on nonvolatile media. Normal versus power-fail atomicity, boundaries, volatile write cache, and FUA/Flush rules still need separate consideration.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.4; 4.1.3.4; 5.9, printed pages 15-21,66-67,165, PDF pages 15-21,66-67,165

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.4; 3.3.6, printed pages 48-51,53-56, PDF pages 48-51,53-56

4. Given expected data, should Compare or Verify be used to check that stored content matches it?

10.04.Compare uses comparison data supplied by the host. Verify checks readability and applicable integrity conditions for the range without returning its data or comparing it against a host-supplied expected payload.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.1; 3.3.5, printed pages 27-30,51-53, PDF pages 27-30,51-53

5. Can an all-zero Read establish that the LBAs remain allocated?

10.05.No. Written zero data and zero-valued reads from deallocated blocks can produce the same result. Interpret the read using allocation state, DRB, and whether DULBE is supported and enabled.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.3.2.1; 4.1.3.3, printed pages 47-48,66, PDF pages 47-48,66

Source: NVME-NVM-CS-1.3, Rev. 1.3, §3.3.7-3.3.8, printed pages 56-61, PDF pages 56-61

6. Is a data size of 4096 bytes enough to construct the correct I/O buffer?

10.06.No. The active Format Index, metadata size, PI format, and metadata transfer mechanism are also needed. A separate buffer and extended LBAs use different memory layouts.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §1.1-1.6; 4.1.3.9; 4.1.4.8; 4.1.5, printed pages 9-12,73-75,79-83, PDF pages 9-12,73-75,79-83

Source: NVME-NVM-CS-1.3, Rev. 1.3, §4.1.5.1; 4.1.5.3; 5.6, printed pages 85-94,96-102,160-162, PDF pages 85-94,96-102,160-162

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.6; 5.2.3; 5.3, printed pages 22,129-131, PDF pages 22,129-131

7. Does PRACT=1 turn off every PI check?

10.07.PRACT controls how PI is handled during transfer; it is not a master check switch. PRCHK specifies Guard, Application Tag, and Reference Tag checks, while STC controls Storage Tag checking. Transferred contents also depend on metadata and PI sizes.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §2.1.5; 5.3.2-5.3.3, printed pages 21-22,141-152, PDF pages 21-22,141-152

8. Why can throughput under a Soft Limit exceed its configured value?

10.08.A Soft Limit permits use of bandwidth or IOPS left unused by other work; resource shortfalls are distributed proportionally to configured limits. A Hard Limit supplies a ceiling. Appendix A token buckets are an illustrative implementation, not a required controller architecture.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §5.10.1-5.10.2; Appendix A, printed pages 166-168,176-177, PDF pages 166-168,176-177

Specification editions

NVM Express NVM Command Set Specification, Revision 1.3

NVM Express Base Specification, Revision 2.4

Jia-Chang

Jia-Chang

Human

Comments

  Write a comment ...