NVMe Directives and Streams: Host Grouping, Writes, and Resources

26 minute read

繁體中文

00.01.This report explains how a host tells an SSD which data belongs together. Establish the relationships among host, controller, namespace, and stream, then follow capability checks, enablement, allocation, writes, and release. Finally examine identifier sharing across paths and the meaning of SWS/SGS for data organization.

controller
Controller, the entity that implements the NVMe interface, fetches commands, and reports completions.
namespace
Namespace, a formatted quantity of non-volatile memory accessed by a host through a controller.
Host
The system running the operating system and issuing NVMe commands.
SGS
Stream Granularity Size; allocation granularity measured in SWS units.
SWS
Stream Write Size; the optimal write alignment/size unit, measured in logical blocks for NVM.

The main ideas

01

Why group data?

01-01A Stream Identifier adds a relationship between writes while LBA still selects the destination. A stream is not another namespace.

Stream Identifier
A host-selected 16-bit grouping label interpreted with namespace, host identity, and sharing configuration.
02

Management and data flow

02-01Capability queries, identity registration, enablement, and allocation prepare the system; tagged Writes actually use streams. Management commands are not repeated for every payload.

payload
Data carried by a command transfer, distinct from command fields and completion values.
03

Identity and resource sharing

03-01Host Identifier and SSID determine which paths share a stream. NSA, NSSA, and NSO distinguish capacity, the resource pool, and current use.

Host Identifier
The identity used to associate controllers with a host, distinct from a Stream Identifier.
NSSA
NVM Subsystem Streams Available; the non-exclusively allocated pool size, not its idle-resource count.
SSID
Shared Stream Identifiers; controls sharing of equal identifiers across nonzero host identities within a namespace.
NSA
Namespace Streams Allocated; the exclusive allocation for the namespace and visible host association, also returned by Allocate.
NSO
Namespace Streams Open; the open count for the selected namespace and visible host association.
04

Size and lifetime

04-01SWS/SGS guide writes and deallocation. Release, disablement, and reset affect tracking state independently of the continued existence of user data.

00.02.The assumed background is operating systems, computer organization, and basic SSD concepts. Examples use PCIe controllers and cover the specified Directives/Streams scope, excluding Data Placement §8.1.9.4. Counts, LBAs, and host labels are illustrative; actual capabilities come from the device.

Directives
A mechanism for exchanging additional host/controller information, with type-specific management and I/O semantics.
Streams
A Directive that groups related writes using stream identifiers without selecting physical NAND addresses.
PCIe
PCI Express, the transport and device interconnect used by an NVMe memory-based controller.

Connecting the main ideas

00.03.Follow a complete data-use cycle: decide which data is related, verify support, establish host identities and enablement, and choose a resource source. Writes create stream activity; release ends an identifier’s current grouping when it is no longer needed.

00.04.Equal numbers need not mean equal things: the same Stream Identifier may belong to different hosts, and SWS=8 versus NSA=8 means data size versus resource count. The examples hold other conditions fixed while changing one factor.

The live specification route

R-1Use the examples to establish the overall picture, then open Base at PDF page 227. Proceed forward through the viewer pages below and switch once to NVM. On shared pages, stop at the named heading; use viewer pages rather than stale contents-page numbers.

NVM
Non-Volatile Memory, memory that retains data without power.
Page sequenceSectionsWhat to explain here
R1 · Base PDF 227–228§5.2.7 → §5.2.8Compare DPTR, NUMD, and DTYPE/DOPER for Receive/Send; stop before §5.2.9 Firmware Commit.
R2 · Base PDF 534–536§5.2.30.1.35 → .35.1Read EXHID, HOSTID, zero/nonzero identity, and non-saveability under PCIe; stop before .35.2.
R3 · Base PDF 642–646§8.1.9 → §8.1.9.2Start at Directives. Use 702/703 for types, then 704–707 for queries and enablement.
R4 · Base PDF 646–653§8.1.9.3708 sizes → 709 operations → 710 sharing → 711–715 parameters/status/allocation → the two releases. Stop before §8.1.9.4.
R5 · NVM PDF 175§5.13Confirm SWS is in logical blocks and finish the 4 KiB, SWS=8, SGS=4 calculation.
HOSTID
The actual identity value in the Host Identifier data structure.
DOPER
Directive Operation; selects an operation within a directive type and command direction.
DTYPE
Directive Type; selects a directive. Enable uses separate outer-operation and inner-target type fields.
EXHID
Enable Extended Host Identifier; zero selects 64 bits and one selects 128 bits.
DPTR
Data Pointer, the SQE field identifying a command data buffer.
NUMD
Number of Dwords; for buffered transfers, encodes the count of four-byte units minus one.

01 What Streams communicates

From preparation to data use
  1. Check Directives/Streams support and register an appropriate nonzero Host Identifier.
  2. Enable Streams through the Identify Directive and inspect state and parameters.
  3. Use exclusive or subsystem resources; request and inspect a grant if exclusive capacity is needed.
  4. Issue Writes with Stream Identifiers; first use of an unopened identifier opens a stream.
  5. Observe open identifiers with Get Status; release an identifier to end its grouping or release resources to return the exclusive allocation.
Allocate Resources selects exclusive capacity; prior exclusive allocation is not required for every Streams user. Source: Base §§8.1.9.2–8.1.9.3.

01.01.Directives exchange additional information between the host and controller. Streams uses a host-selected Stream Identifier to associate writes with a group. The controller may use that grouping for placement or performance improvements; the identifier does not select a physical NAND address.

Sources: Base 2.4 §8.1.9, 8.1.9.1, 8.1.9.3

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9, 8.1.9.1, 8.1.9.3, printed pages 616-623, PDF pages 642-649

02 Identify the host across controller paths

02.01.FID 81h registers a Host Identifier so controllers in an NVM subsystem can recognize a common host. Equal nonzero values establish that identity; 0h does not associate a controller with other controllers. The PCIe default is 0h and this Feature is not saveable.

NVM subsystem
NVM subsystem, the NVMe system boundary containing controllers, ports, namespaces, and non-volatile storage resources.
FID
Feature Identifier: selects the Feature to read or configure.
Sources: Base 2.4 §5.2.30.1.35, 5.2.30.1.35.1

Source: NVME-BASE-2.4, Rev. 2.4, §5.2.30.1.35, 5.2.30.1.35.1, printed pages 508-510, PDF pages 534-536

03 Check support and enable Streams

Enable: outer operation and inner targetOuter DTYPE=0 selects Identify; inner DTYPE=1 targets Streams. Box widths aid reading; the bit labels define actual widths. Source: Base Figures 186/706.CDW1100000001hDTYPE[15:8] = 0IdentifyDOPER[7:0] = 1Enable DirectiveCDW1200000101hDTYPE[15:8] = 1StreamsENDIR[0] = 1Enable
Outer DTYPE=0 selects Identify; inner DTYPE=1 targets Streams. Box widths aid reading; the bit labels define actual widths. Source: Base Figures 186/706.
ENDIR
Enable Directive; one enables and zero disables the type selected in CDW12.
CDW
CDW (Command Dword): a 32-bit command field. In CDW10, 10 is the field index, not a byte offset.

03.01.First check OACS.DIRS, then query Streams support and enablement through the Identify Directive. For Enable Directive, CDW11 selects the Identify operation and CDW12 selects Streams as the target. Namespace and Host Identifier determine the shared enable state.

DIRS
Directives Supported; the general capability bit in OACS.
Sources: Base 2.4 §8.1.9, 8.1.9.2, 8.1.9.3

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9, 8.1.9.2, 8.1.9.3, printed pages 616-624, PDF pages 642-650

04 Read the Directive command envelope

04.01.Directive Receive transfers data toward the host and Directive Send toward the controller, when the selected operation transfers data. CDW11 selects DTYPE/DOPER and type-specific DSPEC. For buffer transfers, NUMD encodes the Dword count minus one.

buffer
A memory buffer prepared by the host to supply or receive data.
DSPEC
Directive Specific; interpreted by type/operation and used as the Stream Identifier in Streams I/O.
Dword
Dword (double word): 32 bits, or 4 bytes. A word is 16 bits; for example, a zero-based dword count of 3 represents 4 Dwords, or 16 bytes.
Sources: Base 2.4 §5.2.7, 5.2.8

Source: NVME-BASE-2.4, Rev. 2.4, §5.2.7, 5.2.8, printed pages 201-202, PDF pages 227-228

05 Separate allocated capacity from open streams

Eight resources: exclusive capacity and the shared poolIllustrative snapshot with no other exclusive allocation: MSL=8, NSA=3 with NSO=1, and NSSA=5 with NSSO=2. Open counts differ from pool capacities. Source: Base Figure 712.MSL = 8NSA = 3Namespace-exclusiveNSSA = 5Nonexclusive subsystem poolNSO = 1One stream openNSSO = 2Two open in the pool
Illustrative snapshot with no other exclusive allocation: MSL=8, NSA=3 with NSO=1, and NSSA=5 with NSSO=2. Open counts differ from pool capacities. Source: Base Figure 712.
NSSO
NVM Subsystem Streams Open; open streams using the nonexclusive pool.
MSL
Max Streams Limit; the subsystem-wide concurrent-open limit.

05.01.Streams uses either namespace-exclusive resources or subsystem resources not exclusively allocated. Allocate Resources requests concurrent tracking capacity; returned NSA may be smaller than NSR. A stream is opened when a later Write uses an identifier that is not already open.

NSR
Namespace Streams Requested; the directly encoded exclusive resource request count.
Sources: Base 2.4 §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.3

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.3, printed pages 621-626, PDF pages 647-652

06 When equal Stream Identifiers mean the same stream

Same namespace, same identifier 7
Controller pathNonzero Host IdentifierStream group: SSID=0Stream group: SSID=1
C1AA / 77
C2AA / 77
C3BB / 77
C4CC / 77
A / 7 is a table label for identity and identifier, not a command encoding. SSID=0 gives three groups; SSID=1 gives one. All identities here are nonzero. Adapted relationship from Base Figure 710, illustrated with identifier 7.

06.01.Interpret a Stream Identifier together with namespace, Host Identifier, and NSSC.SSID. With SSID=0, equal identifiers belonging to different nonzero host identities are separate streams. With SSID=1, those hosts may share the same numbered stream in the same namespace.

NSSC
NVM Subsystem Stream Capability; contains identity requirements and sharing capabilities.
Sources: Base 2.4 §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.2

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.2, printed pages 621-626, PDF pages 647-652

07 Attach a Write to a stream

07.01.A Streams Write uses DTYPE=1 and a nonzero Stream Identifier in DSPEC. First use of an unopened identifier opens a stream. DTYPE=0 or a zero Streams DSPEC omits stream grouping; handling an unenabled type also depends on whether any I/O Directive is enabled.

I/O
Input/Output, the class of data operations performed on a namespace.
Sources: Base 2.4 §8.1.9.1, 8.1.9.3

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.1, 8.1.9.3, printed pages 617-623, PDF pages 643-649

08 Convert SWS and SGS into data sizes

Multiply SGS by SWS, then by block sizeFour 32 KiB SWS units form one 128 KiB granularity unit. This is size composition, not the execution order of four commands. Source: Base Figures 708/712 and NVM §5.13.Each logical block = 4096 bytesSWS = 8 blocks32 KiBSWS = 8 blocks32 KiBSWS = 8 blocks32 KiBSWS = 8 blocks32 KiBSGS = 4 SWS = 32 blocks = 128 KiB
Four 32 KiB SWS units form one 128 KiB granularity unit. This is size composition, not the execution order of four commands. Source: Base Figures 708/712 and NVM §5.13.
logical block
An addressable unit in a namespace; its data size is determined by the active format.
granularity
Granularity; a size unit used for writes or allocation, not a resource count.

08.01.For the NVM Command Set, SWS is measured in logical blocks; SGS is measured in SWS units. Convert through the namespace logical block size before comparing write alignment, transfer length, and group-sized deallocation ranges.

alignment
Alignment; the start position is a multiple of the required unit, checked separately from transfer length.
Sources: NVM Command Set 1.3 §5.13

Source: NVME-NVM-CS-1.3, Rev. 1.3, §5.13, printed pages 175, PDF pages 175

09 Observe streams and end their use

Ending identifiers versus returning resources
Event or operationIdentifier/open stateExclusive allocationUser data
Release IdentifierEnds this use of one identifierRetained for namespace reuseDoes not delete data
Release ResourcesNSA=0 alone is not a complete identifier-state reportReturned; NSA becomes zeroDoes not delete data
Disable StreamsReleases affected host identifiersReleases affected resourcesNot secure erasure
Format NVMReleases affected open identifiersDo not infer release of all capacity from this ruleData effects follow Format
Delete / become write-protectedReleases namespace stream identifiersReleases all stream resourcesTracking release is not media erasure
Reset additionally depends on enabled controllers associated with the same host; it is not interchangeable with explicit disablement. Source: Base §§8.1.9.2.2.1, 8.1.9.3, and 8.1.9.3.2.

09.01.Get Status reports open identifiers. Release Identifier ends the current association of one identifier while exclusive capacity can remain allocated; Release Resources returns the namespace-exclusive allocation. Disablement, Format, deletion, write protection, and reset have distinct effects.

Sources: Base 2.4 §8.1.9.2.2.1, 8.1.9.3, 8.1.9.3.1.2, 8.1.9.3.2

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.2.2.1, 8.1.9.3, 8.1.9.3.1.2, 8.1.9.3.2, printed pages 619-627, PDF pages 645-653

Where to continue in the specification

10.01.Use the flow above to frame the problem, then open the corresponding sections for fields and full conditions. The Chinese tutorial also explains every in-scope figure with its takeaway, example, and details.

Concept to explainSpecification sections
What Streams communicatesBase 2.4 §8.1.9, 8.1.9.1, 8.1.9.3
Identify the host across controller pathsBase 2.4 §5.2.30.1.35, 5.2.30.1.35.1
Check support and enable StreamsBase 2.4 §8.1.9, 8.1.9.2, 8.1.9.3
Read the Directive command envelopeBase 2.4 §5.2.7, 5.2.8
Separate allocated capacity from open streamsBase 2.4 §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.3
When equal Stream Identifiers mean the same streamBase 2.4 §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.2
Attach a Write to a streamBase 2.4 §8.1.9.1, 8.1.9.3
Convert SWS and SGS into data sizesNVM 1.3 §5.13
Observe streams and end their useBase 2.4 §8.1.9.2.2.1, 8.1.9.3, 8.1.9.3.1.2, 8.1.9.3.2
Open the complete Chinese tutorial and figure explanations →

Check your understanding

1. Are SDIRS=1, NSA=3, and NSO=0 contradictory?

11.01.No. They describe support, exclusive capacity, and the current open count. The host can have enabled and allocated capacity without yet issuing a stream-tagged write.

Sources

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9, 8.1.9.2, 8.1.9.3, printed pages 616-624, PDF pages 642-650

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.3, printed pages 621-626, PDF pages 647-652

2. Do C1 and C2 with Host Identifier=0h share host identity for stream 7?

11.02.No. Zero explicitly does not establish a cross-controller host association. Matching nonzero identities are needed for that common-host relationship.

Sources

Source: NVME-BASE-2.4, Rev. 2.4, §5.2.30.1.35, 5.2.30.1.35.1, printed pages 508-510, PDF pages 534-536

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.2, printed pages 621-626, PDF pages 647-652

3. After receiving three exclusive resources, can another NSR=2 request make five?

11.03.No. A request while that exclusive allocation exists is rejected. Release the allocation, request the full new count, and inspect the grant. Releasing one identifier is not releasing the whole allocation.

Sources

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.3, 8.1.9.3.1.1, 8.1.9.3.1.3, printed pages 621-626, PDF pages 647-652

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.2.2.1, 8.1.9.3, 8.1.9.3.1.2, 8.1.9.3.2, printed pages 619-627, PDF pages 645-653

4. With 4 KiB/block, SWS=8, and SGS=4, which size recommendations are met by a 16-block Write and deallocation starting at LBA 136?

11.04.The Write start and length are multiples of eight and satisfy SWS. Stream granularity is 32 blocks; neither 136 nor 16 is a multiple of 32, so the deallocation optimization recommendation is not met. This does not declare the command invalid.

Sources

Source: NVME-NVM-CS-1.3, Rev. 1.3, §5.13, printed pages 175, PDF pages 175

5. If subsystem Get Status lists identifier 7 once, does that prove only one namespace uses it?

11.05.No. The subsystem query covers nonexclusive resources and deduplicates equal identifiers across namespaces. Query the particular NSID under the appropriate host association for its details.

Sources

Source: NVME-BASE-2.4, Rev. 2.4, §8.1.9.2.2.1, 8.1.9.3, 8.1.9.3.1.2, 8.1.9.3.2, printed pages 619-627, PDF pages 645-653

Specification editions

NVM Express Base Specification, Revision 2.4

NVM Express NVM Command Set Specification, Revision 1.3

Jia-Chang

Jia-Chang

Human

Comments

  Write a comment ...