NVMe Self-Study Bank: Multi-controller and virtualization
read more
NVMe Self-Study Bank: Multi-controller and virtualization
NVMe · Q1–328
NVM Express Base 2.4NVM Command Set 1.3NVMe over PCIe Transport 1.4
Q246–Q255
Multi-controller and virtualization
Separate multipath, shared storage and virtualization before combining topology, attachment, ANA and resources.
Each question is followed by its explanation. Start with the volume overview, then follow the field interpretations, comparisons or processes. Essential conditions and the normal path are visible from the start. All numerical examples are hypothetical. Status is written SCT/SC; h indicates hexadecimal.
- SCT
- Status Code Type
Three state dimensions on one topology
Online, enabled and namespace-accessible are different.
| State | Discovery | Interpretation |
|---|---|---|
| Role | Primary/secondary structures | Resource-management authority |
| Online status | State and resource counts | Online prerequisites |
| Interface | CC/CSTS | Enabled/readiness |
| Namespace path | Active inventory and ANA | Current path availability |
Worked interpretation: Online does not set RDY1; host initialization and enable remain necessary.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §2.4.2, 8.1.1 (PCIe namespace access) · Base 2.4 §5.2.13.1.13
Review the terminology when needed
- Controller / namespace
- A controller receives commands and manages access. A namespace is a logical storage space that commands can address. An NVM subsystem contains controllers and nonvolatile storage resources.
- SQ / CQ / SQE / CQE
- Submission and Completion Queues carry command entries (SQEs) and completion entries (CQEs). QID identifies a queue, CID distinguishes outstanding commands in one SQ, and NSID identifies a namespace.
- Register / Identify / Feature / Log
- A register exposes control or state. Identify queries capabilities and attributes; features query or configure operation; log pages report specific state or records. FID, LID, CNS and CSI select features, logs, Identify structures and command sets.
- index / offset / zero-based
- An index selects an entry, usually starting at 0; an offset measures distance from an origin in specified units. A zero-based count encodes count−1, but not every zero-valued field is a count. A Dword is 4 bytes; a byte is 8 bits.
- Scope / reset / retention
- Scope names the affected objects; retention means preserving state. Controller Reset (clearing CC.EN) is one form of Controller Level Reset, or CLR. Different CLR triggers can retain different registers.
Questions in this volume
- Q246 · How are subsystems, primary and secondary controllers related?
- Q247 · How are private and shared namespaces attached?
- Q248 · How is a namespace’s attached-controller list retrieved?
- Q249 · Can other paths continue during one controller’s reset?
- Q250 · Which controllers are affected by subsystem reset?
- Q251 · How are namespace changes reported through other controllers?
- Q252 · How do asymmetric behavior and ANA guide path selection?
- Q253 · How does a secondary transition Online or Offline?
- Q254 · How are virtualization resources assigned and released?
- Q255 · How are virtualization resources restored after reset and power cycle?
Q246 How are subsystems, primary and secondary controllers related?
Follow the explanation
Controllers are command endpoints, namespaces storage objects; virtualization lets a primary allocate queue/interrupt resources.
1. What differs
A subsystem can contain multiple primary groups, with each secondary in its primary’s domain.
VQ manages an SQ/CQ pair, VI a vector; private resources are fixed and flexible resources reallocatable.
- SQ
- Submission Queue
- CQ
- Completion Queue
2. How to choose and verify
Discover VMS and the primary/secondary structures rather than inferring parentage from function numbers.
Map parentage, resources and attachment before creating queues.
An enabled Online secondary remains a compliant NVMe controller.
3. Conditions that shape the comparison
Resource management requires support and the associated primary endpoint.
PF/VF map to primary/secondary when both are supported; not all multi-controller devices implement SR-IOV.
- PF
- Physical Function
- VF
- Virtual Function
- SR-IOV
- Single Root I/O Virtualization
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q247 How are private and shared namespaces attached?
Follow the explanation
Attachment creates access paths, not copies; private/shared determines simultaneous attachments.
1. Prepare the operation
Private permits one attached controller at a time; shared permits several subject to constraints.
Inspect NMIC, controller capabilities and attachment list.
Attachment uses target NSID, Attach/Detach selector and controller-ID list.
- NSID
- Namespace Identifier
2. Sequence and completion conditions
Confirm allocation/sharing, attach and verify active lists per controller.
Shared paths see the same data; a private namespace cannot acquire a second simultaneous attachment.
3. Handle unmet conditions
A private namespace attached elsewhere differs from an already-attached target.
Cross-check per-controller active lists against the namespace controller list.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q248 How is a namespace’s attached-controller list retrieved?
Follow the explanation
It lists attachment endpoints, not every controller in the subsystem.
1. Select the target and information
Scope is the selected namespace; unattached controllers are not access paths.
Select Attached Controller List with the proper CNS, NSID and starting controller identifier.
- CNS
- Controller or Namespace Structure, selecting Identify data
Parse count/IDs and obey the inclusive starting-controller rule, not a different namespace-list pagination rule.
2. Query sequence and interpretation
Read successive pages using the next ID after the last entry, considering concurrent changes.
Attachment discovery precedes Online/Ready/ANA availability checks.
3. Handle missing or inconsistent evidence
Invalid selectors follow Identify rules; an empty attachment list does not prove deletion.
Cross-check per-controller active lists at consistent times.
First check which controller-list selection was used.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q249 Can other paths continue during one controller’s reset?
Follow the explanation
Shared namespaces provide paths, not immunity to every shared failure.
1. Mechanism and scope
CLR usually targets one controller, but primary resets offline secondaries and shared media/domain faults can broaden impact.
Independent queues can remain valid while namespace/ANA/access restrictions still govern I/O.
2. Understand it through actions and results
Check parentage, domain, attachment, ANA and readiness of alternate paths.
Quiesce the failed path, establish an alternative and resolve old/new command dependencies.
A healthy alternative can access the same data without creating another namespace.
3. Avoid a misleading conclusion
Successful failover does not prove old Writes unexecuted; retries may overlap effects.
Use topology and outcomes rather than assuming universal reset isolation.
Sources: Base 2.4 §2.4.2, 8.1.1 (PCIe namespace access) · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.2, 8.1.1 (PCIe namespace access) · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q250 Which controllers are affected by subsystem reset?
Follow the explanation
Multi-domain implementations may reset one domain despite the subsystem-reset name.
1. Mechanism and scope
Single-domain scope is all; multi-domain scope is one or all domains as implemented.
Supported NSSR triggers reset; affected controllers undergo CLR and their PMRs are disabled.
2. Understand it through actions and results
Inspect topology, host-reset capability and NSSRO evidence.
Quiesce the intended scope, reset and restore each affected path.
Rebuild affected queues; outside paths depend on their own/shared-resource health.
3. Avoid a misleading conclusion
Register-triggered reset has no CQE or DNR/More.
- CQE
- Completion Queue Entry
- DNR
- Do Not Retry, a completion retry indication
Compare all affected controllers with domain configuration.
4. Whether CQE, DNR and More apply here
- CQE
- Completion Queue Entry
- DNR
- Do Not Retry, a completion retry indication
NSSR writes have no CQE/DNR/More; post-recovery query CQEs belong to those queries.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q251 How are namespace changes reported through other controllers?
Follow the explanation
Management on one path can require rediscovery by hosts using other paths.
1. Prepare the operation
Notification follows affected attachment/views rather than unconditional broadcast.
Check NAN, AERs, changed/active lists and separate ANA notice configuration.
The notice prompts rediscovery; changed IDs narrow it, while FFFFFFFFh requires broad rediscovery.
2. Sequence and completion conditions
Process notice, read changes and refresh Identify with acknowledgment/masking accounted for.
Affected views converge with configuration without requiring unlimited one-event-per-change history.
3. Handle unmet conditions
The processing controller has a specific Admin Delete notice exclusion.
Namespace and ANA notices describe different changes and logs.
Sources: Base 2.4 §5.2.13.1.5 · Base 2.4 §5.2.2 (PCIe-applicable events) · Base 2.4 §5.2.13.1.13 · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §5.2.13.1.5 · Base 2.4 §5.2.2 (PCIe-applicable events) · Base 2.4 §5.2.13.1.13 · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q252 How do asymmetric behavior and ANA guide path selection?
Follow the explanation
Access characteristics can differ across controllers, including PCIe paths.
1. Select the target and information
ANA state describes a controller-to-group relationship, not one global namespace state.
Inspect ANA capability, transition time, group membership and LID 0Ch.
- LID
- Log Page Identifier
Optimized/non-optimized support normal commands; inaccessible, persistent loss and change impose distinct restrictions.
2. Query sequence and interpretation
Select accessible paths, prefer optimized access and refresh after notices.
Non-optimized is usable, and another path may report a different state for the same group.
3. Handle missing or inconsistent evidence
Restricted states use their corresponding asymmetric-access status for nonexempt commands, not every Admin command.
Different performance does not prove ANA reporting support; check capability first.
First match group state to the actual controller path.
Sources: Base 2.4 §2.4.2, 8.1.1 (PCIe namespace access) · Base 2.4 §5.2.13.1.13 · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.2, 8.1.1 (PCIe namespace access) · Base 2.4 §5.2.13.1.13 · Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q253 How does a secondary transition Online or Offline?
Follow the explanation
Online permits use but is not the same as enabled/ready.
1. Prepare the operation
The associated primary manages it; Offline sets CFS and leaves other properties undefined.
Inspect secondary state/resource counts and primary enablement.
ACT7 offlines,8 assigns and 9 onlines; offlining removes flexible resources.
2. Sequence and completion conditions
Recommended sequence: Offline, assign, CLR (prefer VF FLR for a VF), Online, then enable and initialize queues.
- VF
- Virtual Function
- FLR
- Function Level Reset
With flexible VQ support, at least two resources are needed; with VI support, vector 0 must be assigned.
3. Handle unmet conditions
Wrong state or insufficient Online prerequisites cause Invalid Secondary Controller State; repeated same-state requests are allowed.
Validate Online and Enable separately; expected Offline CFS is not unexplained hardware failure.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q254 How are virtualization resources assigned and released?
Follow the explanation
Flexible allocation redistributes a finite pool rather than creating resources.
1. Prepare the operation
VQ and VI are separate pools; private resources cannot be reassigned.
Read resource types, totals, assignments, per-secondary maxima and granularity.
CNTLID/RT/ACT select target/type/action, NR requests a count and NRM reports the modified count.
2. Sequence and completion conditions
Assign only while Offline; offlining removes secondary resources. Primary allocation changes need the specified later reset.
NRM may be smaller or larger than NR; verify actual allocation.
3. Handle unmet conditions
Invalid total/per-controller count differs from invalid/private/in-use resource ranges.
Reconcile pool ownership and distinguish preferred granularity from mandatory rejection.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Q255 How are virtualization resources restored after reset and power cycle?
Follow the explanation
Primary persistent allocation and secondary live assignment have different lifetimes.
1. Identify the trigger and affected objects
Include dependent secondaries, not only the reset controller.
Snapshot capabilities, secondary list, reset source and function relationships.
2. State changes and recovery
Primary allocation is persistent and new values take effect after CLR other than CC.EN Controller Reset.
After primary recovery, rediscover and rebuild secondary operation through the required sequence.
Persistent allocation and freshly established secondary resources/queues are validated separately.
3. Checks across reset or power loss
| Trigger | Effect on this operation or state |
|---|---|
| What survives or continues after Controller Reset? | Primary disable/CLR/shutdown takes secondaries Offline, removing their flexible resources on the transition. Primary allocation is persistent but a new value takes effect after a CLR other than CC.EN Controller Reset. |
| What survives or continues after NVM Subsystem Reset? | Rebuild affected primary/secondary operation after subsystem reset; persistent primary allocation does not preserve Online state or queues. |
| What survives or continues after a power cycle? | Primary flexible allocation persists across power cycles; rediscover/reconfigure secondary resources and state. Offline does not delete persistent namespace/attachment configuration. |
4. Verify retention and recovery
Retained primary allocation does not preserve secondary queues or Online state.
Persistent attachment can remain while secondary is Offline; do not recreate the namespace merely for that reason.
First check the primary-reset cascade into secondary Offline/resource removal.
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations for this question
Sources: Base 2.4 §2.4.1–2.4.2 (PCIe topology) · Base 2.4 §8.2.7 · Base 2.4 §5.3.6 · Base 2.4 §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List) · Base 2.4 §5.2.24–5.2.25, 8.1.17–8.1.17.2 · Base 2.4 §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2 · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Source locations and existing figure guides
Base printed page = PDF page−26; the other two use identical numbers. Locations follow the supplied PDF body and retain figure numbers. Shared pages contribute only the relevant definitions, excluding Fabrics and PCIe link/packet content.
- Base 2.4 · §2.4.1–2.4.2 (PCIe topology)
Printed pages 34–37 · PDF 60–63 · Figure 19–22 - Base 2.4 · §2.4.2, 8.1.1 (PCIe namespace access)
Printed pages 37, 578–584 · PDF 63, 604–610 · Figure 675–678 - Base 2.4 · §3.7.1–3.7.4
Printed pages 120–124 · PDF 146–150 - Base 2.4 · §4.2.3
Printed pages 145–155 · PDF 171–181 · Figure 101–105 - Base 2.4 · §5.2.2
Printed pages 183–190 · PDF 209–216 · Figure 150–156 - Base 2.4 · §5.2.2 (PCIe-applicable events)
Printed pages 183–191 · PDF 209–217 · Figure 150–160 - Base 2.4 · §5.2.13.1.2
Printed pages 218–220 · PDF 244–246 · Figure 212 - Base 2.4 · §5.2.13.1.5
Printed pages 226 · PDF 252 - Base 2.4 · §5.2.13.1.13
Printed pages 241–244 · PDF 267–270 · Figure 229–231 - Base 2.4 · §5.2.13.1.14 (header, reset, hardware, Set Feature events)
Printed pages 244–256, 258, 262–264 · PDF 270–282, 284, 288–290 · Figure 232–244, 246, 252–253 - Base 2.4 · §5.2.14.2.2–5.2.14.2.19, 5.2.14.3.1–5.2.14.3.2
Printed pages 387–399, 402–404 · PDF 413–425, 428–430 · Figure 342–355, 360–362 - Base 2.4 · §5.2.14.3.1–5.2.14.3.2 (Primary Capabilities, Secondary List)
Printed pages 402–404 · PDF 428–430 · Figure 360–362 - Base 2.4 · §5.2.24–5.2.25, 8.1.17–8.1.17.2
Printed pages 444–448, 660–663 · PDF 470–474, 686–689 · Figure 442–450 - Base 2.4 · §5.3.6
Printed pages 533–535 · PDF 559–561 · Figure 587–590 - Base 2.4 · §8.2.7
Printed pages 754–758 · PDF 780–784 · Figure 796
When you need a field guide
Existing figure explanations have canonical locations; use these links instead of duplicating the same guide.
- Base 2.4 Figure 101 · Completion Queue Entry: Status Field
- Base 2.4 Figure 104 · Status Code – Command Specific Status Values
Original documents used
- NVM Express Base Specification · Revision 2.4 · 2026-07-31
NVM-Express-Base-Specification-Revision-2.4-Ratified-2026.07.31.pdf - NVM Express NVM Command Set Specification · Revision 1.3 · 2026-07-31
NVM-Express-NVM-Command-Set-Specification-Revision-1.3-Ratified-2026.07.31.pdf - NVM Express NVMe over PCIe Transport Specification · Revision 1.4 · 2026-07-31
NVM-Express-NVMe-over-PCIe-Transport-Specification-Revision-1.4-Ratified-2026.07.31.pdf
Comments