NVMe Self-Study Bank: Interrupt configuration and completion delivery

NVMe · Q1–328

NVM Express Base 2.4NVM Command Set 1.3NVMe over PCIe Transport 1.4

Q286–Q296

Interrupt configuration and completion delivery

Establish CQE posting before diagnosing routing, aggregation, masks and host consumption.

CQE
Completion Queue Entry

Each question is followed by its explanation. Start with the volume overview, then follow the field interpretations, comparisons or processes. Essential conditions and the normal path are visible from the start. All numerical examples are hypothetical. Status is written SCT/SC; h indicates hexadecimal.

SCT
Status Code Type

Four controls answer different questions

FID 09h.CD disables coalescing, not interrupts.

FID
Feature Identifier
ControlInterfaceMeaning
CQ routingIV/IENVector and notification enable
AggregationTIME/THRRecommended delay/count
Per-vector overrideCDDisable aggregation for this vector
MaskingMode-specific masksAllow delivery
CQ
Completion Queue

Worked interpretation: Shared masked CQs can accumulate entries; one later interrupt can cover many completions.

Sources: Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §3.1.4 (INTMS, INTMC) · PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §5.3.1–5.3.2

Review the terminology when needed
Controller / namespace
A controller receives commands and manages access. A namespace is a logical storage space that commands can address. An NVM subsystem contains controllers and nonvolatile storage resources.
SQ / CQ / SQE / CQE
Submission and Completion Queues carry command entries (SQEs) and completion entries (CQEs). QID identifies a queue, CID distinguishes outstanding commands in one SQ, and NSID identifies a namespace.
Register / Identify / Feature / Log
A register exposes control or state. Identify queries capabilities and attributes; features query or configure operation; log pages report specific state or records. FID, LID, CNS and CSI select features, logs, Identify structures and command sets.
index / offset / zero-based
An index selects an entry, usually starting at 0; an offset measures distance from an origin in specified units. A zero-based count encodes count−1, but not every zero-valued field is a count. A Dword is 4 bytes; a byte is 8 bits.
Scope / reset / retention
Scope names the affected objects; retention means preserving state. Controller Reset (clearing CC.EN) is one form of Controller Level Reset, or CLR. Different CLR triggers can retain different registers.

Questions in this volume

  1. Q286 · How does a CQ select an interrupt vector?
  2. Q287 · How do shared and dedicated CQ vectors differ?
  3. Q288 · How do coalescing threshold and time work?
  4. Q289 · How is interrupt coalescing disabled?
  5. Q290 · Does FID 09h mask interrupts, and where are actual masks configured?
  6. Q291 · Can completions be posted while a vector is masked?
  7. Q292 · How are accumulated completions handled after unmasking?
  8. Q293 · Can an in-use vector be reconfigured?
  9. Q294 · How are interrupts restored after reset?
  10. Q295 · How are interrupt resources reclaimed after queue deletion?
  11. Q296 · How are interrupt settings, CQ state and completions cross-checked?

Q286 How does a CQ select an interrupt vector?

CQ
Completion Queue
Follow the explanation

CQ identity selects completion storage, while IV selects notification; the numbers need not match.

1. Prepare the operation

I/O CQs own vector associations; SQs use them through CQID.

CQID
Completion Queue Identifier

Establish interrupt mode and allocated vectors before CQ creation.

IV selects the vector, IEN enables CQ interrupts and PC selects memory contiguity.

2. Sequence and completion conditions

Allocate vectors, create CQ and SQ, then verify with I/O.

SQ
Submission Queue

CQ5 can use IV2; an SQ7 completion goes to CQ5 and is identified by SQID/CID after IV2 notification.

SQID
Submission Queue Identifier
CID
Command Identifier, unique among outstanding commands in one SQ

3. Handle unmet conditions

Invalid vector selection maps to 1/08h, independently of QID validity.

Correlate successful creation, IV/IEN and the host handler’s CQ list.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q287 How do shared and dedicated CQ vectors differ?

Follow the explanation

Sharing saves vectors but requires scanning more CQs; dedicated vectors simplify distribution.

1. What differs

Aggregation thresholds apply per vector and may cover several CQs.

CQEs retain command identity; the interrupt does not enumerate completed commands.

2. How to choose and verify

Inspect available vectors and CQ IV/IEN associations.

Maintain vector-to-CQ membership, consume valid entries and acknowledge each CQ.

One IV3 interrupt can cover multiple completions in two CQs.

3. Conditions that shape the comparison

Interrupt and CQE counts need not match under sharing and aggregation.

CQE
Completion Queue Entry

Measure per-CQ posting and consumption, not only shared-vector counts.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q288 How do coalescing threshold and time work?

Follow the explanation

Aggregation reduces host overhead at possible notification-latency cost without requiring delayed CQE posting.

1. Establish the source and scope

FID 08h applies to I/O queues, not Admin CQ. THR is a per-vector threshold; a controller may apply TIME per vector or across all vectors.

FID
Feature Identifier

Use Get/Set FID 08h for TIME and THR; Get with SEL=011b reports Supported Capabilities instead of those values. Also inspect each vector's FID 09h.CD.

2. Fields, units and worked interpretation

Read Base 2.4 Figure 543: TIME[15:8] uses 100 µs units and THR[7:0] is a zero-based recommended minimum completion count. Then read PCIe Transport 1.4 §3.5 for the implementation latitude.

Establish nonzero settings and CD0 before comparing notification timing under a fixed workload.

Simulated data: TIME=5 and THR=7 recommend 500 µs and eight CQEs per vector. If two CQs sharing one vector receive three and five completions, consider both when analyzing that vector's threshold; notification is not guaranteed exactly at the eighth completion or at 500 µs. For another vector, determine whether this controller applies TIME per vector or across all vectors.

3. Conditions that change the interpretation

PCIe Transport 1.4 requires support for the related Get/Set Features commands, but leaves the aggregation algorithm implementation-specific, including no aggregation. Notification before or after a recommended value alone is not a violation.

CQ-head updates may restart aggregation and ongoing servicing can continually postpone another notification.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q289 How is interrupt coalescing disabled?

Follow the explanation

Disabling aggregation removes coalescing, not interrupts.

1. Prepare the operation

FID 08h controls I/O aggregation globally, while FID 09h.CD disables it per vector.

Read aggregation settings, vector overrides and CQ interrupt enable.

Either zero TIME or zero THR implicitly disables aggregation; CD1 prevents aggregation on that vector.

2. Sequence and completion conditions

Use FID 08h globally, or associate an I/O CQ before setting a per-vector override.

Verify readback and apply delivery/mask rules rather than the previous aggregation threshold.

3. Handle unmet conditions

THR0 has an explicit disable meaning despite ordinary zero-based count interpretation.

No aggregation does not require one separate interrupt per CQE.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q290 Does FID 09h mask interrupts, and where are actual masks configured?

Follow the explanation

Interrupt Vector Configuration (FID 09h) does not mask interrupts. Its CD bit controls whether that vector uses interrupt coalescing; inspect the active interrupt mechanism for the actual mask.

1. Prepare the operation

A mask blocks delivery for all CQs sharing that vector.

Select the masking interface after establishing the active interrupt mode.

INTMS write-one sets and INTMC write-one clears pin/MSI masks; MSI-X uses function and table-vector masks.

MSI
Message Signaled Interrupt
MSI-X
Message Signaled Interrupts Extension

2. Sequence and completion conditions

Preserve state, mask, service/acknowledge CQs, unmask and check remaining work.

INTMC requires writing one to clear a mask; zero has no effect.

3. Handle unmet conditions

INTMS/INTMC access is prohibited and undefined under MSI-X, not a guaranteed NVMe status.

FID 09h readback does not establish MSI-X mask state.

4. Whether CQE, DNR and More apply here

DNR
Do Not Retry, a completion retry indication

Mask register accesses have no NVMe CQE; only separate Get/Set Feature commands carry DNR/More.

DNR
Do Not Retry, a completion retry indication

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q291 Can completions be posted while a vector is masked?

Follow the explanation

Masking controls notification rather than command execution or CQ posting.

1. Masking suppresses notification, not CQ progress

Associated CQs can accumulate completions until their capacity limits intervene.

Normal CQE posting continues; MSI-X masks cause pending-interrupt indication.

A new valid CQE without an interrupt can be normal while masked.

2. Observe completions and avoid a full CQ

Inspect masks, IEN, CQ position/phase and applicable MSI-X pending bits.

Poll and consume valid CQEs, then update head to release space.

CQ fullness is a capacity consequence of nonconsumption, not a direct masking rule.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q292 How are accumulated completions handled after unmasking?

Follow the explanation

One pending interrupt can represent many completions whose results remain in CQs.

1. Prepare the operation

Service all relevant CQs sharing the vector.

Check mode-specific masks/pending state and associated queues.

Pending MSI-X delivery requires both mask layers clear.

2. Sequence and completion conditions

Unmask, process valid CQEs and acknowledge heads while handling arrival races.

Ten accumulated completions need not cause ten replayed interrupts.

3. Handle unmet conditions

INTM masking suppresses MSI pending assertion; MSI-X PBA semantics cannot be copied to it.

Judge consumption completeness, not equality of interrupt and command counts.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q293 Can an in-use vector be reconfigured?

Follow the explanation

Distinguish coalescing changes, masking and reassignment of CQ association.

1. Prepare the operation

FID 09h changes aggregation for the selected vector and its associated CQs.

FID 09h requires prior association with an existing I/O CQ.

CD selects aggregation behavior, not reassignment of Create CQ.IV.

2. Sequence and completion conditions

Change/read back vector settings; recreate the queue association when a different IV is needed.

An existing CQ is the feature’s prerequisite, not a prohibition on change.

3. Handle unmet conditions

Invalid/unassociated IV should produce Invalid Field, preserving the recommendation strength.

Preserve shared-CQ and mask state to isolate feature effects.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q294 How are interrupts restored after reset?

Follow the explanation

Queue associations end with queue lifetime; PCIe mode/mask retention depends on the reset source.

1. Identify the trigger and affected objects

One-controller reset differs from broader reset and does not justify reconfiguring unaffected controllers.

Inspect controller state, interrupt mode and feature current values.

2. State changes and recovery

FID 08h resets to zero; default per-vector coalescing permission does not enable a disabled global aggregation policy.

Recover Admin access, vectors and queues before configuring per-vector/global aggregation.

New completions are consumed through the rebuilt path without importing stale CQEs.

3. Verify retention and recovery

Setting FID 09h before rebuilding its CQ can violate the association prerequisite.

Mode changes may lose aggregation settings; CC.EN testing does not establish power-cycle behavior.

First check reset type and successful new-CQ creation time.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q295 How are interrupt resources reclaimed after queue deletion?

Follow the explanation

CQ deletion removes one completion destination, not the host-allocated PCIe vector itself.

1. Prepare the operation

Other CQs may still use the vector, requiring association-aware reclamation.

Preserve SQ-to-CQ-to-IV dependencies and outstanding commands.

Delete dependent SQs and wait, then delete CQ and retire its host handling state.

2. Sequence and completion conditions

Release the handler/vector only after checking remaining CQ users.

Deleting CQ1 leaves IV3 serving CQ2 when both shared it.

3. Handle unmet conditions

Referenced CQs cannot be deleted successfully, nor their memory reclaimed early.

Correlate deletion completion, host membership and remaining notification paths.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Q296 How are interrupt settings, CQ state and completions cross-checked?

Follow the explanation

Separating completion from notification distinguishes execution failures from delivery/consumption failures.

1. Preserve the evidence first

Follow a complete SQ-command-to-CQ-to-vector-to-handler path.

Inspect routing, aggregation, masks and phase.

Preserve submission, CQE posting, notification and head-update times separately.

2. Work through the possible causes

First inspect the command’s CQ position and phase.

Establish a valid CQE, assess notification, then verify consumption and reclamation.

3. Connect observations to specification conditions

A posted CQE consumed by polling while masked can be valid normal behavior.

Resolve wrong phase/CQ or missing head updates before diagnosing a lost command from absent interrupts.

Correlate settings and observations from the same test interval.

Sources: PCIe Transport 1.4 §3.5–3.5.2 (interrupt delivery and masks), 3.8.4 · Base 2.4 §3.1.4 (INTMS, INTMC) · Base 2.4 §5.2.30.2.1–5.2.30.2.2 · Base 2.4 §5.3.1–5.3.2 · Base 2.4 §5.3.3–5.3.4 · Base 2.4 §4.2.1, 4.2.3–4.2.4 · Base 2.4 §4.4 · Base 2.4 §5.2.30.1 (common fields, scope and persistence) · Base 2.4 §3.7.1–3.7.4 · Base 2.4 §4.2.3 · Base 2.4 §5.2.13.1.2 · Base 2.4 §5.2.2 · Base 2.4 §5.2.13.1.14 (header, reset, hardware, Set Feature events)

Back to questions

Source locations and existing figure guides

Base printed page = PDF page−26; the other two use identical numbers. Locations follow the supplied PDF body and retain figure numbers. Shared pages contribute only the relevant definitions, excluding Fabrics and PCIe link/packet content.

  • Base 2.4 · §3.1.4 (INTMS, INTMC)
    Printed pages 59 · PDF 85 · Figure 39–40
  • Base 2.4 · §3.7.1–3.7.4
    Printed pages 120–124 · PDF 146–150
  • Base 2.4 · §4.2.1, 4.2.3–4.2.4
    Printed pages 144–157 · PDF 170–183 · Figure 97–105, 109
  • Base 2.4 · §4.2.3
    Printed pages 145–155 · PDF 171–181 · Figure 101–105
  • Base 2.4 · §4.4
    Printed pages 166–169 · PDF 192–195 · Figure 126–127
  • Base 2.4 · §5.2.2
    Printed pages 183–190 · PDF 209–216 · Figure 150–156
  • Base 2.4 · §5.2.13.1.2
    Printed pages 218–220 · PDF 244–246 · Figure 212
  • Base 2.4 · §5.2.13.1.14 (header, reset, hardware, Set Feature events)
    Printed pages 244–256, 258, 262–264 · PDF 270–282, 284, 288–290 · Figure 232–244, 246, 252–253
  • Base 2.4 · §5.2.30.1 (common fields, scope and persistence)
    Printed pages 456–460 · PDF 482–486 · Figure 463–466
  • Base 2.4 · §5.2.30.2.1–5.2.30.2.2
    Printed pages 514–515 · PDF 540–541 · Figure 543–544
  • Base 2.4 · §5.3.1–5.3.2
    Printed pages 527–531 · PDF 553–557 · Figure 571–579
  • Base 2.4 · §5.3.3–5.3.4
    Printed pages 531–532 · PDF 557–558 · Figure 580–583
  • PCIe Transport 1.4 · §3.5–3.5.2 (interrupt delivery and masks), 3.8.4
    Printed pages 13–16, 24–26 · PDF 13–16, 24–26 · Figure 9, 42–48

When you need a field guide

Existing figure explanations have canonical locations; use these links instead of duplicating the same guide.

Original documents used
  • NVM Express Base Specification · Revision 2.4 · 2026-07-31
    NVM-Express-Base-Specification-Revision-2.4-Ratified-2026.07.31.pdf
  • NVM Express NVM Command Set Specification · Revision 1.3 · 2026-07-31
    NVM-Express-NVM-Command-Set-Specification-Revision-1.3-Ratified-2026.07.31.pdf
  • NVM Express NVMe over PCIe Transport Specification · Revision 1.4 · 2026-07-31
    NVM-Express-NVMe-over-PCIe-Transport-Specification-Revision-1.4-Ratified-2026.07.31.pdf
Jia-Chang

Jia-Chang

Human

Comments

  Write a comment ...