PCIe Chapter 15: Request Handling, Completions, and Advanced Transactions

After parsing a TLP (Transaction Layer Packet), follow the receiver into validity and acceptance checks. An AtomicOp (Atomic Operation) performs an atomic read-modify-write; DMWr (Deferrable Memory Write) supports shared work submission. Both return completions, with different operation and status rules.

Roles and events in this chapterTrace reception checks, request execution, and the required completion response.Receiver: parse requestCheck target, support, size, and accessAtomicOp: atomic read-modify-writeReturn the original valueDMWr: accept shared workReturn applicable statusRequester: handle completionAtomicOpDMWrCompletionCompletion
Author-drawn teaching diagram based on PCI Express Base 6.0. Exchanges unrelated to this lesson are omitted.

What happens after a request reaches the receiver?

The previous chapter established the packet structure. Here, the Data Link Layer has checked transmission integrity, and the Transaction Layer applies packet formation and request-specific rules. Start at Figure 2-81 (p.206), follow the valid-format path, then separate Requests from Completions.

The first decision explicitly says Ignoring Reserved Fields. A receiver must ignore values in Reserved fields; a nonzero Reserved value alone does not establish a Malformed TLP. Other constrained fields retain their own mandatory or optional checks. This rule concerns fields marked Reserved, not Reserved encodings within active fields such as Fmt/Type; those follow their mode-specific rules. A TLP determined to be Malformed must be discarded.

TLP
Transaction Layer Packet

For a Request, continue to Figure 2-83 (p.213): determine type support, then apply the rules for that Request. The dashed programming-model check is optional. It permits certain devices to handle accesses that violate their programming model as Completer Abort; it is not a mandatory stage of every transaction. The flowchart directs the analysis, while sections 6.15 and 6.32 supply the operation-specific rules for CAS and DMWr.

Reading route: §2.3 Handling of Received TLPs, pp.204–206, Figure 2-81 Flowchart for Handling of Received TLPs, p.206; §2.3.1 Request Handling Rules, pp.208–213, Figure 2-83 Flowchart for Handling of Received Request, p.213. Document and PDF pages match.

What happens to Flow Control when a TLP is Malformed?

Continue from Figure 2-81: the Receiver has obtained a TLP from the Data Link Layer and now checks Transaction Layer formation. Every received Malformed TLP must be discarded. Read the text in §2.3 as well to determine whether Receiver Flow Control information is updated; the diagram's discard box alone does not show every condition.

Teaching example: Two independent received packets have each been determined Malformed. Assume the Receiver has the VC (Virtual Channel) and buffer observations needed for the corresponding case.

On a phone, scroll the table horizontally to see all columns.

Receiver observationDiscard and Flow Control handling
The buffer to release is ambiguous, or the TLP maps to an uninitialized or disabled VCDiscard without updating Receiver Flow Control information.
Other Malformed TLPDiscard; the Receiver may choose not to update Flow Control information.
VC
Virtual Channel

The second row permits not updating; it does not require an update or require every device to omit one. These rules concern Receiver tracking. A single discard does not establish the state of every physical buffer or that the transmitter has received new credits. See Chapter 17 for the normal credit-update flow.

Separate Reserved fields from reserved encodings. In Non-Flit Mode, with Extended Fmt Field Supported=1, reserved Fmt/Type encodings are Malformed. With that capability clear, processing received TLPs with Fmt[2]=1 is undefined. With Fmt[2]=0 and an undefined Type value, the TLP is Malformed. These rules differ from ignoring values in fields marked Reserved. See Chapter 14 for Prefix decoding.

PCI Express Base 6.0 §2.3 and Figure 2-81, printed/PDF pp.204–206; §7.5.3.15 and Table 7-33, p.992. Check individual format rules for mandatory or optional validation.

What must support an atomic update?

An AtomicOp (Atomic Operation) reads a target location, potentially updates it, and returns its original value in one transaction. The read-modify-write sequence is atomic with respect to accesses to that location. First establish the operation in section 6.15, then follow its actual path.

OperationRequest dataNormal execution and return
FetchAddOne add operandWrite the sum; return the original value
SwapOne swap operandWrite the replacement; return the original value
CAS (Compare and Swap)Compare operand, then swap operandReplace only on equality; return the original value in either case

【模擬資料】Teaching assumptions, not measurements of a particular device or platform.Endpoint 03:00.0 sends a 32-bit CAS through one Switch toward host memory served by Root Port 00:01.0. The Request enters Switch Downstream Port 02:01.0 and exits Upstream Port 01:00.0. Both Links are operational. Address 0x80001FFC is a platform-authorized AtomicOp target.

ParticipantFields to inspectAssumptions and purpose
Source Endpoint 03:00.0Device Control 2: AtomicOp Requester Enable, bit 6; Command: Bus Master EnableBoth are 1. Requester capability has also been established through the device interface; a writable enable bit alone does not establish it.
Switch pathDevice Capabilities 2: AtomicOp Routing Supported, bit 6; actual egress Device Control 2: AtomicOp Egress Blocking, bit 7Relevant path Ports report Routing Supported = 1. The actual egress is Upstream Port 01:00.0, with Egress Blocking = 0.
Target Root Port 00:01.0Device Capabilities 2: 32-bit AtomicOp Completer Supported, bit 71, with target permissions and platform configuration established. Routing support cannot substitute for completer support.

These capability fields are RO in Table 7-33. The two AtomicOp controls are RW where applicable in Table 7-34. Offsets are relative to the PCI Express Capability: Device Capabilities 2 at +24h, Device Control 2 at +28h. Locate the Capability before adding the offset. Base 6.0 does not define a universal requester-capability discovery interface; it even permits a writable Requester Enable without requester capability.

Reading route: §6.15 Atomic Operations (AtomicOps), pp.772–773; §6.15.2 AtomicOp Transaction Protocol Summary, pp.774–775; §6.15.3.1 Root Ports with AtomicOp Completer Capabilities, p.775; §7.5.3.15 Device Capabilities 2 Register (Offset 24h), Table 7-33, p.990; §7.5.3.16 Device Control 2 Register (Offset 28h), Table 7-34, p.996.

Does a successful CAS always replace the value?

No. CAS (Compare and Swap) writes its swap operand only when the original value equals the compare operand. A mismatch is also a normal execution result. The requester compares the returned original value with its submitted compare operand. Successful Completion (SC) has no separate comparison-match flag.

【模擬資料】Teaching assumptions, not measurements of a particular device or platform. Use the preceding path with both Links in Non-Flit Mode. Each alternative independently starts at 7, with no concurrent updates. These are not sequential transactions. Requests are properly formed, EP=0, TH=0, with no Prefix and no access error.

Requester ID is 0300h, Tag is 2Ah, TC=0, and the address is 0x80001FFC. For replacement, Request Data contains compare value 7 followed by swap value 9, both 32-bit operands. Values are already decoded; this example does not prescribe the target memory's endian format.

TC
Traffic Class
A completed CAS can either replace or preserve the valueTwo independent cases start at 7. Compare 7 writes 9; compare 5 leaves 7. Both return original value 7 with SC.Supported, valid 32-bit CASRead original value 7 atomicallyOriginal = compare?Yes: compare 7No: compare 5Write swap value 9Keep target value 7Return SC + original value 7Requester compares returned value with its compare operand
Author-drawn decision flow based on §6.15. These are alternative cases, not two CAS Requests in sequence. SC = Successful Completion.
Two independent casesReplacementComparison mismatch
Original value before execution77
Request Data: compare / swap7 / 95 / 9
Target after execution97, unchanged
Completion StatusSC, 000bSC, 000b
Completion DataOriginal value 7Original value 7
Requester interpretationReturned 7 equals submitted compare 7: replacement occurredReturned 7 differs from submitted compare 5: no replacement

Why does an 8-byte Request operate on only 4 bytes?

Read the CAS row and 32 Bits column of Table 2-15 (p.162): Request Length=2 DW, carrying two operands totaling 8 bytes. The target read, possible write, and returned original value each use one 32-bit value. A DW (Double Word) is 4 bytes.

Then read the natural-alignment and 4-KB-boundary rules on p.163. The address is divisible by 4. The target range 0x80001FFC–0x80001FFF occupies the final 4 bytes of a 4-KB page without crossing it. Adding all 8 Request Data bytes to the target address would falsely flag a boundary violation. Both CAS operands act on one location, not two adjacent locations.

Which Completion fields have AtomicOp-specific meaning?

Use Figure 2-73 (p.196) to locate Requester ID and Tag and confirm 0300h / 2Ah; Chapter 5 explains matching. The successful response is CplD with Length=1 DW, Byte Count=4, and original value 7. A 2-DW Request does not require a 2-DW Completion.

Section 2.2.9.1 specifies that AtomicOp Completion Byte Count equals the operand size and Lower Address is Reserved. Although the target's low seven address bits are 7Ch, do not use Lower Address to reconstruct this result. For a Non-Flit AtomicOp Request with TH=0, First/Last DW BE are also Reserved. Memory Read byte-enable and Split Completion calculations do not apply to this example.

Connected sources: §6.15, pp.772–773, establishes CAS semantics; §2.2.7.1 Non-Flit Mode, Table 2-15 Length Field Values for AtomicOp Requests, pp.162–163, establishes operand count, alignment, and target range; §2.2.9.1 Completion Rules for Non-Flit Mode, Figure 2-73 Completion Header Format - Non-Flit Mode, p.196, and Table 2-37 Completion Status Field Values, p.197, establish response fields and the SC encoding. Document and PDF pages match.

The field layout here is Non-Flit. Flit Mode retains the section 6.15 comparison, replacement, and original-value semantics, but uses its own Header Base/OHC rules. Do not reuse these field positions; see Chapter 14 for the layout-reading method.

OHC
Orthogonal Header Content

How does DMWr retry differ from a CAS mismatch?

DMWr (Deferrable Memory Write, a non-posted write whose acceptance may be deferred) supports uses such as shared work queues. It requires a Completion but does not compare and exchange an operand. For a temporary condition, the completer may return Request Retry Status (RRS), leaving the target unchanged.

【模擬資料】Teaching assumptions, not measurements of a particular device or platform. A separate path and completer support DMWr, all required controls are enabled, and DMWr Requester Enable is 1. A first submission encounters a temporarily unavailable queue; the completer returns RRS without enqueuing the item. The requester may later submit a new Request under its implementation policy. A successful submission then proceeds according to the queue interface; acceptance of an item and completion of its work are separate events.

Observed resultCompleter actionRequester decision
CAS: SC, returned value differs from compareComparison completed; no replacementThis transaction has a known outcome. Any further attempt follows the synchronization protocol.
DMWr: RRS for a temporary conditionNo acceptance of this update; target unchangedRetry is permitted, not required; timing and whether to modify a new Request are implementation specific.
Ordinary Posted Memory WriteNo ordinary Completion conveys temporary rejectionUse the device interface or another synchronization mechanism to establish acceptance or work completion.

The temporary-condition RRS in section 6.32 is not an error, and the completer must not assume another attempt. DMWr has additional atomicity, size, routing, Egress Blocking, and error rules. A shared status name does not make Configuration Request retry handling applicable.

Reading route: §6.32 Deferrable Memory Write, pp.857–860; p.858 covers temporary-condition RRS and unchanged targets; p.860 covers Requester Enable and retry discretion. Table 2-37, p.197, encodes RRS as 010b.

Can a timed-out CAS simply be sent again?

First identify the retry layer. Data Link Replay protects one transmission between adjacent partners and prevents duplicate delivery. A requester-generated new CAS causes another operation. See Chapter 8 for that distinction.

Completion Timeout records the requester's failure to receive a required response. For the preceding CAS, at least two timelines remain possible: the Request never reached the target, or the target performed the comparison and replacement but the requester did not receive the Completion. A requester-side Timeout alone cannot distinguish them.

Recovery therefore cannot treat an unconfirmed operation as unexecuted and blindly issue it again. Preserve the original Request, Requester ID/Tag, path traces, and error evidence, then follow the platform or device synchronization and recovery protocol. A later memory value may reflect other agents' updates and cannot replace the missing Completion. This is an analysis boundary derived from the response mechanism; Base 6.0 supplies no universal software-retry algorithm.

A normal SC also needs request-specific interpretation. CAS returns the original value to establish whether replacement occurred. A split Memory Read needs all required Completions. Section 2.8 timeout ranges detect abnormal conditions and must not estimate normal service latency. If a subsequent Request must follow observation of an AtomicOp at the completer, section 6.15.2 requires the requester to wait for its Completion before issuing that Request.

Atomicity applies to the target operation. For Root Ports supporting AtomicOps, section 6.15.3.1 additionally requires host-memory accesses on behalf of an AtomicOp to be atomic relative to processor/device accesses to that range and to preserve processor atomic-operation atomicity. The complete synchronization algorithm, publication ordering at other locations, and software memory attributes still require platform and CPU documentation; one CAS does not establish those conditions.

Reading route: §2.8 Completion Timeout Mechanism, pp.259–260; §6.15.2, p.774; §6.15.3.1, p.775. Replay versus software retry is applied here without repeating its state machines.

Are Locked Transactions the same mechanism?

Locked Transactions have different section 6.5.1 restrictions: only the Root Complex may initiate them; PCI Express Endpoints must not support locked accesses; new software must not cause locked accesses to I/O devices. They are not an unrestricted fallback for unsupported AtomicOps.

Edition and sources

Technical authority: PCI Express Base Specification Revision 6.0 (6.0-1.0-PUB). Printed document and PDF viewer pages match. Diagrams and numerical scenarios are author-created from the cited rules, not copied specification figures or hardware measurements. This revision teaches CAS outcomes and DMWr retry; full AtomicOp formats, DMWr capabilities, and all error conditions remain for deeper coverage.

Complete source locations and purpose
  • §2.3 Handling of Received TLPs, pp.204–206; Figure 2-81 Flowchart for Handling of Received TLPs, p.206: receive entry, Reserved fields, and Request/Completion branches.
  • §2.3.1 Request Handling Rules, pp.208–213; Figure 2-83 Flowchart for Handling of Received Request, p.213: support, optional programming-model checks, and response.
  • §2.2.7.1 Non-Flit Mode, pp.162–163; Table 2-15 Length Field Values for AtomicOp Requests, p.162: operands, Length, alignment, and CAS target range.
  • §2.2.9.1 Completion Rules for Non-Flit Mode, pp.196–198; Figure 2-73 Completion Header Format - Non-Flit Mode, p.196; Table 2-37 Completion Status Field Values, p.197: AtomicOp Byte Count, Reserved Lower Address, Requester ID/Tag, and status encodings.
  • §6.15 Atomic Operations (AtomicOps), pp.772–773; §6.15.2 AtomicOp Transaction Protocol Summary, pp.774–775; §6.15.3.1 Root Ports with AtomicOp Completer Capabilities, p.775: operations, path support, waiting, and host-memory atomicity.
  • §7.5.3.15 Device Capabilities 2 Register (Offset 24h), Table 7-33, p.990; §7.5.3.16 Device Control 2 Register (Offset 28h), Table 7-34, p.996: AtomicOp support, enable, blocking, field scope, and attributes.
  • §6.32 Deferrable Memory Write, pp.857–860: temporary-condition RRS, unchanged target, enable, and retry discretion.
  • §2.8 Completion Timeout Mechanism, pp.259–260: incomplete responses and service-latency limits; §6.5.1 Introduction, p.718: Locked Transaction initiator and support restrictions.
Jia-Chang

Jia-Chang

Human

Comments

  Write a comment ...