PCIe Chapter 22: Link Diagnostics, Lane Margining, and Flit Records

Which figures lead from an Endpoint to Lane 3?

[Simulated data] The following data is created for teaching and does not represent specific hardware or a platform. A Root Port connects directly to an Endpoint, with no Retimer (signal-regenerating component) or Lane Reversal (reversed lane ordering). The Link is in L0 at 16.0 GT/s x4; both ports have a maximum width of x8. We examine active default Lane 3 at the Endpoint receiver, receiving data from the Root Port.

The Root Port is a Downstream Port; the Endpoint port is an Upstream Port. This case controls the Endpoint receiver through its own configuration space. For a multifunction Endpoint, only Function 0 contains this Lane Margining (receiver-margin measurement) capability.

Entries use default lane numbers, unaffected by negotiated width or Lane Reversal. Entries beyond Maximum Link Width have undefined values. Separately confirm that the chosen lane belongs to the current width before testing it; our active x4 includes Lane 3.

Source: §§7.7.10–7.7.10.5, document/PDF pp.1077–1083; §4.2.18.1 and Figure 4-60, pp.552–553.

How does one command reach a usable response?

First complete §4.2.18.3 preparation: the current rate must be at least 16.0 GT/s and the tested lane must be active. Program the Upstream Port's Functions to D-states preventing L1 entry, and set ASPM Control (autonomous Link power-saving control) to 00b at both ports. Save Hardware Autonomous Speed Disable and Hardware Autonomous Width Disable; set each to 1 at both ports if writable, and restore the saved values after testing.

Assume preparation is complete and only one lane is tested at a time. Figure 7-129 Control and Figure 7-130 Status share corresponding field positions. Bit 7 is RsvdP (preserve the reserved value), initially 0 here. Values below are whole 16-bit register values, not a memory byte-order illustration.

Complete a No Command handshake before changing commands. Table 4-63 defines Receiver=000b, Type=111b, Payload=9Ch and Usage Model=0b, giving Control=9C38h in this case. Write each new Control atomically, then read Status atomically. Match receiver, type, usage model and the command-specific expected payload. For a No Command response, check Receiver Number Status=000b, Margin Type Status=111b and Usage Model Status=0b. Under Table 7-115, Margin Payload Status is not meaningful for this comparison.

If the Link remains in L0 at 16.0 GT/s or higher, the target must apply and respond to a valid command within 1 ms of receiving it; a Step Margin response can still indicate setup in progress. Section 4.2.18.3 recommends spacing Status reads at least 10 μs apart. If no matching response arrives within 10 ms after a new command, software may declare receiver margining failure. A Configuration Write Completion is a separate event; a margin-command response is also distinct from a Data Link Layer Ack.

Source: §§4.2.18.2–4.2.18.3, document/PDF pp.557–561; Figures 7-129 / 7-130 and Tables 7-114 / 7-115, pp.1082–1083.

Query capabilities, then move five timing steps right

Continue the simulated Rx(F) case. Every new command row below follows a completed No Command handshake, including changes only to Payload. Table 4-63 places commands on the left and responses on the right. A Report response replaces the query payload with the requested value.

For example, 880Eh is (88h << 8) | (001b << 3) | 110b. Report Margin Control Capabilities payload 15h maps bit 4 to MIndErrorSampler, bit 3 to MSampleReportingMethod, bit 2 to MIndLeftRightTiming, bit 1 to MIndUpDownVoltage and bit 0 to MVoltageSupported. Bit 3=0 selects sample-count reporting. Bit 1=0 disallows selecting up/down independently; voltage support itself is still advertised by bit 0.

For Step Margin payload 05h, §4.2.18.1.2 applies: MIndLeftRightTiming=1 makes bit 6=0 select right, and bits [5:0]=5 select the step count. Table 8-12 requires monotonically increasing offsets, not equally spaced steps. Knowing that 32 steps reach a maximum 25% UI therefore does not establish a proportional time conversion for step 5.

Status 821Eh instead has payload 82h: execution status in the high two bits and MErrorCount (the margin-test error count) in the low six. With the assumed independent sampler, the two errors are sampler bit errors; normal Link traffic may remain error-free. With MIndErrorSampler=0, apply the logical-error conditions in §4.2.18.2, such as a Port entering Recovery. These counts measure different events.

Obtain the sampled amount through the advertised reporting method; our case can use Report MSampleCount after the required handshakes while margining continues. The specification recommends at least 10⁸ bits per position before moving on, unless the error limit is exceeded. Exclude setup time from actual sampling. The case has not supplied a sampled amount, so these Status reads demonstrate command and execution-state decoding rather than a completed margin assessment at this position. A zero-error snapshot describes only the observed samples.

The other Step Margin statuses are 11b, NAK (unsupported margin command), with MErrorCount=0; and 00b, too many errors, with automatic return to the default sample position. With an independent sampler, ending the test upon exceeding the limit is optional, so read execution status as well. Hardware must not remain in 01b setup for more than 100 ms after receiving a new valid command. Software may declare failure if it still observes 01b 200 ms after issuing the command. The NAK here is a margin-command execution response, distinct from a Data Link Layer Nak requesting replay.

Source: §§4.2.18.1–4.2.18.3, Table 4-63 Margin Commands and Corresponding Responses, document/PDF pp.553–561; §8.4.4 and Table 8-12 Lane Margining, pp.1395–1398.

How is testing in one direction ended?

No Command separates commands; it does not return an ongoing Step Margin to the normal sample position. At the end of each voltage/timing direction, first complete the No Command handshake required before changing commands. Then perform the §4.2.18.3 broadcast sequence below in both Downstream and Upstream Ports. Wait for each target response before proceeding. With no Retimer, the responders are the Root Port's Rx(A) and Endpoint's Rx(F).

1. Go to Normal Settings: Control=0F10h in this case; return to the normal sample position.
2. No Command: Control=9C38h; wait for the handshake.
3. Clear Error Log: Control=5510h; clear accumulated margin-test errors.
4. No Command: Control=9C38h; check the response again.

Receiver=000b broadcasts these commands. Except for No Command, a response identifies its actual responder: the Endpoint's assumed Go to Normal Settings response is 0F16h, while the Root Port's is 0F11h. Control and Status need not be bit-for-bit identical. Clear Error Log clears margining error records, not every AER (Advanced Error Reporting) status.

When testing finishes, restore the saved autonomous speed/width disable values. If the rate changes during margining, results are inaccurate; software must exit and set Control to No Command so later return to 16.0 GT/s or higher does not restart margining unexpectedly. For Recovery, apply the independent/shared-sampler rules governing suspension, default-position restoration and resumption. A later L0 snapshot does not establish an uninterrupted measurement.

Source: §4.2.18, document/PDF p.550; §§4.2.18.1–4.2.18.3, pp.553–561.

Which questions belong to margining or Flit records?

Margining changes test conditions and reports behavior at a selected sample position. For errors recorded during transmission, choose a structure by purpose below. See Chapter 18 for Flit Mode (fixed-length transfer-unit mode), formats and replay.

For a Retimer, the Downstream Port sends commands downstream through Control SKP Ordered Sets (SKP sequences carrying Link-control information) and receives responses upstream. These receivers remain within the same Link. Our Endpoint Rx(F) uses its own Control/Status rather than this Retimer relay path.

What context makes a result reusable?

Save port/function, default lane, receiver number, rate/mode, reported capabilities, direction, steps, start/end times, execution status, errors and sampled amount. Record retraining, rate changes and restoration commands as well. These identify whether two observations used the same position, error definition and comparable sample populations.

Section 7.7.7 notes that 64.0 GT/s parity errors are recorded in the 16.0 GT/s Local, First Retimer and Second Retimer Data Parity Mismatch Status Registers. To track one speed, software should clear these records on speed changes; preserve the old records before beginning the next observation interval.

This case covers capability queries, one timing position and restoration. A full margin profile requires more positions/directions and adequate sampling. Parameters may differ across rates. Voltage margining is optional at 16.0 GT/s and required at 32.0/64.0 GT/s. At 64.0 GT/s, Step Margin applies to all three PAM4 (four-level signaling) eyes simultaneously; use that rate's parameters. Internal receiver observations differ from the external stressed-eye and BER tests in Chapter 21 and alone do not establish full electrical compliance.

PAM4

Pulse Amplitude Modulation with 4 levels

Edition and sources

This chapter uses PCI Express Base Specification Revision 6.0 (6.0-1.0-PUB). Document and PDF page numbers match. Example values are author assumptions, not hardware measurements.