Flit、編碼與錯誤更正，怎麼放在同一張圖裡？
read more
PCIe Chapter 18: Encoding, Flits, CRC, and FEC
The previous chapter separated credit, Ack, and transaction completion. Here a TLP (Transaction Layer Packet) enters a Flit (fixed-size transfer unit) and crosses the Lanes to the receiver. First follow normal placement and checks, then use Selective Replay to see why TLP content and DLP are handled differently.
- PAM4
- Pulse Amplitude Modulation with 4 levels
Choose the mode before the wire decoder
64.0 GT/s uses PAM4 (four-level pulse-amplitude modulation) and Flit Mode. Lower rates can also use Flit Mode. Establish the Link, direction, rate, and mode at the observation time before decoding a trace.
- PAM4
- Pulse Amplitude Modulation with 4 levels
The identification note on p.309 distinguishes Flit Mode Supported in the PCI Express Capabilities Register, Flit Mode Disable in Link Control, and Flit Mode Status in Link Status 2. The last field describes operation in Flit Mode now or in the future. Training advertisements and the successfully negotiated variable Flit_Mode_Enabled are separate information.
| Rate | Wire encoding | Flit Mode detail |
|---|---|---|
| 2.5 / 5.0 GT/s | 8b/10b: 8-bit data characters become 10-bit Symbols | Traditional STP, SDP, END, and EDB packet markers are not used |
| 8.0 / 16.0 / 32.0 GT/s | 128b/130b: 128-bit payload plus a 2-bit Sync Header | Per-Lane Block encoding remains; Non-Flit Mode Framing Tokens are not used |
| 64.0 GT/s | 1b/1b and PAM4: two bits per signaling interval | Use Flit Mode and the transmit/receive processing in Figures 4-22 and 4-23 |
Packaging and wire encoding answer different questions. The name 1b/1b does not remove Flit management, CRC, or correction overhead.
Base 6.0 §§4.2.1, 4.2.2, 4.2.3, 4.2.3.1 (pp.309, 318, 340); Figures 4-22 and 4-23 (p.342).
Follow a Flit byte to its Lane
Read the fixed areas in §4.2.3.3, then select the placement table for the Link width. Offsets below start at byte 0 of one Flit. The TLP area may contain several TLPs or a portion of a TLP.
| Flit byte range | Size | Purpose |
|---|---|---|
| 0–235 | 236 bytes | TLP content: headers, data, and required NOP (no-operation) TLPs |
| 236–241 | 6 bytes | DLP (Data Link Layer Payload), numbered DLP0–DLP5 |
| 242–249 | 8 bytes | CRC (Cyclic Redundancy Check) over the first 242 bytes, excluding ECC |
| 250–255 | 6 bytes | ECC (Error Correcting Code); protection covers the entire Flit, including CRC |
Simulated data: This author-created teaching case is not a device or platform measurement. Assume an established 64.0 GT/s x4 Link with aligned, decoded data and specification Logical Lane numbers.
Select Table 4-12 for x4. Read Lane 0 through Lane 3 across a row, then move down: bytes 0–3 occupy Lanes 0–3, and byte 4 returns to Lane 0. The selected positions below show how this continues into DLP and protection bytes.
| Flit byte | Content | Logical Lane | Position check |
|---|---|---|---|
| 236 | DLP0 | 0 | Bytes 236–239 carry DLP0–DLP3 |
| 237 | DLP1 | 1 | Adjacent Lane after DLP0 |
| 242 | CRC0 | 2 | Bytes 240–241 still carry DLP4–DLP5 |
| 250 | ECC1[0] | 2 | Bytes 248–249 carry CRC6–CRC7 |
| 252 | ECC0[0] | 0 | The final group is ECC0[0], ECC1[1], ECC2[1], ECC0[1] |
Keep the content label and position together. The ECC area starts at byte 250, but that byte is ECC1[0], not ECC0[0]. This example follows placement without treating table rows as training or protocol wait times.
Next read Table 4-15 from the continuation of TLP 19 through subsequent headers, data, and NOPs. It shows why one Flit need not begin with a new TLP. When no TLP is ready, §4.2.3.4.1 requires NOP TLPs (no-operation packets) with alignment and packing rules; unused space is not arbitrary padding.
Author calculation: 236/256=92.1875% is the TLP-area fraction, before TLP headers and NOPs. It is not application throughput; see Chapter 37 for performance estimates.
Base 6.0 §4.2.3.3 (pp.351–352); Table 4-12 Flit interleaving in a x4 Link (p.354); §4.2.3.4.1 and Table 4-15 Example TLP Placement in Flit Mode on a x16 Link (pp.357–359).
Follow normal correction and checking first
PAM4 (two bits per interval using four voltage levels) increases signaling density, but smaller eye height and width increase susceptibility to errors. FEC (Forward Error Correction) attempts correction before the Flit CRC check.
Read Figures 4-22 and 4-23 together. The transmitter generates eight CRC bytes from the 236-byte TLP area and six-byte DLP, generates ECC, then distributes the Flit across Lanes. The receiver performs applicable Lane decoding and descrambling, aggregates the Flit, applies FEC decoding/correction, and checks CRC. These are conceptual processing diagrams, not a mandate for identical internal datapath widths.
- FEC
- Forward Error Correction
For the x4 case, assume one correctable byte error is corrected, no ECC group reports an uncorrectable error, CRC passes, and Flit Usage and Flit_Status use legal encodings. This meets the valid-Flit conditions in §4.2.3.4.2. Sequence, replay state, and TLP-format rules still determine which TLP content can be delivered; a passing CRC does not complete a Request.
The three colors in Table 4-12 identify ECC groups, not Lanes. Each group can correct one byte error. The distribution across groups matters, so a total count of erroneous bytes alone is insufficient. A CRC failure or an ECC-group uncorrectable indication prevents valid-Flit treatment; receive and replay rules then apply.
Base 6.0 §4.2.3.1; Figures 4-22 Transmit side at 64.0 GT/s and 4-23 Receive side at 64.0 GT/s (pp.341–342); §4.2.3.1.1 (pp.342–343); §4.2.3.4.2 (p.359); §§4.2.3.4.3–4.2.3.4.4 (pp.384–385).
Must a replayed Flit be byte-for-byte identical?
TLP content is replayed; DLP carries current management information. Section 4.2.3.3 excludes DLP from the Link Layer Retry Buffer. Since CRC covers DLP, CRC and ECC are generated for the content being transmitted now.
Simulated data: Author-created data for teaching, not measurements. Root Port 00:01.0 and Endpoint 01:00.0 share the 64.0 GT/s x4 Link. Both are in Normal Flit Exchange Phase with Data Link and Flow Control initialization complete. The following is an event excerpt, not a continuous full trace.
Decode the transmitted DLP before assigning a direction to its sequence
The Root Port transmits a Payload Flit with Explicit Sequence Number 12Ah. Its saved pre-channel DLP0–DLP5 are 51 2A 80 00 00 00. Assume its previous transmitted Flit was NOP, making Prior Flit was Payload zero. These bytes are the transmitter's original content; a damaged receiver-side DLP is not reliable evidence after a CRC failure.
| Field in Tables 4-17 / 4-19 | Decoded value | Use in this case |
|---|---|---|
| DLP0[7:6]: Flit Usage | 01b | Payload Flit contains part of at least one non-NOP TLP |
| DLP0[4]: Type of DLLP Payload | 1b | Inspect DLP2[7] to select Optimized_Update_FC or Flit_Marker |
| DLP0[3:2]: Replay Command | 00b | The included sequence explicitly identifies the transmitted Flit |
| {DLP0[1:0], DLP1[7:0]}: Flit Sequence Number | (1h << 8) | 2Ah = 12Ah | Combine high two bits and low eight bits, rather than reading DLP1 alone |
| DLP2–DLP5: Flit_Marker | 80000000h | bit31=1 selects Flit_Marker; Flit_Status=00b supplies no Nullified or Poisoned indication here |
- DLLP
- Data Link Layer Packet
Decode the reverse-direction Selective Nak
CRC still fails after FEC processing at the Endpoint. Assume later received information identifies the invalid Flit as Payload, sufficient RX Retry Buffer exists, and the Endpoint chooses Selective Nak (a request to replay one specified Flit). One valid reverse-direction NOP Flit carries 1D 29 80 00 00 00. Its preceding Flit in that direction was also NOP, so Prior Flit was Payload is zero.
Here Replay Command=11b. Sequence 129h means the last valid received Flit, and the requested replay is 129h + 1 = 12Ah. It refers to Root Port → Endpoint traffic, not a newly assigned sequence for this reverse NOP Flit. Command 10b would mean Standard Nak, requesting all unacknowledged Flits rather than this single-Flit scope.
| Direction and capture point | DLP0–DLP5 | Replay Command | Sequence field | Meaning in this row |
|---|---|---|---|---|
| Root Port → Endpoint, before transmission | 51 2A 80 00 00 00 | 00b | 12Ah | The transmitted Payload Flit |
| Endpoint → Root Port, valid reception | 1D 29 80 00 00 00 | 11b | 129h | Requests replay of 12Ah |
Compare replay content and the receive outcome
The Root Port schedules replay of 12Ah, retrieving TLP content from its TX Retry Buffer and using current DLP information. When it sends the replay with an Explicit Sequence Number, it uses the stored 12Ah. The Endpoint applies validity and sequence checks and processes stored/replayed content in receive order, preventing replay from becoming a newly delivered transaction.
The excerpt omits scheduling waits, other Flits, and repeated Naks. Section 4.2.3.4.2.1.3 requires at least three consecutive Nak Flits in Normal Flit Exchange Phase, subject to its exceptions. One arrow is therefore not a complete wire schedule. Read Figure 4-33 next: first follow the single invalid Payload Flit and Selective Replay, then the later consecutive invalid Flits and Standard Nak.
Successful replay establishes a recovered Link-transfer outcome for this case. The original Transaction still determines the required Completion and final result. See Chapter 14 for TLP formats and Chapter 5 for Request/Completion analysis.
Base 6.0 §4.2.3.3 (p.352); Figure 4-29 DLP Byte to Bit Number Assignment (p.359); Figure 4-30 DLP Bit usage, Tables 4-16 Flit Types, 4-17 DLP Bytes in the Flit (pp.360–361), and 4-19 Flit_Marker (p.362); §§4.2.3.4.2.1.3–4.2.3.4.2.1.7 (pp.370–383), including Figure 4-33 Flit Ack/Nak/Replay Example (p.383).
Use appendices when verifying algorithm outputs
The layout, DLP, and receive rules are enough for the case above. Use the appendices when checking a specific encoder, decoder, or check-code output.
| Verification task | Source | Condition to retain |
|---|---|---|
| 2.5 / 5.0 GT/s data-character encoding | Appendix B, Table B-1 8b/10b Data Symbol Codes (p.1531 onward) | Select by byte and Running Disparity, the per-Lane accumulated imbalance; this is not a 64.0 GT/s coding table |
| 8b/10b scrambling output | §C.1 8b/10b Data Scrambling Example (p.1541) | Alternative implementations must produce the same output; this example is not a universal scrambler for every rate |
| Flit ECC encode/decode | §4.2.3.4.4; Appendix J (p.1595); §J.2 84 Byte to 86 Byte Encoder (p.1607) | Apply the interleaving before checking check/parity results; a different pipeline must preserve reference outputs |
| Eight-byte Flit CRC | §4.2.3.4.3; Appendix K MATLAB created generator matrix for CRC code (p.1619) | Input is 242 TLP/DLP bytes; preserve the specified matrix and bit ordering |
Edition and sources
PCI Express Base Specification Revision 6.0 (6.0-1.0-PUB) is the technical authority. Printed and PDF-viewer page numbers match for the cited locations. Assumed values, topology, and event excerpts are marked beside the cases.
Complete reading locations
- §4.2.1 8b/10b Encoding for 2.5 GT/s and 5.0 GT/s Data Rates (p.309); §4.2.2 128b/130b Encoding for 8.0 GT/s, 16.0 GT/s, and 32.0 GT/s Data Rates (p.318); §4.2.3 Flit Mode Operation and §4.2.3.1 1b/1b Encoding for 64.0 GT/s and higher Data Rates (p.340).
- Figures 4-22 / 4-23 (p.342) explain processing order; §4.2.3.1.1 PAM4 Signaling and Figure 4-24 (pp.342–343) supply signaling/error context.
- §4.2.3.3 Data Stream in Flit Mode (pp.351–352); Tables 4-10–4-14 (pp.352–356) select x16 / x8 / x4 / x2 / x1 placement. This case reads Table 4-12 (p.354).
- §4.2.3.4.1 TLP Bytes in Flit, Table 4-15 (pp.357–359); §4.2.3.4.2 DLP Bytes in Flit, Figures 4-29–4-30, Tables 4-16–4-17 and 4-19 (pp.359–362).
- §4.2.3.4.2.1.3 Normal Flit Exchange Phase (pp.370–372); §4.2.3.4.2.1.4 Received Ack and Nak Processing (p.372); §4.2.3.4.2.1.5 Ack, Nak, and Discard Rules (pp.373–378); §4.2.3.4.2.1.6 Flit Replay Scheduling (pp.378–379); §4.2.3.4.2.1.7 Flit Replay Transmit Rules (pp.380–383, including the Figure 4-33 example).
- §4.2.3.4.3 CRC Bytes in Flit (pp.384–385); §4.2.3.4.4 ECC Bytes in Flit (p.385 onward); Appendices B, C.1, J, J.2, and K as mapped in the preceding table.
Comments