PCIe Chapter 37: Performance Estimation and Integrated Observation

Calculate a clearly labeled upper bound

Author derivation: 64 GT/s ×4÷8 = 32 GB/s per direction. Accounting only for the 236/256 TLP region gives 29.5 GB/s, still before transaction headers, padding, control traffic, and stalls. Section 8.1 already includes PAM4’s two bits in GT/s; do not multiply by two again. Label any full-duplex aggregate separately from a one-direction workload.

TLP

Transaction Layer Packet

PAM4

Pulse Amplitude Modulation with 4 levels

How do two devices share an upstream Link?

Teaching assumption: Two Endpoints send data through a Switch to the Root Complex; their shared upstream Link currently operates at 64.0 GT/s ×4 in FLIT Mode. Identify the shared Link and use one traffic direction for the estimate.

The earlier calculation gives 32 GB/s of line capacity and 29.5 GB/s after applying only the FLIT TLP-area ratio. This ideal capacity is shared by both flows, not separately available to each Endpoint. Other traffic in that direction also uses the shared path.

Check each downstream Link's current speed and width and the Switch, Root Complex, and destination's processing capacity. Even with unused upstream capacity, a downstream Link, Flow Control, response latency, or the destination can limit a workload. Observations at each point help narrow the location.

Estimate enough outstanding data

Author assumptions: 16 GB/s useful throughput, 512 bytes per average request, and 2 µs average request-to-complete delay. Covering the delay needs 32,000 bytes outstanding, or ceil(32,000/512)=63 requests. Eight requests cover only 4,096 bytes, about 2.048 GB/s in this simplified model. Tags, credits, service, and shared paths remain limits. The assumed 2 µs is not a Completion Timeout estimate; section 2.8 prohibits using timeout ranges for typical latency.

When speed stays constant, where is the wait?

Teaching scenario: One Link remains at its expected speed and width, but application read throughput is low. First align the measurement interval, direction, request size, and outstanding count.

On a phone, scroll the table sideways to see all columns.

The 512-byte, 2-µs, and eight-request values are author assumptions. The estimate covers waiting time only; it is not a performance guarantee. Chapter 17 covers Credit, Chapter 18 Replay, and Chapter 36 ordering dependencies.

Configure the measurement that matches the wait

Use Figure 7-205 for layout and Figure 7-207/Table 7-188 for supported tracker count. Figure 7-208/Table 7-189 selects Flit-to-Ack, Flit-to-Nak, or Nak-to-Replay, plus Flit type and count. Do not exceed supported state trackers or change specified controls during active measurement. Read tracking status in Figure 7-209/Table 7-190 before using a result.

Source discrepancy: Table 7-189 describes an 8 ns measurement unit, while Table 7-190 names its counter LTSSM State Transition Tracking Counter and lists 64 µs. This lesson does not convert that counter into Flit latency without matching errata or implementation evidence.

LTSSM

Link Training and Status State Machine

How do we track one LTSSM transition?

LTSSM

Link Training and Status State Machine

Teaching assumption: A Downstream Port in FLIT Mode has a Flit Performance Measurement Extended Capability whose LTSSM Tracking Register Count is 1. This supports one simultaneous LTSSM transition tracking event; it does not limit every other measurement to one.

Use Figure 7-205 for register locations, then Figure 7-208 and Table 7-189 for controls. To track L1.0 to L0, select LTSSM State Transition Tracker bit 16. With Count 1, bit 18 for L1.2 to L0 cannot be enabled concurrently; stop and complete one measurement before configuring the other.

Flit Response Type is a separate choice: bits 1, 2, and 3 select Flit-to-Ack, Flit-to-Nak, and Nak-to-Replay. Choose the Link event of interest. Do not change controls that the specification identifies as undefined to modify during a running measurement.

Table 7-190 defines Flit Latency Tracking Status 00b, 01b, 10b, and 11b as Not started, Started, Completed, and Error. Error includes counter overflow. Completed indicates that tracking finished; verify the selected event and scope before interpreting it. The counter-name and unit discrepancy in this edition is described earlier in the chapter.

Connect observations to one mechanism at a time

Confirm negotiated rate/width, then whether request size and concurrency can use that capacity. Narrow remaining stalls by credit availability, replay, target service, and shared upstream paths. Use measurements whose scope matches throughput or latency. Return to Chapter 14 for packet layout, 16–17 for resources, 19–21 for training/electrical conditions, and 36 for ordering/quiescence rather than duplicating those explanations.

Edition and sources

This chapter uses PCI Express Base Specification Revision 6.0 (6.0-1.0-PUB). Document and PDF page numbers match. Example configurations and values are author assumptions, not device measurements.