PCIe Chapter 3: From Unknown Topology to Function Discovery

Simulated data: The topology, Bus Numbers and values are created for teaching and do not represent a particular device or platform.

Follow the same tree to discover functions

The diagram shows completed bus ranges for this example. The text separately explains temporary probe ranges and later updates. The chosen numbering policy is illustrative.The diagram shows completed bus ranges for this example. The text separately explains temporary probe ranges and later updates. The chosen numbering policy is illustrative.Root Port 00:01.0Completed downstream range 01–04Switch Upstream 01:00.0Completed downstream range 02–04Downstream A 02:01.0Completed range 03–03Downstream B 02:02.0Completed range 04–04Endpoint:03:00.0/03:00.1Endpoint:04:00.0
The diagram shows completed bus ranges for this example. The text separately explains temporary probe ranges and later updates. The chosen numbering policy is illustrative.

Enumeration starts at a platform-provided root, establishes bus ranges for the bridge path, probes downstream functions and records their identities. Chapters 1 and 2 explain the topology and configuration fields used here.

Specification route: Figures 7-1/7-2 → bridge bus-number fields in Figure 7-14 → identity and header layout in Figures 7-7/7-8.

Enumeration establishes usable configuration paths before discovering downstream Functions. It must also distinguish valid identity, temporary unreadiness, and failed access. This chapter connects Bridge Bus Number programming with those different results.

Full Chinese tutorial · Course roadmap · 繁體中文

The platform supplies an entry point; Function reads supply evidence

Software starts with platform-provided Host/Root Bridge information: Segment, Bus ranges, and configuration access services. It does not search arbitrary physical memory. The Root Bridge's platform interface differs from a Root Port's external Link role. Firmware 3.3 requires all Host Bridges to be configured so that the OS can manage their downstream devices.

Vendor ID is the specification's recommended presence probe, with FFFFh indicating no Function. Software-visible data is not necessarily Completion payload: for software depending on all-one responses, the Root Complex synthesizes that value from a Configuration Request's UR result. Failed access or an unconfigured path cannot establish that a physical card is absent merely from arbitrary buffer contents.

Sources

Firmware 3.3 §3.1.1, p.32; §3.5, pp.34–35. Base 6.0 Vendor ID: §7.5.1.1.1, p.917; unimplemented targets and UR: §7.3.1, p.911; synthesized all-one data: §2.3.2, p.221.

0xFFFF0001 is not an ordinary Vendor/Device ID

RRS (Request Retry Status) is Base 6.0's status for temporary inability to service a Request in specified circumstances. It is legal only under the applicable Reset/Readiness conditions, not as a general busy response at any time.

Check capability and control before interpreting synthesized data
ReadAssumed valueMeaning
Root Capabilities: 0x8E, 16 bits0x0001Bit 0 supports Configuration RRS Software Visibility.
Root Control: 0x8C, 16 bits0x0010Bit 4 enables visibility.
Target offset 0x00, 32 bits0xFFFF0001After RRS, the Root Complex synthesizes Vendor ID 0001h and ones in the remaining bytes.

The read must cover both Vendor ID bytes. A read of only that 16-bit field returns 0001h, which is not an assigned Vendor ID. With visibility disabled or other locations read, software must not expect this same format; the Root Complex reissues the Request where required. Waiting limits depend on Reset, Readiness, and platform policy rather than one universal delay.

Reading route

Figure 7-21 (p.952) locates +1Eh/+1Ch; Figure 7-34/Table 7-31 (p.988) identify support, and Figure 7-33/Table 7-30 (pp.986–988) identify control. RRS conditions: §2.3.1, pp.211 and 214; synthesized data and retry: §2.3.2, pp.220–221.

Open a temporary downstream range, then finalize it

The example follows one branch to completion before returning to its sibling. It temporarily uses the platform's 3F limit as Subordinate, then narrows the range to the discovered topology. This is a teaching strategy, not a mandatory universal algorithm.

Teaching sequence for Bus Number assignment
StepAction
1Set Root Port 00:01.0 to Secondary=01, Subordinate=3F.
2Find Switch Upstream Port 01:00.0; set Secondary=02, Subordinate=3F.
3Find Downstream A at 02:01.0; set Secondary=03, Subordinate=3F.
4Confirm 03:00.0 and 03:00.1; after completing this branch, narrow A's Subordinate to 03.
5Return to Bus 02 and set Downstream B at 02:02.0 to Secondary=04, Subordinate=3F.
6Confirm 04:00.0; narrow B to 04, then finalize the Upstream Port and Root Port at 04.

Header Type 80h at 03:00.0 only permits probing other Functions; Function 1's presence is separate evidence. Header Type 00h at 04:00.0, without another mechanism indicating otherwise, prohibits blanket probing of Functions 1–7. Ordinary Link partners use Device 0 in this example, while internal Switch Downstream Ports can use other Device Numbers.

Figure 7-14 locates Secondary/Subordinate at 0x19/0x1A, each eight bits, with meanings on p.937. Configuration Writes must complete before software uses the new path. The example does not derive a Bridge's own BDF from Primary Bus Number. A platform with reservation requirements need not narrow its final limit to 04.

BDF
Bus/Device/Function
Sources

Base 6.0 §7.5.1.3, Figure 7-14, p.936; §7.5.1.3.2–§7.5.1.3.4, p.937; Header Type: Table 7-7, p.925; Device Number: §7.3.1, p.911; ECAM write completion guarantees: p.908.

Header Type and Configuration Request Type are distinct

Configuration Header Type 1 describes a Bridge layout. Configuration Request Type 1 routes toward a deeper target. For 03:00.1, the Root Port's next Bus is 01, so the Request continues to the Switch as Type 1. The Switch recognizes Bus 03 as Downstream A's immediate downstream Bus and converts it to Type 0 before sending it to the Endpoint. A Bridge also processes valid Type 0 Requests targeting its own Configuration Space.

This describes externally visible routing, without assuming the Switch's internal logical Bus is another physical PCIe Link. Targets outside valid ranges follow UR handling. (Base 6.0 §7.3.2–§7.3.3, pp.912–913.)

Where does an Endpoint's Requester ID come from?

The preceding section routes a Configuration Request to an Endpoint; now look at Requests the Endpoint initiates. Non-Flit Configuration Request Figure 2-33 contains both Requester ID and Destination ID. The former identifies the initiator of the Configuration Request; the latter identifies its target Function. Do not use the host's Requester ID from that Configuration Write as the Endpoint's own future Requester ID.

Teaching example: Use ordinary non-ARI Endpoint Function 03:00.0, without Shadow or Phantom Functions. Assume the configuration path is established and the host sends a valid Type 0 Configuration Write to this Function, which completes successfully.

Section 2.2.6.2 requires a Function to capture the Bus and Device Numbers from Type 0 Configuration Write Requests it completes and use the captured values in Requests it initiates. The specification recommends capturing only from successfully completed Requests; this example assumes success. The Function Number still identifies the Function, so its ordinary Requester ID here is 0300h, not the Requester ID of the host's Configuration Write.

If the platform later renumbers the Bus, first coordinate stopped work and outstanding transactions under §6.4. Assume the new configuration assigns the same Function 04:00.0, and it completes a new Type 0 Configuration Write. The Function must recapture the Bus/Device values and use the updated identity in subsequent Requests it initiates. This example does not authorize changing numbering while transactions remain active.

Before its initial Configuration Write, a Function must not initiate Non-Posted Requests, because their Completions need a valid Requester ID. The specification provides an exception for Functions within a Root Complex accessing system boot devices; it does not apply to this ordinary Endpoint example. A completed configuration write alone does not establish readiness for DMA: Bus Master Enable, resources and device initialization must also be checked.

DMA
Direct Memory Access, device access to memory

For ARI (Alternative Routing-ID Interpretation) Bus capture, see Chapter 26. Flit Mode Segments and Shadow Functions also have separate identification rules; consult §2.2.6.2 rather than extending this example’s BDF decomposition to them.

PCI Express Base 6.0 §2.2.6.2, printed/PDF pp.156–157; §2.2.7.1 and Figure 2-33, pp.164–165; §6.4 Device Synchronization, pp.717–718.

How does a Flit Mode Configuration Write supply a Segment?

The preceding example identifies an Endpoint within one Hierarchy using Bus, Device and Function. Flit Mode also has separate Segment fields for identification and routing across Hierarchies. In the §2.2.1.2 Segment Rules, a Configuration Request sent by a Root Port in Flit Mode must include OHC-A3 with Destination Segment Valid (DSV) set and the correct Destination Segment, even in a single-Segment system.

Teaching example: A Root Port directly connects to ordinary Endpoint 03:00.0 over a Link operating in Flit Mode. The platform assigns that Hierarchy Segment 02h and sends a legal Type 0 Configuration Write. Assume all bytes of its 1-DW write are enabled and the chosen Register permits this write; Payload and other Header Base fields are omitted.

Use Figure 2-6 to locate the Header Base, then Table 2-6 to select OHC-A3 for a Configuration Request. Figure 2-9 shows a 1-DW OHC-A3: byte 0 is Destination Segment, byte 1 is reserved, byte 2 bit 7 is DSV, and byte 3 contains Last/First DW Byte Enable. The example bytes 02 00 80 0F encode Segment=02h, DSV=1, Last DW BE=0000b and First DW BE=1111b; remaining reserved bits are zero in this simulated packet.

DW
Double Word, a four-byte unit

If the Endpoint successfully completes this Flit Mode Type 0 Configuration Write, §2.2.6.2 requires it to capture Destination Segment. For the Device 3 Extended Capability, assume its base has been found at 200h. Figure 7-124 and Table 7-110 place Device Status 3 at relative offset 0Ch, so read 20Ch. Segment Captured is read-only bit 3 and takes the received DSV value of 1. That bit reports a valid capture; it does not contain the Segment value. The assumed 02h came from the Configuration Request.

Suppose this Configuration Write's Completion includes OHC-A5 because another applicable condition requires it. Figure 2-11 and the Segment Rules give Completer Segment=02h from the capture, while a Configuration Completion has DSV clear and Destination Segment=00h. OHC-A5 can be omitted when its applicable omission conditions hold; a Request DSV of 1 does not require every Completion to copy that DSV.

If the Function receives a Configuration Write in Non-Flit Mode, §7.7.9.4 requires Segment Captured to clear. That does not prove the Segment value is zero; when DSV is clear the corresponding Destination Segment field is Reserved. A status snapshot alone cannot reconstruct the entire history of path modes.

Capturing a Segment, having a Requester ID and being permitted to initiate DMA are separate checks. Whether a Memory Request includes Requester Segment depends on its type and the §2.2.1.2 conditions; Segment Captured=1 does not mean every Memory Request must carry OHC-C.

DMA
Direct Memory Access, device access to memory

PCI Express Base 6.0 §2.2.1.2, Figures 2-6, 2-9, 2-11 and Table 2-6, printed/PDF pp.123, 131–133, 134–135 and 140; §2.2.6.2, p.157; §2.2.9.2, p.199; §§7.7.9.1 and 7.7.9.4, Figures 7-121 and 7-124, Tables 7-107 and 7-110, pp.1072–1073 and 1076–1077.

Where should we look when a known BDF cannot be read?

Use the same topology and target Endpoint 03:00.1. Identify the branch leading to it, then inspect the Secondary and Subordinate Bus Numbers of each Bridge from the root downward. This is a reading sequence for the evidence, not a claim about the order in which hardware processes a Request.

Use Figure 7-14 to locate Secondary and Subordinate; read §7.5.1.3.3–7.5.1.3.4 for their meanings, then §7.3.3 for Type 1 forwarding and conversion. For ECAM writes, §7.2.2 requires a platform method for software to ensure the Completer has finished the write; it does not require one particular method. (Base 6.0, printed/PDF pp.908, 912–913, 936–937.)

How is an absent Function different from an unimplemented Register?

When PCIe Completion information is available, §7.3.1 requires Unsupported Request (UR) for an unimplemented Bus, Device or Function. A read of an unimplemented Register within an implemented Function returns Successful Completion and zero data under §7.3.3, provided no other reportable error applies. The target and the result differ.

When only a software read value is available, first establish whether access succeeded, which BDF and offset were read, and how the platform reports transaction results. Under the conditions described in §2.3.2, a Root Complex synthesizes all-one data from UR for software that relies on it; that data is not carried in the UR Completion. Interpret RRS using the readiness scenario above. (Base 6.0, printed/PDF pp.220–221, 911, 913.)

Discovery does not finish resource and workload setup

Enumeration provides Function inventory, bridge paths, and identity clues, not proof that MMIO or DMA is ready. Firmware 3.3 §3.5 (pp.34–35) requires the specified boot/console paths and all Host Bridges to be configured; other devices may be left to the OS. An arbitrary unconfigured Endpoint alone does not establish a firmware violation.

MMIO
Memory-Mapped I/O, mapped device-register access
Can 0xFFFF0001 be treated as Device ID FFFFh when selecting a driver?

Not under the example's RRS conditions. It is synthesized unreadiness data, not actual identity. Handle readiness and retry instead of initializing an unknown product. (§2.3.2, p.220.)

Is a final Subordinate above 04 necessarily an enumeration error?

No. Check the platform's granted range, branch consistency, and reservation policy. 04 is this example's finalized result, not a universal constant.

Continue with Chapter 4 on BARs and resource allocation.

Jia-Chang

Jia-Chang

Human

Comments

  Write a comment ...