PCIe Chapter 29: Enhanced Allocation, Resizable BARs, and Topology Translation

From supported BAR sizes to an allocatable range

Find Extended Capability ID 0015h, Version 1. Figure 7-174 locates a Capability and Control pair for each resizable BAR; the first Control gives the number of pairs. Figure 7-177 maps BAR Index 0 to Configuration Space offset 10h. For a 64-bit BAR the index identifies its lower DWORD.

Simulated data: an Endpoint has a 32-bit Memory BAR0 and one pair at capability base R. Capability R+04h=00005000h sets bits 12 and 14, indicating 256 MiB and 1 GiB. Control R+08h=00000820h identifies BAR0, one resizable BAR and BAR Size=8, the current 256 MiB selection. Larger-size support bits in Control are zero. The platform currently has 90000000h–9FFFFFFFh available, a naturally aligned 256 MiB range, but no available 1 GiB range.

Table 7-153 encodes size as 2^{20+BAR Size} bytes: 8 selects 256 MiB and 10 selects 1 GiB. Writing an unsupported encoding has undefined results. A 32-bit BAR may support 1 GiB; sizes of 4 GiB or more require a 64-bit BAR.

Assume the platform later makes C0000000h–FFFFFFFFh available and configures the ancestor Bridge Window. Stop resource use according to device/platform requirements, clear Memory Space Enable, write BAR Size 10, and reprogram the now-undefined BAR to C0000000h. Verify the path before setting Memory Space Enable. Leave decode disabled if the resource is not allocated. See Chapter 4 for bridge windows.

A PF's ordinary Resizable BAR does not change SR-IOV VF BAR settings. The separate VF Resizable BAR Capability, if implemented, is in the PF's Configuration Space, not in the VF. Clear VF MSE before changing VF BAR Size and reprogram the VF BAR afterwards; System Page Size also affects the inferred size. See Chapter 26.

PF

Physical Function

SR-IOV

Single Root I/O Virtualization

VF

Virtual Function

PCIe Base 6.0 §7.8.6, Figures 7-174, 7-176, 7-177 and Tables 7-152–7-153, printed/PDF pp.1134–1142; §7.8.7, p.1142.

Read an EA entry before counting resources

EA

Enhanced Allocation

Enhanced Allocation (EA) applies to eligible permanently connected topologies. Use Figure 6-26 for applicability and §7.8.5 for the entry layout. A Type 0 Function starts entries at capability offset 04h; a Type 1 Function has a second DWORD and starts at 08h.

Simulated data: a separate permanently connected Type 0 Function has an EA capability at E. Its first three DWORDs are 80FF0022h, A0000000h and 0000FFFCh. Figure 7-168/Table 7-148 gives Entry Size=2: two DWORDs follow the first, so the next entry begins at E+10h. BEI (BAR Equivalent Indicator)=2 identifies the BAR at offset 18h; Enable=1 and Writable=0. Primary Properties 00h identifies non-prefetchable memory; Secondary Properties FFh requires use of the Primary definition.

Figure 7-169 identifies both Base and MaxOffset as 32-bit fields. Restore Base's low address bits as 00b and MaxOffset's low bits as 11b: A0000000h + 0000FFFFh gives the inclusive range A0000000h–A000FFFFh, or 64 KiB. The equivalent BAR2 must be hardwired to zero for the resource described by EA; do not allocate the same resource a second time.

Disabling the entry does not free the resource for another Function. Bridge EA need not list every downstream resource, so software must still search downstream Functions. Memory and I/O decode-enable rules still apply.

PCIe Base 6.0 §§6.23, 7.8.5.2–7.8.5.3, Figure 6-26, Figures 7-168–7-169, Tables 7-148 and 7-150, printed/PDF pp.810–812, 1129–1133.

Hierarchy identity and FPB allocation answer different questions

FPB

Flattening Portal Bridge

Section 6.25 combines System GUID and Authority ID, Hierarchy ID, and Routing ID. In a simulated system with valid system identity, Hierarchies 0001h and 0002h may each contain Routing ID 0300h; these are different Functions. Authority ID 00h makes System GUID not meaningful and cannot establish a valid global system identity.

A Downstream Port sends the Hierarchy ID Message at software request; the receiving Upstream Port can report it through Hierarchy ID Extended Capability. Figures 2-55/2-56 define the Non-Flit/Flit Mode Message; §7.9.17 defines the receiving Capability. Platform use is outside this specification's scope.

FPB (Flattening Portal Bridge) addresses allocation of Routing IDs and Memory Space. Figure 6-28 locates logical bridges, Figure 6-29 illustrates Switch flattening, and Figure 6-30 shows how a bit vector assigns resource bins to the Secondary Side. This optional mechanism needs explicit support and configuration.

PCIe Base 6.0 §2.2.8.6.4, Figures 2-55/2-56, printed/PDF pp.185–186; §6.25, pp.815–816; §6.26.1, Figures 6-28–6-30, pp.819–821; §7.9.17, pp.1268–1271.

Where is the System GUID in a Hierarchy ID Message?

Start with §2.2.8.6.4 and Table 2-31: this PCI-SIG Defined Type 1 VDM uses Message Code 7Fh, Vendor ID 0001h, and Subtype 01h. It uses MsgD with Length 4 DW, or 16 bytes. TC is 000b, Attr and Tag are Reserved, and Routing 011b means Broadcast from Root Complex. For the difference between Message Code and Subtype, see Chapter 36.

DW

Double Word, a four-byte unit

TC

Traffic Class

Select Figure 2-55 for Non-Flit Mode or Figure 2-56 for Flit Mode. Hierarchy ID occupies header bytes 8–9; byte 13 is GUID Authority ID; bytes 14–15 contain the high 16 bits of System GUID. Payload bytes 16–31 contain its low 128 bits. The full System GUID is therefore 144 bits: preserving only the payload loses the upper bits. Section 6.25 maps each byte to its GUID bits.

For identity comparisons, §6.25 requires comparing all bits of Authority ID, System GUID and Hierarchy ID; combine these with Routing ID within the hierarchy. Under Table 6-17, Authority ID 00h makes System GUID not meaningful and requires it to be zero; it cannot establish a valid global system identity. Receiver format checks are independently optional and must not inspect Reserved bits; a violation detected by an implemented check is reported as Malformed TLP at the receiving Port.

TLP

Transaction Layer Packet

Edition and sources

This chapter uses PCI Express Base Specification Revision 6.0 (6.0-1.0-PUB). Document and PDF page numbers match. Example configurations and values are author assumptions, not device measurements.