PCIe Chapter 6: DMA, Visibility, and Transaction Ordering

Connect descriptors, DMA data and completion flags

DMA

Direct Memory Access, device access to memory

DMA

Direct Memory Access, device access to memory

DMA (Direct Memory Access) allows a device to initiate memory transactions. A working cycle also needs buffer ownership transfer and platform synchronization. This chapter connects descriptor publication, device writes and host consumption.

Specification route: DMA mapping and addresses → descriptor publication → transaction ordering in Table 2-42 → packet attributes and host ownership.

DMA needs addresses reachable by the device

The device reads host memory with MRd and receives CplD, or writes results with ordinary MWr without an ordinary Write Completion. BME permits relevant Request generation/forwarding; it does not create buffers or DMA mappings.

Linux 6.12 distinguishes CPU virtual, CPU physical, and DMA addresses. The authored example maps CPU pointer p to physical 0x90001000–0x90001FFF and device DMA range 0x40001000–0x40001FFF. Thus DMA 0x40001080 reaches physical 0x90001080. These numbers define no universal conversion. BAR0 at 0x80004000–0x80007FFF is a separate resource.

Coherence and access ordering need separate treatment

Assume a descriptor contains DMA address 0x40001000, length 256 bytes, and OWN. The device interface permits reading the other fields after acquiring OWN. The CPU must publish the fields before ownership, then notify the device through appropriate MMIO access. It must also follow platform rules when reclaiming data after the device releases ownership.

MMIO

Memory-Mapped I/O, mapped device-register access

Base 6.0 §2.4.2 does not make a single MRd an atomic snapshot of changing data. Linux 6.12 coherent memory can still need barriers; dma_* barriers do not order MMIO. Streaming mappings require their DMA API synchronization. PCIe ordering cannot replace these responsibilities.

MMIO

Memory-Mapped I/O, mapped device-register access

Use Table 2-42 to decide whether later traffic can pass earlier traffic

Assume the device writes 64-byte data D at 0x40001000, then writes flag F=1 at 0x40001FF0. In Table 2-42, columns represent earlier transactions and rows later ones. Both are Posted Requests, selecting A2.

Yes requires the later transaction to be able to pass to avoid deadlock; No prohibits passing; Y/N permits implementation choice. These labels do not indicate transaction success. Ordered packets do not perform CPU-side ownership transfer and synchronization automatically.

Does an interrupt mean the CPU can already use the data?

Keep data write D from the preceding section and replace the later action with an MSI-X (Message Signaled Interrupts Extended) transaction. First inspect PCIe ordering, then check how the platform makes the result available to the CPU. Each step needs its own evidence.

MSI-X

Message Signaled Interrupts Extension

Reading route: §6.1.4 defines MSI-X as a Memory Write and requires RO and No Snoop clear; IDO may be set when enabled, but is explicitly clear in this example. In §2.4.1 Table 2-42, locate earlier D by column and later MSI-X by row, then read A2a/A2b. Finally read the §6.1.4 implementation note on synchronizing data and interrupt traffic across TCs. (Base 6.0, printed/PDF pp.222–223, 661.)

Platform context: a Linux driver still follows the buffer’s mapping method and device interface for data and ownership. For example, DMA barriers for consistent memory and MMIO accesses have different scopes. See Linux 6.12 memory barriers and the DMA mapping guide; these are platform documents, not Base 6.0 rules for every operating system.

A control bit grants permission; it does not describe every packet

With PCIe Capability at 0x70, Device Control is at 0x78. An assumed value of 0x2810 enables conditional use of RO and No Snoop through bits 4 and 11; bits 14:12=010b set MRRS to 512 bytes. It does not assert RO or No Snoop in every TLP.

TLP

Transaction Layer Packet

Inspect the actual packet's Attr[1] for RO, Attr[0] for No Snoop, and Attr[2] for IDO; these bits are not all adjacent in a Non-Flit header. No Snoop does not change ordering and is not a cache-flush command. Its use still requires a guarantee that the target address is not cached anywhere in the system. IDO has separate enable and identity conditions and stays clear here. MSI/MSI-X specifically require RO and No Snoop clear.

MSI

Message Signaled Interrupt

Check the scope of the conclusion