LLVM on CDNA: MFMA dependencies and waits
CDNA 3 / 4 · gfx942 / gfx950 · Pinned compiler analysis
Read the compiler’s decisions alongside their target, instruction and visibility conditions. The companion architecture comparison covers ISA rules, lane layouts, data movement and hardware differences.
1. Findings and scope
Targets: CDNA 3 gfx942 and CDNA 4 gfx950, with wave64 matrix operations. The worked dependency table uses the shared FP16 16×16×16 instruction and VGPR C/D operands; other opcodes and register files need their own checks.
Basis: LLVM 4e9d349ab86d, source review of recognized instructions in the shown pipelines. Compilation, regression tests and GPU execution were not performed.
- Exact D → C reuse has a conditional zero case; partial C and A/B overlap use different thresholds. For the shared FP16 P = 4 example, the paired values are 0/0, 5/6 and 7/8 compiler wait states for gfx942/gfx950. Cases and full conditions.
- Already-counted spacing reduces the remaining deficit. The threshold is not a fixed number of NOP instructions and is not measured GPU latency. Distance accounting.
- Builtin availability and dependency handling are separate checks. Match the exact opcode, target features and operand contract. Target interfaces.
Memory readiness is separate: the wait-counter path tracks load-produced operands and direct-to-LDS writes; MFMA spacing does not replace it.
Automatic handling has a visibility boundary: dependencies hidden inside an inline-assembly string are not guaranteed to be repaired. Audit the register constraints and internal sequence when applying these rules to hand-written assembly.
2. Operands and wait types
For D = A × B + C, A/B are multiplicands, C is the accumulator input and D is the result. In the cases below, D belongs to the producer; A/B/C belong to the consumer. Physical register overlap is different from equal matrix values, and MFMA can continue reading inputs after issue. ISA dependency background.
v[0:3] names four physical VGPRs, v0–v3. Exact reuse means the same register tuple, including its extent. A shared starting register or equal values alone do not establish exact reuse. Opcode and tuple conditions.
Three checks remain separate: MFMA register dependencies use instruction-specific scheduling; memory counters track asynchronous producers; a workgroup barrier coordinates participating waves. A barrier instruction does not implicitly drain outstanding memory counters. ISA completion and barrier conditions.
The architecture comparison retains the wave64 lane layouts and the dependency overview diagram. The dependency-threshold table in §5 uses wait states from LLVM’s distance model, not a GPU timing measurement.
3. Target support and interfaces
In the table, LLVM: Enabled / Not enabled refers to the builtin’s required features in the processor’s default feature set at the pinned revision. The shared evidence definitions separate this from runtime or library adoption. Forced feature overrides are outside this comparison.
| Interface | CDNA 3 · gfx942 | CDNA 4 · gfx950 |
|---|---|---|
| Common FP16 MFMA16×16×16 | CDNA 3 · gfx942LLVM: Enabled | CDNA 4 · gfx950LLVM: Enabled |
| Wider FP16 MFMA16×16×32 | CDNA 3 · gfx942LLVM: Not enabled | CDNA 4 · gfx950LLVM: Enabled |
| Scaled MFMAF8F6F4 | CDNA 3 · gfx942LLVM: Not enabled | CDNA 4 · gfx950LLVM: Enabled |
| Transpose LDS loadsds_read_tr* | CDNA 3 · gfx942LLVM: Not enabled | CDNA 4 · gfx950LLVM: Enabled |
Target bindings: gfx942 / gfx950 processors, shared matrix feature, and generation-specific features. The gfx950 definitions also enable the larger LDS resource model.
Builtin contracts and source gates
Names below omit the __builtin_amdgcn_ prefix. Vector lengths are per lane; they do not prescribe a global-memory layout or validate tail handling.
- Common FP16 MFMA
mfma_f32_16x16x16f16takes four FP16 values for each A/B input and four FP32 accumulator values, and returns four FP32 values. Its gate ismai-insts. This is the shared layout example. Clang declaration.- Wider FP16 MFMA
mfma_f32_16x16x32_f16takes eight FP16 values per A/B input and four FP32 accumulator values, and returns four FP32 values. Its gate isgfx950-insts. Adopting it changes the input contract; changing the compile target alone does not select it. Clang declaration.- Scaled MFMA
mfma_scale_*_f8f6f4takes 8×32-bit packed vectors for A/B, with separate format selectors and scale arguments. Its gate isgfx950-insts. Check the packing boundary and scale mapping for the selected format. Clang declarations.- Transpose LDS loads
ds_read_tr4/6/8/16interfaces requiregfx950-insts. Select the pointer and return types together with the LDS layout and element format. These declarations establish compiler interfaces; runtime and library adoption require separate checks. Clang declarations.
4. Pass pipeline
The pinned legacy and new pass-manager pipelines run SIInsertWaitcnts before an independent post-register-allocation hazard pass. The latter checks instructions in their final positions, including scheduling-region boundaries. This establishes an automatic handling path for recognized machine instructions and visible dependencies. Legacy pipeline · New pipeline.
MFMA register hazards take a separate path through GCNHazardRecognizer. Its rules inspect opcodes, operand roles and physical register overlap, then account for intervening instructions. The interface declarations establish types and target gates; they do not select a wider instruction automatically. MFMA input checks.
5. One instruction, three dependency cases
Reusing a result as C differs from reusing it as A/B. Consider the shared V_MFMA_F32_16X16X16F16, using VGPR C/D operands. It belongs to LLVM’s XDL class of matrix instructions and is modeled as P = 4 passes on both targets. Here P comes from the pinned scheduling model; these thresholds are compiler wait states, not measured GPU clock latency. Pass definition · gfx942/gfx950 instruction mapping.
Read the register ranges: v[0:3] means the four physical VGPRs v0–v3, also called a register tuple. Each example keeps the preceding result in that range and changes the following instruction’s input.
| Dependency on the producer's D | CDNA 3 · gfx942 | CDNA 4 · gfx950 |
|---|---|---|
Exact D → C reuseSame normalized MFMA; identical register rangeD=v[0:3] → C=v[0:3] | CDNA 3 · gfx9420 for this dependency | CDNA 4 · gfx9500 for this dependency |
Partial D → C overlapXDL consumer; overlapping, unequal register rangesD=v[0:3] → C=v[2:5] | CDNA 3 · gfx9425 = P + 1 | CDNA 4 · gfx9506 = P + 2 |
D → A/B overlapThe result becomes a multiplicand inputD=v[0:3] → A=v[2:3] | CDNA 3 · gfx9427 = P + 3 | CDNA 4 · gfx9508 = P + 4 |
Scope: the listed producer, LLVM-recognized operands and the stated overlap. A/B inputs must not introduce an additional dependency in the C-only rows. Zero applies only to the listed D → C relationship, not to every hazard in the instruction stream. The partial-C and A/B values follow different helpers; do not transfer them to every MFMA shape, register file or modifier combination. Partial-C and A/B rules · Full-C and operand selection.
A threshold is not an unconditional NOP count. LLVM subtracts the already-counted distance and considers earlier relevant producers within the scan window. Its distance model counts an ordinary non-meta instruction as 1 and S_NOP n as n + 1; inserted spacing covers the remaining deficit. Distance accounting · Dependency-window scan.
Why exact C reuse needs conditions
“Exact” means the producer destination and consumer C are the same physical register tuple, including its extent. LLVM also compares normalized MFMA opcodes: supported MAC/non-MAC and accumulator-register/VGPR encoding variants normalize to a common operation. Equal matrix values or a shared starting register alone do not establish this path. Opcode normalization · Full-C decision.
For a different normalized opcode, the gfx940-family path also checks the partial-overlap helper. This does not mean every change of opcode needs a nonzero wait: family-specific zero cases remain. For an exact-C chain using the same normalized MFMA opcode, two-pass operations require 2 wait states, and FP64 4×4×4 self-chains require 4. Changing the opcode still invokes the family-specific checks above. Family cases and exceptions.
The table uses one common FP16 instruction. A wider-K or scaled instruction needs its own opcode classification and pass count; the word “MFMA” alone is not enough to reuse these numbers.
The inspected regression expectations distinguish the FP16 partial-C and A/B cases from corroborating INT8 and VALU-input cases. They were not executed.
Input overwrite and result access
A register can still be in use after MFMA issues. Besides MFMA-to-MFMA reads, the recognizer checks an MFMA result being read or overwritten by later VALU/memory instructions, and a later write overwriting an accumulator input still being read. For the XDL P = 4 path, overwriting C has a 3-state write-after-read window on both targets; result-access and write-after-write checks are separate and can require more. Do not treat input lifetime as result latency. Result access and overwrite · C-input lifetime check.
6. Memory-produced operands
A direct VMEM-to-LDS operation produces an LDS value without producing a payload VGPR. Waiting only for ordinary LDS operations is therefore insufficient. The pinned LLVM wait pass tracks these pending writes through the VM counter and LDS aliasing when deciding whether a later LDS access needs a wait. SIInsertWaitcnts, LDS DMA tracking · LDS dependency checks.
For a load returning an MFMA input, the wait pass tracks modeled register uses and pending events separately from MFMA register spacing. Load-result dependency tracking.
For cross-wave use of the resulting LDS tile, also follow the architecture article’s copy-completion and barrier conditions. The memory protocol and the MFMA wait-state table answer different dependency questions.
7. Developer checks and optional controls
When applying this analysis, inspect the actual target and emitted sequence: opcode changes, partial register overlap, reuse of D as A/B, early C overwrite and register-file transitions. Check asynchronous load completion separately; a load returning an MFMA input is tracked by the wait-counter pass. The numbers above do not replace that memory protocol. Load-result dependency tracking · MFMA audit with source locators.
Inline assembly needs an explicit dependency audit. LLVM does not guarantee automatic repair of MFMA dependencies hidden inside an asm string. Supply accurate register constraints and a correct internal sequence, then inspect the resulting assembly. Checker scope.
Why inline-assembly coverage is limited
LLVM's inline-asm hazard checker documents limited coverage and applies feature-specific checks. Distance accounting treats an asm statement as zero because it may be empty. These checks do not establish that LLVM parses the instructions inside every asm string. Checker scope · Asm distance accounting.
Optional scheduling controls
amdgpu-mfma-padding-ratio defaults to 0. Its optional inter-wave performance padding is separate from mandatory dependency spacing; the recognizer combines them by taking the maximum. Likewise, the scheduler's MIMFMARead conservative read model is not an extra universal hardware wait to add to every table entry. Padding default · Padding implementation · Combined requirement · Read-model explanation.
8. Evidence and validation
Reviewed revision: LLVM 4e9d349ab86dc0a913db3cba11aa2763d6985aff. All source and regression-test links for the MFMA handling analysis use this commit. Source-review date: November 13, 2026. This is an independently organized analysis.
Completed checks
Source inspected: gfx942/gfx950 target features and builtin contracts, the shown pass pipelines, MFMA dependency rules and distance accounting, memory waits, and inline-assembly boundaries and optional controls. The MFMA compiler audit records source locators, dependency formulas and instruction-gate boundaries.
Regression expectations inspected: the cases below distinguish the shared FP16 partial-C and A/B examples from corroborating INT8 and VALU-input cases. Source rules and the P = 4 model establish the FP16 table’s exact-C zero; the INT8 regression is not a direct FP16 test. The gfx942/gfx950 RUN lines were inspected, not executed.
Document checked: site rendering, local links and legacy fragments, disclosure behavior, and desktop/narrow-screen layout. These checks validate the document’s presentation and navigation.
Regression expectations inspected, not executed
mai-hazards-gfx942.mir has RUN lines for both amdgpu9.42 and amdgpu9.50, including a gfx950 run with --schedmodel=0. The file name alone understates its target coverage. These are checked-in expectations, not results from this document's validation. RUN lines.
These RUN lines exercise the hazard pass, not instruction availability. In the audit's different-opcode case, the wider FP16 16×16×32 instruction requires HasGFX950Insts; its gfx942 MIR expectation does not establish a compilable or executable gfx942 sequence. The shared 16×16×16 examples below retain their stated scope. Wider-instruction gate · Different-opcode MIR case.
- FP16 partial C
The adjacent
16×16×16pair expectsS_NOP 4on gfx942 andS_NOP 5on gfx950: 5 / 6 wait states. Partial-C case.- FP16 result → A/B
The two adjacent-pair cases expect
S_NOP 6/S_NOP 7: 7 / 8 wait states. The MIR supplies concrete physical registers and modifiers; it is not an executed matrix-layout check. A/B cases.- Exact accumulator chain
A separate INT8
16×16×32case with accumulator registers expects consecutive MFMA instructions without a NOP. The FP16 table's zero is derived from the source rule and P = 4 model; this INT8 case is corroborating coverage, not a direct FP16 test. Exact-C case.- VALU writes an input
An F32
4×4×1case expectsS_NOP 1, matching the 2-state non-dot VALU-input rule. This checks a different producer/consumer relationship from the FP16 rows. VALU-input case.
Applicability and validation limits
Not performed: kernel compilation, LLVM regression execution, GPU execution, or GPU correctness/performance benchmarking for this analysis. No result here depends on those tests having passed.
Compiler actions apply to recognized machine instructions and visible dependencies in the reviewed pipelines. This source snapshot does not establish behavior for every ROCm release, downstream compiler or custom pipeline. Dependencies hidden inside an assembly string remain subject to the inline-assembly boundary.
For kernel-level follow-up, use the CDNA architecture article’s port-validation steps. The CDNA architecture article, Appendix A, separately records architecture-source review and CPU enumeration of the illustrated lane layouts.
Evidence labels and scope
Read each status with its named targets, source revision and instruction scope. These four checks answer separate questions.
- ISA inventory
Listed / Not listed reports whether the specified instruction appears in the cited manual or machine-readable inventory. The source is named explicitly, for example AMD XML. An omitted entry does not establish hardware absence.
- LLVM target support
Enabled / Not enabled refers to the required builtin or native-instruction features in the exact processor's default feature set at the pinned revision. Forced overrides, software implementations, library dispatch and runtime support are separate questions.
- Mechanism
Documented applies only to the behavior described in the cited source. Details unspecified means that source leaves the relevant mechanism or conditions open. Mechanism equivalence unconfirmed means the evidence does not establish that two reported changes share an implementation.
- Validation
Source inspected, CPU checked and GPU tested identify the work actually completed, with its stated coverage. Not executed means the named test was not run. Source review, including reading regression expectations, is separate from executing those tests.
Feature reports: Confirmed by AMD means AMD explicitly reports the feature for the named generation or product. Support unconfirmed means the reviewed evidence does not establish presence or absence. Feature confirmation can coexist with unspecified implementation details.
Derived identifies arithmetic or indexing calculated from the cited definitions. Unmeasured identifies a performance or accuracy claim without a local GPU measurement. Neither label is a substitute for a test result.