LLVM on RDNA: WMMA dependencies and waits

RDNA 3 / 3.5 / 4 · Pinned compiler analysis

Read the compiler’s decisions alongside their target, instruction and visibility conditions. The companion architecture comparison covers ISA rules, lane layouts, data movement and hardware differences.

1. Findings and scope

Targets: RDNA 3 gfx1100–gfx1103, RDNA 3.5 gfx1150–gfx1153, and RDNA 4 gfx1200/1201; gfx1170 is outside scope. Wave-size conditions follow each case.

Basis: LLVM 4e9d349ab86d, source review of recognized instructions in the shown pipelines. Compilation, regression tests and GPU execution were not performed.

Overlapping WMMA registers: an unseparated D → A/B dependency gets one vector no-op; GFX12 also checks the current sparse index. D → C alone does not trigger that fix, and its hardware stall conditions still apply. Overlap handling · C-only handling.
Memory-produced operands: wait insertion uses modeled register dependencies and pending events. It can emit no new wait or a thresholded wait, rather than always waiting for all memory operations. Operand readiness · Counter selection.
Optional ALU delays: delay insertion requires hasDelayAlu() and skips functions limited to one wave per execution unit. Its C-reuse calculation does not relax the ISA spacing or stall conditions. Eligibility and calculation.

Check these boundaries when applying the result: dependencies hidden inside an inline-assembly string are not established as automatically repaired. Supported GFX12 expert scheduling also tracks extra dependency fields and can emit S_WAITCNT_DEPCTR.

2. Operands and wait types

For a matrix operation D = A × B + C, A/B are multiplicands, C is the accumulator input and D is the result. In a dependency pair, D refers to the preceding instruction; A/B/C refer to the following one. A register hazard is about overlapping physical registers, not merely equal matrix values. ISA register-dependency conditions.

Register spacing: an unseparated D → A/B overlap needs the applicable WMMA separator; RDNA 4 also has a sparse-index case. Scalar S_NOP and memory-counter waits do not provide the required intervening VALU. ISA spacing and stall table.
Memory readiness: an asynchronous load that produces an operand needs its completion tracked separately. Memory counters do not provide a WMMA arithmetic-completion counter. ISA completion rules.
Optional ALU delay: LLVM’s delay calculation and the ISA’s D → C stall conditions are separate. Absence of an inserted delay does not establish zero-stall execution. C-only case · Delay calculation.

The architecture comparison retains the lane layouts and the dependency-and-wait diagram.

3. Target support and tools

Native-instruction comparisons: SWMMAC, FP8 conversions, and packed global atomics. Their LLVM status uses the exact targets and default features at the pinned revision.

Feature	RDNA 3	RDNA 3.5	RDNA 4
LLVM generic processor grouping	RDNA 3 and RDNA 3.5 The covered targets are listed under `gfx11-generic`. This is LLVM’s generic-target grouping; the same table lists processor-specific restrictions. LLVM: processors		The covered targets are listed under `gfx12-generic`. LLVM: processors
Machine-readable ISA	RDNA 3, RDNA 3.5, and RDNA 4 AMD lists a separate XML ISA specification for each generation and IsaDecoder support for all three. AMD: architectures · IsaDecoder API

LLVM documents specific processors and generic processors separately; the generic-processor table gives supported targets and restrictions. LLVM generic processors.

AMD’s XML specifications contain instructions, encodings, operands, data formats, and human-readable descriptions. AMD machine-readable ISA documentation.

Native availability and evidence scope

The SWMMAC, FP32-to-FP8/BF8, and packed 16-bit global-atomic comparisons report two results separately: LLVM native-instruction support and entries in AMD’s XML inventory. The LLVM result uses default features for the exact listed targets at the pinned revision; an XML omission alone does not establish hardware absence.

Use the evidence conventions to interpret these labels. The RDNA architecture article, Appendix A, records the XML inventory checks and pinned processor-feature bindings. Its architecture evidence-status table separately tracks presentation-level memory and register-write claims.

4. Pass pipeline

The shown AMDGPU pipelines include SIInsertWaitcnts followed by the standalone post-register-allocation hazard recognizer at every optimization level; AMDGPUInsertDelayAlu is separately conditional. The AMDGPU recognizer calls fixWMMAHazards. Legacy pipeline · New pass-manager pipeline · WMMA fix invocation.

5. Register dependency cases

D → A/B; RDNA 4 D → sparse index

Detection

Physical-register overlap between recognized WMMA/SWMMAC instructions.

Compiler action

One V_NOP_e32 for unseparated D → A/B overlap; GFX12+ also checks the current SWMMAC index.

Scan and separator

Scan: backward across predecessor blocks, expiring at an intervening instruction classified by LLVM as VALU.

Scalar instructions: S_NOP and memory-counter waits do not satisfy the intervening-VALU predicate.

Boundary

The WMMA hazard fix implements the register-spacing rule; it does not test memory completion.

Sources by check

Detection
LLVM: overlap fix
Compiler action
LLVM: overlap fix
Scan and separator
LLVM: scan · LLVM: spacing
Boundary
LLVM: scan · LLVM: spacing

D → C only

Compiler action

V_NOP: D → C alone does not trigger this WMMA hazard fix.

Regression evidence

The cited GFX11 tests expect no added V_NOP for D → C cases with differing opcodes or source modifiers.

Hardware boundary

Hardware stalls: absence of an inserted V_NOP is not a zero-stall guarantee. RDNA 3.5 / 4 retain their D/C-overlap, instruction-type, and SRC2-modifier conditions.

Combined overlap: D → A/B overlap (or RDNA 4 sparse-index overlap) still invokes the WMMA spacing rule.

Sources by check

Compiler action
LLVM: return paths
Regression evidence
LLVM: D → C tests
Hardware boundary
ISA p.86 · ISA p.101

6. Memory-produced operands

Memory read → WMMA operand

Check: machine-instruction register uses/definitions against pending event scores.

Coverage: the generic operand loop includes recognized WMMA inputs and register-reuse hazards involving pending writes or asynchronous source reads.

Wait decision

Threshold: depends on outstanding events and the register score.

Outcomes: no new wait for a satisfied dependency; potentially nonzero for ordered completion; potentially zero for out-of-order cases.

Boundary

The operand-wait logic has no unconditional full-memory wait before every WMMA and no dedicated WMMA-completion counter.

Sources by check

Register analysis
LLVM: operands
Wait decision
LLVM: thresholds · LLVM: scores
Boundary
LLVM: thresholds · LLVM: scores

Memory-counter selection

Instruction encoding

Before GFX12: internal LOAD_CNT/DS_CNT encode VMcnt/LGKMcnt in S_WAITCNT.

GFX12: separate counter instructions; LOADcnt or STOREcnt can be combined with DScnt when both need waits.

Producer classification

Producer classes: regular VMEM reads and indexed LDS accesses use their corresponding counters; direct LDS uses EXP_LDS_ACCESS.

FLAT: classified by possible VMEM and/or LDS access.

Naming boundary

Shared compiler field names do not rename pre-GFX12 ISA counters.

Sources by check

Instruction encoding
LLVM: legacy fields · LLVM: S_WAITCNT · LLVM: GFX12 waits
Producer classification
LLVM: GFX11 events · LLVM: GFX12 events · LLVM: direct LDS · LLVM: DS / FLAT
Naming boundary
LLVM: legacy fields · LLVM: S_WAITCNT · LLVM: GFX12 waits

7. Developer checks and optional controls

Inspect the actual target and emitted instruction sequence in this order:

Target and pipeline: match the processor, compiler revision, function attributes and options to the target scope and reviewed pass pipeline.
Register overlap and spacing: identify physical D → A/B overlap and, on RDNA 4, the sparse-index input; inspect the intervening VALU or inserted vector no-op. Check D → C stall conditions separately. Overlap handling · C-only handling.
Memory readiness: identify the operand’s producer, pending dependencies and emitted wait counters. A memory-counter wait does not supply the WMMA register separator. Operand readiness · Counter selection.
Inline assembly: check the exposed register operands and the internal instruction sequence; do not assume LLVM repairs dependencies hidden inside the string. Coverage boundary.
Optional controls: check ALU-delay eligibility and whether supported GFX12 expert scheduling is enabled. Absence of an optional delay does not remove mandatory spacing or establish zero-stall execution.

Inline assembly

Compiler handling

WMMA fix: requires recognized WMMA/SWMMAC opcodes; its backward scan skips inline assembly.

Other handling: the dedicated inline-assembly checker limits its coverage; wait insertion analyzes modeled register operands and events.

Boundary

These paths do not establish automatic repair of dependencies hidden inside an assembly string.

Checking exposed operands is distinct from analyzing internal instructions. The ISA rules inside the string remain unchanged.

Sources by check

Compiler handling
LLVM: opcodes · LLVM: asm scan · LLVM: asm limits · LLVM: operands
Boundary
LLVM: asm limits · LLVM: operands

Optional S_DELAY_ALU and C reuse

Pass eligibility

Requires hasDelayAlu(); skips functions with a maximum of one wave per execution unit.

C-reuse calculation

Excludes a tied explicit source equal to the remembered preceding WMMA/SWMMAC destination from delay calculation.

Predicate: register equality and current instruction class; no opcode-type or SRC2-modifier comparison.

Tracking

Another processed instruction that is neither WMMA nor SWMMAC clears the remembered destination.

Hardware boundary

Meaning: an optional compiler delay calculation, not a broader hardware-forwarding guarantee.

Neither the exclusion nor absence of S_DELAY_ALU removes mandatory WMMA spacing or ISA D → C stall conditions.

Sources by check

Pass eligibility
LLVM: eligibility
C-reuse calculation
LLVM: C reuse
Tracking
LLVM: tracking
Hardware boundary
ISA p.54 · ISA p.86 · ISA p.101

GFX12 expert-scheduling exception

GFX12 expert-scheduling exception: SIInsertWaitcnts is not exclusively a memory-wait pass in every mode.

Enabled modeWith supported expert scheduling enabled by the option/function-attribute selection, LLVM also tracks VALU VGPR reads/writes and memory-instruction VGPR source reads, and can emit S_WAITCNT_DEPCTR.

VALU consumersIts VA_VDST waits are cleared before VALU consumers because the source identifies that dependency class as hardware-handled.

Dependency fieldsThese extra dependency fields are separate from the memory-completion counters above and from fixWMMAHazards.

Sources for this noteGFX12+ support gate · Enablement · Extra events · DEPCTR emission · VALU-consumer exception.

8. Evidence and validation

Reviewed revision: LLVM 4e9d349ab86dc0a913db3cba11aa2763d6985aff. All source and regression-test links for the WMMA handling analysis use this commit. Source-review date: November 13, 2026. This is an independently organized analysis.

Completed checks

Source inspected: target and feature bindings, the shown pass pipelines, WMMA overlap handling, memory waits, and optional controls and inline-assembly boundaries.

Target coverage follows the processor bindings, shared feature sets and recognizer’s common code path; it is not a per-chip execution result. The underlying bindings are recorded in the RDNA architecture article, Appendix A.

Regression expectations inspected: pinned post-RA cases cover GFX11 D → A/B and intervening VALU, GFX12 wave32 D → A/B/C, GFX12 sparse index, and GFX12 wave64. These are checked-in expectations, not locally executed test results. Wave-size conditions remain specific to each case.

Document checked: site rendering, local links and legacy fragments, disclosure behavior, and desktop/narrow-screen layout. These checks validate the document’s presentation and navigation.

Applicability and validation limits

Not performed: kernel compilation, LLVM regression execution, GPU execution, or GPU correctness/performance benchmarking for this analysis. No result here depends on those tests having passed.

Compiler actions apply to recognized machine instructions in the reviewed pipelines. This source snapshot does not establish behavior for every ROCm release, downstream compiler or custom pipeline. Dependencies hidden inside an assembly string remain subject to the inline-assembly boundary.

Evidence labels and scope

Read each status with its named targets, source revision and instruction scope. These four checks answer separate questions.

ISA inventory: Listed / Not listed reports whether the specified instruction appears in the cited manual or machine-readable inventory. The source is named explicitly, for example AMD XML. An omitted entry does not establish hardware absence.
LLVM target support: Enabled / Not enabled refers to the required builtin or native-instruction features in the exact processor's default feature set at the pinned revision. Forced overrides, software implementations, library dispatch and runtime support are separate questions.
Mechanism: Documented applies only to the behavior described in the cited source. Details unspecified means that source leaves the relevant mechanism or conditions open. Mechanism equivalence unconfirmed means the evidence does not establish that two reported changes share an implementation.
Validation: Source inspected, CPU checked and GPU tested identify the work actually completed, with its stated coverage. Not executed means the named test was not run. Source review, including reading regression expectations, is separate from executing those tests.

Feature reports: Confirmed by AMD means AMD explicitly reports the feature for the named generation or product. Support unconfirmed means the reviewed evidence does not establish presence or absence. Feature confirmation can coexist with unspecified implementation details.

Derived identifies arithmetic or indexing calculated from the cited definitions. Unmeasured identifies a performance or accuracy claim without a local GPU measurement. Neither label is a substitute for a test result.