The Workload Suite: PyTorch MLPs on TinyTPU¶

Every number this project had before this page was a bare GEMM at a shape somebody chose. That is enough to rank two datapaths and not enough to rank two designs, because a design decision is worth whatever it does to the mixture of shapes a real program actually issues. This page is about the smallest thing that fixes that: four PyTorch MLPs, extracted layer by layer into the workload specs act/corpus/ already uses, mapped onto the fixed TinyTPU by the ACT flow, verified bit-exact, and measured.

It is deliberately small. Four models that run are worth more than twelve that are described, and the point of the suite is not coverage — it is that a workload is now the unit of evaluation.

The sentence to take away first¶

Allo’s PyTorch path synthesises a machine for a model. ACT’s flow maps a model onto a machine. The co-design loop needs the second.

allo/frontend/pytorch.py’s from_pytorch is a real, working PyTorch front end, and it is not the one this suite uses. Its front half traces the module; its back half hands the traced graph to TorchBuilder, which emits Allo DSL source for that model, runs it through customize, composes kernels out of allo/library/nn.py and calls s.build(target=...). The artefact is a bespoke accelerator for the MLP. That is a legitimate and interesting thing to build, and it is the opposite of what an evaluation needs, because a machine that changes with every model cannot tell you what changing the machine is worth.

So the suite uses the front half only:

PyTorch nn.Module
  -> AlloTracer + torch.fx ShapeProp      shapes and op types, nothing more
  -> per-layer workload specs             act/corpus/'s JSON convention
  -> the ACT mapper (act_compile)         onto the FIXED TinyTPU
  -> a TinyTPU program                    verified against isa_ref
  -> cycles                               estimated, then measured

AlloTracer (allo/frontend/tracer.py) is an fx.Tracer subclass and nothing more; ShapeProp is stock torch.fx. Neither TorchBuilder nor customize nor allo/library/nn.py is imported by anything under workloads/.

Why not TOSA¶

The recollection that ACT consumed TOSA extracted from PyTorch is worth answering directly, because the answer is a scoping decision rather than a correction. A workload spec in this project needs five things: the einsum, the extents, the operand and accumulator dtypes, the epilogue, and where each tensor sits in DRAM. For an MLP, fx plus ShapeProp supplies the first four exactly and the fifth is the mapper’s business. Going through TOSA would mean importing torch-mlir, lowering to a dialect, and then reading M, K and N back out of a tosa.matmul — the same five facts, through two more tools, with two more versions to pin.

Where TOSA would start to earn its place:

a wider op set. The moment the suite wants convolutions, pooling, normalisation or quantisation-aware graphs, hand-written fx rules stop being a page of code and start being a compiler. TOSA has already made those choices, and it has made them the same way for everyone.
a second front end. TOSA is an interchange. Today there is one producer (PyTorch) and one consumer (ACT), so the interchange carries no traffic. A TensorFlow or ONNX front end, or a second target reading the same specs, would change that.

Neither is true yet. When one becomes true, the boundary to replace is workloads/extract.py and nothing downstream of it, because what it emits is already the corpus’s JSON.

The suite¶

examples/tinytpu/workloads/models.py. Ordinary nn.Modules with nothing Allo-specific in them, sized so every layer fits one build (M, K, N all at most MAXDIM), with bias=False for the reason in What maps, and what does not.

model	batch	layers	what it is for
`mlp_tiny`	4	16 -> 16 -> 16	the smallest thing that is still a model; the one measured on RTL
`mlp_deep`	4	16 -> 16 -> 16 -> 12 -> 8	a run of small layers, where almost everything is fixed cost
`mlp_small`	8	32 -> 32 -> 16	a middle shape, and the one where `Kt = QD`
`mlp_wide`	64	64 -> 64 -> 64	two layers at exactly the steady-state shape the GEMM table quotes
`mlp_bias`	4	16 -> 16, with bias and a sigmoid	the probe: it is meant not to map

mlp_deep and mlp_wide are the two ends of one axis on purpose. They are what lets the page say something about a design decision that the shape table could not: see The measurement that justifies the suite.

Weights are drawn in [-8, 8) and used as integers, so “quantising” the module is a cast rather than a calibration. That is a deliberate limit — a real quantisation scheme is a second unvalidated thing between the model and the number, and this suite is about the mapping.

What maps, and what does not¶

The design computes int8 x int8 -> int32 with an optional ReLU on the accumulator before the narrowing. So:

Maps. nn.Linear without bias, with M any integer in [AR_RAW_DIST, MAXDIM] and K and N multiples of ``T`` up to MAXDIM; and a ReLU that consumes a Linear’s result directly and is its only consumer, which becomes the fused epilogue rather than a second pass.

Note

This sentence used to read “at any (M, K, N) within MAXDIM, including extents that are not multiples of T”. That is true of M and false of K and N: mapspace.residual admits an intrinsic only where it divides the extent, and the intrinsic is T on the reduce and column ranks. Corrected by the sweep in dev/records/tinytpu/front_half_scope_20260924.rst, which is also where the consequence is recorded — at T=8, mlp_deep’s Linear(16, 12) stops mapping.

Does not map, and is reported rather than approximated.

``nn.Linear`` with bias. The ISA has vadd between accumulator rows, but no mapping that broadcasts one bias row across the M rows of the accumulator, so the baseline cannot express it. This is a mapper gap, not an encoding gap — vadd exists.
Any other activation. sigmoid, tanh, gelu and the rest have no unit. vru does ReLU.
A ReLU that is not the sole consumer of a Linear. With two consumers the unfused value is needed as well, and there is no epilogue to ride on.
A ``K`` or ``N`` that is not a multiple of ``T``. Linear(6, 8) passes the extractor and then the mapper enumerates no nest and names no constraint, so what reaches the user is every nest refused. The shape is genuinely out of scope; the message is a bug, recorded in the dated note and not fixed there.
Anything above ``MAXDIM`` in any of M, K or N. The operands are one int8[MAXDIM*MAXDIM] region, so a shape has to fit the addressable space rather than being tiled into it. This is the capability limit TinyTPU-isa vs Gemmini: A Matched Comparison calls worse than the cycle deficit, and the suite inherits it exactly. Below MAXDIM the mapper does tile: an 8x8x8 GEMM on a T=4 instruction is enumerated as 680 nests and mapped, not rejected.
Everything else in a graph — reshape, concatenation, attention, normalisation, residual adds. The extractor names the node and the op.

mlp_bias exists so that this list is demonstrated and not merely asserted: it reports two refusals and maps its one remaining layer. The list as a whole is demonstrated by the scope map, which walks each boundary and records how it fails rather than that it does.

The extractor¶

workloads/extract.py. It walks the ShapeProp-annotated graph once and turns each nn.Linear into a spec in the same convention act/corpus/ uses — same required fields, same einsum string, same epilogue rules, and validated by the same act.spec.validate the corpus is held to. There is no second convention. run.py --emit writes them to workloads/specs/, where they are readable next to the corpus’s own.

M is the product of every activation dimension but the last, so a batch is rows; K and N come from the Linear’s in_features and out_features. The weight is transposed on the way into DRAM, because nn.Linear holds (out, in) and the spec’s B is kn.

Verification¶

Three checks, and a layer that fails any of them is reported, never rounded.

The program against the spec. act.correctness.check runs isa_ref.run — the ISA as numpy — over four operand distributions and compares against spec.gold, the einsum in int64, over the output region, with everything outside the write window required to be untouched. This is the corpus’s own bar and every layer of every model clears it.
The model against PyTorch. The suite then runs the model: each layer’s program in turn through isa_ref, with the real weights in DRAM and each layer fed the activation the fx graph says it consumes (run.layer_sources), against PyTorch’s own evaluation of the same dataflow with the machine’s epilogue (ReLU where fused, then the clip to int8) applied in torch. Every model matches on every byte. This is what makes the suite a workload rather than a shape table: the machine computes the model, not a GEMM of the model’s size.

Both sides used to chain layer i into i+1 regardless of the graph, and that was wrong for any graph that is not a chain. A graph neither side can represent now raises run.Unrepresentable, which gate.py turns into a failure rather than an approximation.
The RTL. Three of the four models’ layers go through cosim.py’s machinery, below, each compared against isa_ref over all 4096 bytes of C rather than over the result region alone.

What is verified, and what is only executed¶

These are different claims and the suite keeps them apart, on the ladder act/judge.py already defines — legal, then correct, then confirmed.

And a gate enforces the difference, which it did not when this page was first written. workloads/claims.json declares, per model, its tier and — for a model at confirmed — the RTL measurement that earned it together with the configuration it was measured at. workloads/gate.py extracts every model, maps every layer, checks bit-exactness against isa_ref and against PyTorch, and then checks the tier:

python examples/tinytpu/workloads/gate.py          # ~30 s, no Vitis
python examples/tinytpu/workloads/gate.py --simulator

It refuses more than it accepts, and that is the point. A measurement whose T, MAXDIM, QD or DMA_WORDS is not the configuration the gate is running is refused as unpairable rather than quoted — and a configuration key that is absent is “cannot pair”, not “matches”, because an older row carries no QD field at all and the absence dates it. mlp_bias must still refuse, and for the stated reason: a run in which the probe maps cleanly fails here, since a gate that made every model pass would be the wrong gate. mlp_wide must carry no measurement, so the ladder cannot be climbed by editing a table. The failure path is tested rather than assumed, in tests/act/test_gates_negative.py.

model	correct (bit-exact vs `isa_ref` and vs PyTorch)	confirmed (RTL ran it)	its cycle figure is
`mlp_tiny`	yes, 2/2 layers	yes, 2/2 layers	measured
`mlp_deep`	yes, 4/4 layers	yes, 4/4 layers	measured
`mlp_small`	yes, 2/2 layers	yes, 2/2 layers	measured
`mlp_wide`	yes, 2/2 layers	no — RTL does not complete	estimated only

So three of four models carry a measured number and one carries a modelled one, and the page never adds them together. mlp_wide is correct and not confirmed: its programs agree with isa_ref and with PyTorch in software, and Vitis csim reports 0 / 4096 mismatches, but no RTL run of it has ever produced a cycle count (why).

Three provenance facts that belong with the claims rather than under them:

The two bit-exactness checks are not the same strength, and only one of them involves PyTorch. correctness.check holds isa_ref — a numpy model of the ISA written here — against spec.gold — a numpy einsum written here; agreeing proves the mapper matches our own semantics and puts PyTorch on neither side. Only the chained check has torch on one side, and what is on it is nn.Linear’s forward on the graph’s own dataflow. Over the whole suite that is 8 912 bytes, of which mlp_wide is 8 192, and mlp_bias contributes none at all: gate.py skips the end-to-end check for a probe, so the probe has no PyTorch comparison. The RTL row is a record that xsim once printed those cycles, declared in claims.json (see its source.transcribed) and checked for self-consistency and configuration, never re-measured. scope.py prints all three per entry.
The bit-exactness claims are pure numpy. isa_ref is numpy and the allo/act/ core imports numpy only, so “correct” is established by code that is entirely this checkout’s — the MLIR bindings are not part of that claim, even though importing allo.act loads them.
The compiled half is borrowed. This worktree has no mlir/build, so allo._mlir resolves to /home/sk3463/allo-bench at ff7beaf1 — tests/act/test_bindings.py fails here and names both paths, which is what it is for. git diff ff7beaf1 HEAD -- mlir/ is empty, so the emitter that produced the HLS for every cosim below is the same source as this branch’s. That is the “borrowed, same commit” case dev/toolchains.rst describes, said out loud as it asks.

The boundary, as a map rather than a list¶

The list above is prose, and prose cannot be diffed. workloads/scope.py measures the same boundary and writes it to workloads/scope_map.json, keyed by T MAXDIM QD DMA_WORDS:

python examples/tinytpu/workloads/scope.py            # print it, ~50 s
python examples/tinytpu/workloads/scope.py --check    # regenerate and diff
python examples/tinytpu/workloads/scope.py --emit     # record this config

Four sections, every cell run and none inferred. ops: one row per kind of torch.fx node, over the suite’s five models plus a pre-norm transformer block and a small CNN defined in the file, so the refusal list is demonstrated rather than asserted. shapes: each of M, K and N swept independently across its ceiling, every mapped cell verified against spec.gold, so a shape that maps to a wrong program is a status and not a silence. entries: per suite entry, which of its checks compares against PyTorch and which compares two things this repository wrote — and which dataflow the comparison read off the graph. assumptions: what the verification itself takes for granted, probed.

There is no hand-maintained expected-value list: the committed map is the program’s own output, so a cell can only move by the measurement moving. --check refuses to compare a configuration it has never recorded, on the same principle claims.json follows.

Two things the first run found; the first is still open, the second is fixed:

A ``K`` or ``N`` that is not a multiple of ``T`` refuses with no reason given — above.
The end-to-end PyTorch check assumed the graph is a chain. FIXED 2026-09-25. run.py fed layer i’s output to layer i+1 on both sides of the comparison, so a fan-out graph was compared against a PyTorch evaluation that was not the model either, and they agreed. Measured on two bias-free Linears off one input: nothing refused, both layers mapped, the suite’s own check reported 0 of 128 bytes differing, and the machine’s output differed from model(x) in 62 of 128. Both sides now take each layer’s activation from run.layer_sources, which reads the fx graph; the same probe reports 0 and 0, and scope.py’s assumptions row – which is the regression test, and keeps its probe – reads sound.

Every committed model is a chain, so no number on this page moved. What moved is that the check is now evidence for graphs that are not chains, and that is what made it safe to gate on: run.py --verify is the CHIA loop’s first gate, and with the ISA editable it is the only comparison with something outside this repository on one side (Why the ISA can move, and what the oracle is).

The numbers¶

Layer by layer, with nothing between layers¶

A model’s cycle figure is a sum over its layers, and that is the honest description of what the machine does today, not a simplification of it. TinyTPU-isa runs one program per layer: operands in from DRAM, result out to DRAM. There is no inter-layer fusion, and nothing — not the activation, not a weight tile — stays resident between layers. So the sum is the whole cost, with one qualification in the other direction: the sum also omits whatever a host would spend between layers, which on this design is not modelled at all.

python workloads/run.py at the shipped build (T=4, MAXDIM=64, DMA_WORDS=1). mapping is what the ACT search picked, and it is worth knowing that the mapper is not the variable here: on eight of the ten layers its pick is bit-identical to the hand-written gemm_program for that shape. The exception is mlp_wide’s two 64x64x64 layers, where ACT prefers M2>N16>K16 — a split of the row dimension the hand mapping does not make — at exactly the same modelled cost, 30 051 and 28 392, so the difference is a tie broken on the label and not a disagreement about which is better.

Per layer, ACT’s pick, estimated¶
layer	shape	epilogue	mapping	dynamic	critical unit	estimate	`isa_ref`
`mlp_tiny_l0`	4x16x16	relu	`N4>K4`	32	`spm` 144	406	bit-exact
`mlp_tiny_l1`	4x16x16	–	`N4>K4`	28	`spm` 144	406	bit-exact
`mlp_deep_l0`	4x16x16	relu	`N4>K4`	32	`spm` 144	406	bit-exact
`mlp_deep_l1`	4x16x16	relu	`N4>K4`	32	`spm` 144	406	bit-exact
`mlp_deep_l2`	4x16x12	relu	`N3>K4`	25	`spm` 108	349	bit-exact
`mlp_deep_l3`	4x12x8	–	`N2>K3`	13	`spm` 54	264	bit-exact
`mlp_small_l0`	8x32x32	relu	`N8>K8`	96	`accu` 640	1 186	bit-exact
`mlp_small_l1`	8x32x16	–	`N4>K8`	48	`vru` 320	682	bit-exact
`mlp_wide_l0`	64x64x64	relu	`M2>N16>K16`	624	`accu` 18 432	29 173	bit-exact
`mlp_wide_l1`	64x64x64	–	`M2>N16>K16`	592	`vru` 17 408	27 562	bit-exact

Warning

These are estimates and one of them is structurally blind. act.cycles.estimate is 179.0 + 1.573 x critical work, refitted on 2026-09-25 to the five published MAXDIM=16 cosim points, which act/cycles.py now derives from reproduce.sh rather than holding as a literal. Every estimate in the table above moved with it, and the refit made the fit worse (worst residual 15.8 to 18.2 cycles on the five points, because TPU_QD=16 moved the small shapes up and the large ones down). Its per-unit work counts come from the header assemble writes — where dma_ld’s entry is the number of instruction rows, not the number of burst iterations. So the estimate cannot see the DMA burst at all. At MAXDIM=64 the burst is a large fixed cost at small shapes (The Benchmark Set prices the MAXDIM=16 -> 64 move at +193 cycles on 16x16x16 alone), and it is exactly the term the burst-widening candidate moves. Every conclusion below about the widening therefore rests on cosim, not on this column.

Measured on RTL¶

Three of the four models, every layer through cosim_design on one csynth of the shipped build, bit-exact against isa_ref over all 4096 bytes of C. These are measurements; the column beside them is the estimate, for scale.

`T=4 MAXDIM=64 DMA_WORDS=1 TPU_QD=16`, `m_axi_latency` 0¶
model	layers	estimated	measured	the estimate is
`mlp_tiny`	584 + 566	811	1 150	29 % low
`mlp_deep`	584 + 584 + 542 + 407	1 424	2 117	33 % low
`mlp_small`	1 636 + 1 145	1 868	2 781	33 % low
`mlp_wide`	–	56 734	no completion	–

mlp_wide is dealt with in its own section. Note that these needed TPU_QD=16: at the shipped TPU_QD=8 not even mlp_tiny’s first layer finishes, which is new information about limitations item 24 and is recorded there.

The measurement that justifies the suite¶

A workload changes which design decisions look good. The suite’s one job, once it ran, was to test that claim on a decision the project already had in front of it.

The decision is the burst-widening candidate, TPU_DMA_WIDEN=1: the dma_ld operand burst reads DMA_WORDS packed words per loop iteration instead of one, up to the 64-byte beat align_value(64) lets Vitis widen the port to. It is parametric, bit-exact at both settings, and costs +123 % BRAM. It is also the knob in the parity-t4 baseline (TPU_T=4 TPU_MAXDIM=64 TPU_DMA_WIDEN=1), so the widened column below is that baseline and not a variant invented here. The Benchmark Set measures it at two shapes:

What the GEMM table says¶
shape	shipped	widened	saving	as a fraction
48x48x48	10 289	9 569	720	7.0 %
64x64x64	22 123	21 163	960	4.3 %

Read as a GEMM table, that is a worthwhile but unspectacular few per cent, shrinking as the shape grows — the profile of a fixed cost being amortised. The question the suite exists to answer is what the same change is worth to a model.

Why the shape table cannot be extrapolated¶

The mechanism is that widening removes burst iterations. A burst of span x WPR words costs ceil(span x WPR / DMA_WORDS) iterations, and dma_ld issues one burst for the activations and one for the weights. Two laws fit that:

charge every removed iteration, activations and weights both;
charge only the longer of the two bursts, the other being hidden.

The first predicts 1440 and 1920 cycles at 48³ and 64³; the second predicts 720 and 960. The measurements are 720 and 960, so the two points settle it: only the longer burst is ever critical.

But both fitted shapes have equal spans. At 48x48x48 the activation span is 48 rows and the weight span is ceil(48/T) x T = 48 rows; at 64x64x64 both are 64. When the two spans are equal the two laws are the same arithmetic, and the shape table never visits a point where they differ.

Every MLP layer is such a point. A 4x16x16 layer bursts 4 activation rows against 16 weight rows — 64 word-iterations against 256. The spans differ by 4x, the weight burst dominates, and which law holds decides the answer. That is not a subtlety the GEMM table could have surfaced, because the GEMM table only ever asked square questions.

So the suite measures.

What it is worth on a model¶

Two builds, one csynth each, every layer through cosim_design on both. T=4, MAXDIM=64, TPU_QD=16, TPU_AXI_LATENCY=0; the only difference between the columns is DMA_WORDS. Every run is bit-exact against isa_ref over all 4096 bytes of C at both settings.

Per layer, measured on RTL¶
layer	shape	shipped	widened	saving	predicted
`mlp_tiny_l0`	4x16x16	584	443	141	240
`mlp_tiny_l1`	4x16x16	566	420	146	240
`mlp_deep_l0`	4x16x16	584	443	141	240
`mlp_deep_l1`	4x16x16	584	443	141	240
`mlp_deep_l2`	4x16x12	542	378	164	240
`mlp_deep_l3`	4x12x8	407	270	137	180
`mlp_small_l0`	8x32x32	1 636	1 156	480	480
`mlp_small_l1`	8x32x16	1 145	665	480	480

Per model, and against the GEMM table¶
model	shipped	widened	saving	as a fraction
`mlp_tiny` (2 layers)	1 150	863	287	25.0 %
`mlp_deep` (4 layers)	2 117	1 534	583	27.5 %
`mlp_small` (2 layers)	2 781	1 821	960	34.5 %
GEMM table, 48x48x48	10 289	9 569	720	7.0 %
GEMM table, 64x64x64	22 123	21 163	960	4.3 %

Warning

Everything in this section is a MAXDIM=64 measurement, and the split does not survive MAXDIM=16. Asked the same question at T=4 MAXDIM=16 QD=16 — both models, every layer, one csynth per burst width — the widening is worth zero cycles: 861 -> 861 and 1 530 -> 1 530, layer by layer, to the cycle. At MAXDIM=16 a DRAM row is 4 packed words instead of 16, the weight burst is four times shorter, and the per-layer prologue covers all of it.

The relationship also inverts: at MAXDIM=16 the GEMM shapes do see the widening (CHIA run 1 measured 0 / 0 / -42 / -59 / -59 on that configuration) and the models do not. So “models are more burst-sensitive than shapes” is a statement about MAXDIM=64, not about models. dev/records/tinytpu/model-term-maxdim-20260924.rst.

The answer is that the widening is worth four to eight times more on a model than the shape table says, at MAXDIM=64. 25 to 34 per cent of a model’s cycles against 4.3 to 7.0 per cent of a big GEMM’s. Read only as a GEMM table, TPU_DMA_WIDEN is a few per cent that shrinks as the shape grows; read on a model, it is a quarter to a third of the whole runtime and it does not shrink, because a model does not get bigger — it gets longer, and every layer pays the prologue again. That is the argument for the suite existing, and it is a conclusion the shape table could not have produced, at any level of detail, because it never asked a question shaped like a model.

Two smaller findings fall out of the same measurement, and both are about our instruments rather than the design:

The prediction is exact where the burst is critical and over-predicts where it is not. At mlp_small’s two layers the measured saving is 480 cycles and the predicted saving is 480 cycles, twice, to the cycle. At the 4-row layers it over-predicts by 40 %: 240 predicted against 141 measured. The reading is that the burst is fully on the critical path once it is long enough, and at four activation rows the rest of the prologue overlaps about a hundred cycles of it. So workloads/burst.py is a usable screen for which layers the widening will help and not a substitute for measuring how much.

The static estimate is accidentally calibrated for a machine we do not ship. Per model, act.cycles.estimate is 29 / 33 / 33 per cent low against the shipped DMA_WORDS=1 build (811, 1 424 and 1 868 against 1 150, 2 117 and 2 781) and 6.0 per cent low, 7.2 per cent low and 2.6 per cent high against the widened one (863, 1 534, 1 821) — it fits the widened machine better than the real one. Per layer the shortfall spreads from 27.5 per cent (mlp_small_l0) to 40.4 per cent (mlp_small_l1). That is not a coincidence: the fit was taken at MAXDIM=16, where WPR is 4 instead of 16 and the burst is four times shorter, so the constant absorbed a burst term that no longer matches. An estimate carries its calibration regime with it, and this one’s regime is not the shipped build. Refitting it on 2026-09-25 did not change that and was not expected to: a refit within the MAXDIM=16 regime cannot fix a regime error (the per-model shortfall moved by under one point).

The 64x64x64 point is missing, and why¶

mlp_wide is in the suite precisely so that the steady-state shape could be measured on these two builds rather than compared across published ones, and it does not complete in cosim at any channel depth tried. Six runs, four builds, both burst widths:

`TPU_QD`	bound	last RTL progress	verdict
16	600 s	`@ 109000` ps — the first periodic report, no second	no completion
32	1500 s	`@ 33400115000` ps = 33.4 ms, about 10 million cycles	no completion

Re-checked on 2026-09-24, after ``QD=16`` became the default (63ee6ec7) and cleared item 24 for the three small models. It changes nothing here: mlp_wide_l0 on a fresh csynth of T=4 MAXDIM=64 QD=16 DMA_WORDS=1 still does not complete inside 600 s, with the same signature as the row above — csim exact at 0 / 4096, last RTL progress @ "109000" ps, no second periodic report. So mlp_wide stays correct and not confirmed, and workloads/gate.py enforces that it carries no cycle count.

Its layers have Kt = 16, so Kt >= QD holds at QD=16, which is the one condition the parity work has a rule for — but QD=32 breaks that condition and does not fix the hang. What QD=32 does change is the evidence: the simulation advanced ten million cycles on a program the model prices at 22 000 and still did not finish, which is a deadlock and not a slow run. So Kt >= QD is not the whole rule, and this is a distinct fact from the QD=8/QD=16 cases at the small shapes, where the sim never got past its first report at all.

One consequence for the headline. The 4.3 % row above is carried from The Benchmark Set, measured at TPU_QD=8 on the hand 64x64x64 mapping, while the three model rows are measured here at TPU_QD=16 — so the comparison crosses a build boundary, and the ratio should be read as four-to-eight-fold rather than as a precise multiple. The three model rows are internally controlled: one csynth per burst width, everything else equal.

Limits¶

Layer by layer, no fusion, no residency — the section above. The sum is what the machine does; it is also why a model with many small layers pays the prologue many times.
No host time. Nothing between layers is modelled: no driver, no scheduling, no cost for the activation making a round trip through DRAM that a fused design would not make.
MLPs only, and small ones. MAXDIM bounds every extent, which rules out the shapes a real network uses. The suite is a way of asking the question, not an answer about real networks.
Integer weights, not a quantisation scheme. See above.
Cosim has no DRAM model. Every RTL figure here is at TPU_AXI_LATENCY=0, a value we chose; m_axi_latency is an HLS scheduling directive and not a memory latency, as The Benchmark Set establishes at length. No row here may be read as behaviour against a memory system.
Every RTL figure on this page needed ``TPU_QD=16``. At the shipped TPU_QD=8 the very first layer of the smallest model does not finish cosim, and it is the shipped GEMM mapping at 4x16x16, not an exotic one. This is a new instance of limitations item 24 and it is recorded there, because the knob that flips it is new information: the earlier bisection never varied a channel depth. run.py --cosim bounds every run and kills by process group; if a layer does not finish, raise TPU_QD before concluding anything from the hang.

Running it¶

One model, every stage printed, run on the design: make mlp. The walk-through, and two changes to the machine put through it, are Tutorial: A PyTorch MLP on TinyTPU, Then Change It.

The suite’s report, every model:

cd examples/tinytpu

python workloads/run.py                      # every model, static, ~1 min
python workloads/run.py --emit               # and write workloads/specs/
python workloads/scope.py --check            # the scope map, ~50 s
python workloads/run.py --simulator          # also check the built design
python workloads/run.py --burst              # what the widening should be worth

TPU_QD=16 python workloads/run.py --cosim mlp_tiny                   # RTL
TPU_QD=16 TPU_DMA_WIDEN=1 python workloads/run.py --cosim mlp_tiny   # widened

pytest tests/act/test_workload_suite.py      # no Vitis, seconds

--cosim synthesises once into --project (default workloads/workload.prj, removed when the run ends) and measures every named model’s layers on that one build, so a sum over layers is a sum over one machine. Put the project inside your worktree: /home is chronically full.

``torch`` is not a dependency of this repository and is not in requirements.txt — upstream keeps it optional, and tests/pytorch/test_linear.py guards its import. The suite needs it, and the allo environment on this host did not have it until 2026-09-22:

pip install --index-url https://download.pytorch.org/whl/cpu torch==2.14.0

pytest tests/act/test_workload_suite.py skips rather than fails when torch is absent, so a machine without it still runs the rest of tests/act. It does not skip when torch is installed but broken: pytest.importorskip re-raises anything that is not a ModuleNotFoundError, which is the behaviour you want and is worth knowing before you read a red collection error as “the suite is wrong”.

A co-design change, read end to end through the suite¶

This is the whole point, so here is the loop once, concretely, with the change that was actually put through it.

The change. Widen the dma_ld operand burst from one packed word per loop iteration to a whole 64-byte beat. It is already parametric, so the change is an environment variable rather than a patch, and it costs +123 % BRAM. The question a co-design loop has to answer is whether that BRAM buys anything on software anyone would run.

Step 1 — what does the workload look like to the machine?

python workloads/run.py --emit

Ten layers, all mappable, each one a spec in workloads/specs/. Six of the ten are four-row GEMMs: the suite is mostly small work, which a shape table chosen by hand would not have told you.

Step 2 — cheap screen: which layers could the change even touch?

python workloads/run.py --burst

Every layer, because every layer’s weight burst is four to sixteen times its activation burst. Seconds, no tools. If this had come back zeros the loop would stop here.

Step 3 — measure the two machines on the workload.

TPU_QD=16 python workloads/run.py --cosim mlp_tiny mlp_deep mlp_small
TPU_QD=16 TPU_DMA_WIDEN=1 python workloads/run.py --cosim mlp_tiny mlp_deep mlp_small

One csynth per build, every layer bit-exact against isa_ref on both. 2 117 cycles becomes 1 534 on mlp_deep; 2 781 becomes 1 821 on mlp_small.

Step 4 — the number the decision turns on. 25 to 34 per cent of a model’s runtime, where the GEMM table said 4.3 to 7.0 per cent. That is the output of the loop: not “the widening is 720 cycles at 48³”, which is true and does not decide anything, but “the widening removes a quarter to a third of the time our software spends”, which does.

What the loop also returned, unasked. Three things the GEMM table could not have produced, and each of them is a correction to an instrument rather than to the design: the static estimate is calibrated for a machine we do not ship; the burst law is exact at eight rows and 40 % optimistic at four; and the shipped channel depth does not run the shipped mapping at 4x16x16. A workload is a better test of the instruments than a shape is, because it exercises them where they were never fitted.