Skip to content

The threaded ic step, counted in micro-panels (#458)

The multi-threaded mc response curve is W-shaped (#458). This page shows what causes it, the change that follows, and its measured effect on three machines.

The serial path steps mc in whole micro-panels and the threaded path does not. balanced_mc rounds a serial mc down to a multiple of the register tile’s mr. In the threaded path it rounded the ic block count up to a multiple of the thread count and kept whatever mc came out. With mr = 6:

machine (Highway build)kcserial mcthreaded mcpanels per threaded block
Xeon E5-2420 v2 (AVX)51230325.33, so 6 are computed
i7-12700K (AVX2)256606410.67, so 11 are computed

The micro-kernel computes whole mr × nr tiles. A block of mc rows therefore costs ceil(mc / mr) panels, and in the threaded path every block carries a partial one.

This explains all three of #458’s observations:

  • The spread is 3.6% at T=1 and 13.0% at T=6. The two thread counts ran different blocks. T=1 always ran whole panels.
  • The curve is W-shaped. A count of whole panels is quantised, so the critical path rises and falls as mc moves through multiples of mr.
  • It is independent of kc. Panel counts don’t depend on kc.

The earlier “mc = 8 lost 17%” result is the same effect: 1.33 panels per block.

The cost is the number of micro-panels on the busiest thread of the round-robin partition (detail::critical_panels). The table fits each candidate to #458’s Xeon curve (2048³, T=6, benchmarks/data/xeon-e5-2420-sweep), as correlation with the best of 3 rounds:

cost modelkc = 512kc = 256
block-count balance, ceil(nib/T)/(nib/T): what balanced_mc optimised+0.230.00
critical rows+0.21+0.02
critical micro-panels+0.83+0.74

On the i7-12700K, an mc sweep at the shipped kc = 256 fits the panel model at r = +0.99 on three of four shapes at T=8. On the fourth, 4096², the whole range is inside the noise floor.

The model has one known residual. On the Xeon, mc = 171 measures about 5% below the model’s prediction. Its A block (171 × 512 × 8 bytes) is much larger than the 256 KB L2, so this is plausibly a cache effect the panel count does not see.

The block-count result is kept as the incumbent. A block size in [ceil(mc_max / 2), mc_max] replaces it only when it gives the busiest thread strictly fewer micro-panels.

  • Ties keep the incumbent. Plans that were already panel-optimal don’t move. That includes #408’s mc = 64 at m = 1024 on 8 threads, and the i7’s shipped plans at T=8.
  • The window guards against small blocks. A panel count alone always prefers a one-panel block, and that is wrong: each ic block is another pass over the shared packed B panel. On the i7 at kc = 1024, mc = 6 ran at ×0.61–0.87 of mc = 16 on eight threads. Searching no lower than half the cache bound allows at most twice the blocks.
  • Unchanged: the serial path, the #488 sub-tile fallback, and callers that pass mr = 1.

The run used benchmarks/machines/i7-12700k-mc.sh:

  • pinned to the 8 P-cores;
  • 4 interleaved rounds with alternating order, min of 5 reps;
  • default (this change) against legacymc (the old rule), both built from the same tree;
  • data in benchmarks/data/i7-12700k-mc458.

These are the points where the plan changed:

shapeTmc old → newold GF/snew GF/sratio
1000×1000×1000263 → 42133.2139.5×1.048
1000×1000×1000463 → 42251.0261.8×1.043
1000×1000×1000656 → 42347.3369.1×1.063
1000×1000×1000863 → 42449.9463.8×1.031
1024×1024×1024264 → 42135.3137.6×1.017
2048×2048×2048264 → 54132.1136.6×1.034
3000×3000×1000263 → 60128.1134.2×1.047
3000×3000×1000463 → 54248.7258.8×1.041
3000×3000×1000663 → 42352.3367.5×1.043
3000×3000×1000863 → 54445.3463.6×1.041
4096×4096×1024264 → 54129.1137.8×1.067
4096×4096×1024464 → 54247.7265.5×1.072
4096×4096×1024663 → 36340.3371.7×1.092
  • Faster: 12 of the 13 changed plans, by 3.1–9.2%. The 13th, 1024² at T=2, is ×1.017, below the 2% threshold the analyzer calls.
  • Slower: none.
  • Control: the 17 points whose plan did not change agree within 2%. That includes every T=1 point, since the serial path is untouched.

The run used benchmarks/machines/xeon-e5-2420-mc.sh:

  • pinned to the six physical cores 0–5;
  • the same protocol as the i7;
  • data in benchmarks/data/xeon-e5-2420-mc458.

The governor was ondemand, which the sidecar records. The median round-to-round spread was still 0.9% (maximum 2.5%).

This is the machine #458’s curve was measured on. The table lists every point where the plan changed:

shapeTmc old → newcritical panelspredictedold GF/snew GF/smeasured
512×8192×1024232 → 2448 → 44×1.09115.617.4×1.115
512×8192×1024432 → 1824 → 23×1.04329.531.9×1.082
1000×1000×1000232 → 2496 → 84×1.14315.918.4×1.155
1000×1000×1000432 → 1848 → 42×1.14330.735.5×1.158
1000×1000×1000628 → 2430 → 28×1.07147.250.7×1.075
1024×1024×1024232 → 3096 → 86×1.11616.218.2×1.121
1024×1024×1024432 → 2448 → 44×1.09131.034.3×1.107
2048×2048×2048232 → 18192 → 171×1.12316.318.6×1.146
2048×2048×2048432 → 3096 → 87×1.10331.635.5×1.123
2048×2048×2048632 → 1866 → 57×1.15845.352.6×1.162
3000×3000×1000232 → 30282 → 250×1.12816.418.6×1.133
3000×3000×1000432 → 30144 → 125×1.15231.435.6×1.133
3000×3000×1000632 → 2496 → 84×1.14346.452.5×1.133
4096×4096×1024232 → 18384 → 342×1.12316.318.6×1.140
4096×4096×1024432 → 18192 → 171×1.12331.536.0×1.143
4096×4096×1024632 → 18132 → 114×1.15846.153.3×1.158
  • Faster: all 16 changed plans, by 7.5–16.2%. The issue’s own point, 2048³ at T=6, went from 45.3 to 52.6 GF/s. That is above the best point of the issue’s override sweep, 51.2 GF/s at mc = 12, which was measured in a different session.
  • Slower: none.
  • Control: the 8 unchanged points, including all four T=1 points, agree within 1%.
  • Per point, the panel model is a prediction rather than just a fit:
    • Xeon: r = +0.91 between predicted and measured ratio, and measured gains average ×1.010 of predicted.
    • i7: the predicted range is narrow compared with the noise, so r is only +0.42, but the calibration is ×1.005.

The run used benchmarks/machines/ryzen-9-8945hs-mc.sh:

  • under WSL2, built with Clang 18 for AVX-512 (build_isa includes AVX3_DL);
  • pinned to one logical CPU per core, 0,2,…,14;
  • the same protocol as the other two machines;
  • data in benchmarks/data/ryzen-9-8945hs-mc458.

Its blocking is mr = 6, nr = 16, kc = 128 and mc_max = 128, not the i7’s 64, so 128 rows is 21.33 panels per block. The rule moves fewer plans here than on the other machines: at T = 6 the old rule’s mc (84, 86, 114) is already panel-optimal, and at T = 8 nothing in the window gives fewer panels.

shapeTmc old → newcritical panelspredictedold GF/snew GF/sbestmedian
1024×1024×10242128 → 10288 → 86×1.023127.8132.2×1.034×1.031
2048×2048×20482128 → 114176 → 171×1.029114.2119.8×1.049×1.009
2048×2048×20484128 → 10288 → 87×1.011208.4216.0×1.037×1.013
4096×4096×10242128 → 114352 → 342×1.029108.0114.9×1.064×1.038
4096×4096×10244128 → 114176 → 171×1.029193.1202.4×1.048×1.048

GF/s is the best of 4 rounds; “median” is the ratio of the round medians.

  • Faster: all 5 changed plans, on both measures. At 1024² T=2 and at both 4096² points, the two arms’ round ranges don’t overlap. At the two 2048³ points they do, so those two are directional rather than resolved.
  • Slower: none.
  • Control, T ≤ 4: the 13 unchanged points agree within 1.7%, including every T=1 point.
  • Control, T = 6 and 8: the unchanged points scatter from ×0.91 to ×1.12 between arms that run the same plan. The round-to-round spread there reaches 23%, against a median of 2.4% over the whole run. This is the noise floor of a WSL2 guest at high thread counts, where Windows places the virtual CPUs; it is not a property of either rule, and no changed plan sits in that range.
  • Model: the gains are small, ×1.01–1.03 predicted, so the per-point correlation is weak (r = +0.66). Measured gains average ×1.021 of predicted.
Terminal window
benchmarks/machines/i7-12700k-mc.sh # or xeon-e5-2420-mc.sh, ryzen-9-8945hs-mc.sh
machine=i7-12700k # or xeon-e5-2420, ryzen-9-8945hs
python3 benchmarks/analyze_blocking_ab.py \
"benchmarks/data/${machine}-mc458/blocking_ab_legacymc.csv" \
"benchmarks/data/${machine}-mc458/blocking_ab_default.csv"

In the analyzer’s output, a ratio below 1 means the new rule (default) is faster. The column headed “detection helped” is the analyzer’s generic wording from #430.