Finding
Outputs were identical before and after, so the tiles are correct and were adopted. Serving throughput didn’t improve. Prefill medians on the 30B-A3B were 2–2.5% lower afterwards, which is well inside that model’s run-to-run variation. The 35B-A3B was effectively unchanged, as expected, since it doesn’t use these CPU kernels.
| Model | Input | Prefill before | Prefill after | Decode before | Decode after |
|---|---|---|---|---|---|
| Qwen3-30B-A3B Q4_K_M | 512 | 215.35 | 210.04 | 37.63 | 37.03 |
| Qwen3-30B-A3B Q4_K_M | 4,096 | 209.28 | 204.53 | 34.09 | 33.48 |
| Qwen3.6-35B-A3B Q6_K | 512 | 1,834.86 | 1,840.07 | 87.04 | 87.62 |
| Qwen3.6-35B-A3B Q6_K | 4,096 | 4,316.62 | 4,343.73 | 89.03 | 89.39 |
After the change, 30B-A3B prefill ranged from 156.5 to 269.0 tokens per second at 512 tokens, and from 150.8 to 260.6 at 4,096. A spread that wide hides any effect of a few percent.
The kernels on their own
Native VNNI execution was confirmed on the Xeon, and 36 CPU parity cases plus CUDA matrix checks on both GPUs passed. Earlier two-thread synthetic tests showed gains of 18–28%. At 20 threads the picture depends on the projection:
| Projection | Original tiles | Hoisted | Hoisted + VNNI |
|---|---|---|---|
| up/gate Q4_K | 165.5 | 167.0 | 166.2 |
| down Q6_K | 296.4 | 333.1 | 349.4 |
| down Q4_K | 161.8 | 196.5 | 196.3 |
The up/gate projection, which covers two of the three expert matrices, barely moves. That is one plausible reason the serving numbers stayed flat; it hasn’t been confirmed.
Method
- Two sequential phases, before and after, roughly 25 minutes apart.
- Fixed 512- and 4,096-token fixtures, 128 output tokens, temperature zero, seed 42, prompt cache off.
- Eight scored runs per model and length in each phase, with model and length order alternated. Four warmups per phase were excluded.
- Speculative draft and accept counts were compared as well as outputs; all 64 scored outputs matched.
- The download includes every run, page-fault counts, output hashes and paired bootstrap intervals (10,000 resamples). With sequential phases these intervals are descriptive, not causal.
Excluded attempts
- An early 35B-A3B warmup stopped at 57 tokens. The fixture was fixed and warmups restarted; no run from that attempt was scored.
- The first activation started the 30B-A3B before the 35B-A3B. That left enough VRAM for n-gram speculation to switch itself on, producing decode rates around 120 tokens per second. Those runs changed the decoding policy, not the tiles, and were excluded. The final activation used a fixed start order and confirmed speculation was off.
Limits
- Sequential phases can’t separate the kernel effect from host variation.
- After activation, the 4,096-token warmup had 15 major page faults and the first scored run had 3; every other scored run had none. Some slow runs with zero faults remain unexplained. No accepted run was discarded.