Finding
With 40 of 48 expert layers on the CPU, expXa 0982fd3 took a median 24.27 s for a 4,096-token request. PXA 4 took 24.38 s in the same session. The two are effectively level: the run ranges overlap, and expXa’s slowest run (27.10 s) was slower than PXA 4’s slowest (26.76 s).
At 512 tokens expXa was faster on the median (6.00 s against 6.95 s), but one PXA 4 run finished in 5.91 s. Three runs per engine aren’t enough to call a winner.
| Engine | Input tokens | Prefill tok/s | Decode tok/s | Request s | Range s |
|---|---|---|---|---|---|
| expXa 0982fd3 | 512 | 203.97 | 37.28 | 6.00 | 5.77–6.07 |
| PXA 4.0.1 | 512 | 180.82 | 31.23 | 6.95 | 5.91–7.45 |
| expXa 0982fd3 | 4,096 | 201.53 | 32.96 | 24.27 | 24.16–27.10 |
| PXA 4.0.1 | 4,096 | 202.00 | 30.41 | 24.38 | 24.01–26.76 |
How the gap closed
These were three separate sessions, each with its own PXA 4 control. Differences between sessions include host noise as well as code changes, so compare each expXa result with the control from its own session.
| Session | expXa change | expXa | PXA 4 |
|---|---|---|---|
| 2 Nov | 208e21d, starting point | 49.42 | 25.16 |
| 3 Nov | a078474, PXA 4 CPU expert queue and tiles ported | 25.82 | 24.01 |
| 3 Nov | 0982fd3, parallel activation preparation | 24.27 | 24.38 |
Almost all of the original gap was prefill: 90.7 tokens per second against PXA 4’s 199.6. Porting PXA 4’s CPU expert queue and quantized tiles closed most of it. The rest came from preparing activations in parallel.
The cleanest evidence for that last step is an ablation inside one build. Holding the binary and weights fixed and switching only the preparation path, the old path took 25.83 s and the parallel path 24.27 s: about 6% faster on this long-prompt workload.
Method
- Same weights, token fixtures and settings for every arm: 8,192 context, Q8_0 KV cache, 40 CPU expert layers, 8 generation and 20 batch threads, GPU 0.
- Each request forced exactly 128 output tokens at temperature zero, seed 42, with prompt caching off.
- Three repeats per arm, with engine order rotated. Short correctness checks ran before scoring; there was no full-length warmup.
- Prefill and decode rates come from each engine’s native timing fields. Request time is wall-clock for the whole request, so it isn’t simply the sum of the two.
- About 21% of the model’s weight buffers sit on the GPU. That is a memory figure, not the share of compute.
All 12 runs from the final session
| Engine | Input | Run | Prefill tok/s | Decode tok/s | Request s |
|---|---|---|---|---|---|
| expXa 0982fd3 | 512 | 1 | 199.39 | 37.28 | 6.003 |
| expXa 0982fd3 | 512 | 2 | 203.97 | 36.02 | 6.067 |
| expXa 0982fd3 | 512 | 3 | 207.86 | 38.70 | 5.772 |
| PXA 4.0.1 | 512 | 1 | 180.82 | 31.07 | 6.954 |
| PXA 4.0.1 | 512 | 2 | 152.69 | 31.23 | 7.454 |
| PXA 4.0.1 | 512 | 3 | 199.26 | 38.33 | 5.911 |
| expXa 0982fd3 | 4,096 | 1 | 201.53 | 32.49 | 24.271 |
| expXa 0982fd3 | 4,096 | 2 | 202.04 | 32.96 | 24.163 |
| expXa 0982fd3 | 4,096 | 3 | 175.58 | 34.05 | 27.095 |
| PXA 4.0.1 | 4,096 | 1 | 182.11 | 30.10 | 26.759 |
| PXA 4.0.1 | 4,096 | 2 | 202.00 | 34.31 | 24.014 |
| PXA 4.0.1 | 4,096 | 3 | 203.14 | 30.41 | 24.382 |
Effect on Qwen3.6-35B-A3B Q6_K
The new CPU path is gated by model geometry, so Qwen3.6-35B-A3B Q6_K keeps its original CPU path along with its 4×8 attention and multi-token prediction. We checked that nothing changed for it:
- All 28 captured responses, with their token and MTP draft counts, matched the pre-change build exactly.
- Both chat-formatted 65K-token retrieval checks passed.
- One raw-completion retrieval variant returns end-of-sequence immediately, both before and after the change. That failure predates this work and is still open.
These are regression checks, not a speed comparison with PXA 4 on this model.
Limits
- Three runs per arm on a shared host. Idle checks reduced interference but can’t rule it out.
- Forced-length synthetic prompts. Conversational workloads with natural stopping may behave differently.
- One GPU generation and one placement. On a Titan Xp with 19–20 CPU expert layers, PXA 4 was clearly faster (63.5 s against 85.7 s at 4,096 tokens) with the older
208e21dbuild. See all configurations.