expXa

Changelog

What changed in each engine revision and what it did to performance. Newest first. The measurements behind each entry are in the linked study.

  1. 30cec2f

    Runtime tile-mode control

    A running server can switch its CPU expert tiles between per-row, hoisted and hoisted + VNNI modes. Switching is only possible while the server is idle, is refused during active or queued work, and is reachable from loopback only. Every completion is stamped with the mode and generation that produced it, so results can be traced to an exact configuration.

    No change to default throughput: this release adds control and traceability. With all 48 expert layers on the CPU, hoisted + VNNI raised prefill 25–31% over per-row tiles.

    Tile-mode study

  2. 47f0b50

    Hoisted and VNNI CPU tiles for Q4_K and Q6_K

    Moves per-column activation work out of the inner tile loop and adds an AVX-512 VNNI path for the quantized dot products.

    Synthetic kernels up to 18% faster at 20 threads (Q6_K down projection). No measurable change in serving throughput at 40 CPU expert layers. Outputs identical.

    Serving before/after

  3. 0982fd3

    Parallel activation preparation

    Activation preparation for the CPU expert queue now runs in parallel.

    About 6% faster on a 4,096-token request in a same-build ablation (25.83 s to 24.27 s). Level with PXA 4 on Qwen3-30B-A3B Q4_K_M.

    Engine comparison

  4. a078474

    CPU expert queue and quantized tiles from PXA 4

    Ports PXA 4’s public CPU expert queue and Q4_K/Q6_K tiles. The new path is gated by model geometry, so Qwen3.6-35B-A3B Q6_K keeps its original CPU path, attention and multi-token prediction unchanged.

    4,096-token request on Qwen3-30B-A3B Q4_K_M: 25.82 s, against 24.01 s for PXA 4 in the same session. The previous build took 49.42 s against PXA 4’s 25.16 s.

    Engine comparison

  5. 208e21d

    First measured baseline

    The first matched comparison against PXA 4 on a CPU-heavy MoE workload, on both a V100 and a Titan Xp.

    About half PXA 4’s prefill speed with 40 CPU expert layers on a V100 (90.7 against 199.6 tokens per second). Decode was level at 4,096 tokens.

    All configurations

Tried and not adopted

Experiments that were built and measured but didn’t ship.

  1. Reordering expert work for weight reuse

    Helped only at 255 active jobs; slowed small batches and the paired kernel.

    Not adopted

  2. Warp sharing in expert and attention kernels

    No gain over a matched control; one attention variant was 19% slower.

    Not adopted