expXa

Research

MoE CPU offload

Running mixture-of-experts models with some expert layers on the CPU: expert queues, activation preparation and quantized tiles.

  1. expXa vs PXA 4 on Qwen3-30B-A3B Q4_K_M

    Matched V100 comparison of expXa and PXA 4.0.1 on Qwen3-30B-A3B Q4_K_M with 40 CPU expert layers: per-run results, method, and how expXa went from 49 s to 24 s.

    Adopted

  2. VNNI CPU tiles in serving

    Before/after serving measurement of hoisted and AVX-512 VNNI Q4_K/Q6_K CPU tiles in expXa on two MoE models: outputs unchanged, no measurable serving speedup.

    Adopted

  3. Switching CPU tile modes at runtime

    expXa can switch between per-row, hoisted and hoisted+VNNI CPU tiles inside one running process. Correctness checks and a three-way A/B on Qwen3-30B-A3B Q4_K_M.

    Adopted

Other topics: Attention AVX-512 VNNI CPU kernels CUDA kernels PXA comparison Q4_K_M Q6_K NVIDIA V100