expXa

Research

CPU kernels

Quantized CPU tiles for expert matrices, including hoisting and AVX-512 VNNI paths.

  1. VNNI CPU tiles in serving

    Before/after serving measurement of hoisted and AVX-512 VNNI Q4_K/Q6_K CPU tiles in expXa on two MoE models: outputs unchanged, no measurable serving speedup.

    Adopted

  2. Switching CPU tile modes at runtime

    expXa can switch between per-row, hoisted and hoisted+VNNI CPU tiles inside one running process. Correctness checks and a three-way A/B on Qwen3-30B-A3B Q4_K_M.

    Adopted

Other topics: Attention AVX-512 VNNI CUDA kernels MoE CPU offload PXA comparison Q4_K_M Q6_K NVIDIA V100