expXa

Research

Q4_K_M

Studies on Q4_K_M models, mainly Qwen3-30B-A3B with expert layers held in system memory.

  1. expXa vs PXA 4 on Qwen3-30B-A3B Q4_K_M

    Matched V100 comparison of expXa and PXA 4.0.1 on Qwen3-30B-A3B Q4_K_M with 40 CPU expert layers: per-run results, method, and how expXa went from 49 s to 24 s.

    Adopted

  2. VNNI CPU tiles in serving

    Before/after serving measurement of hoisted and AVX-512 VNNI Q4_K/Q6_K CPU tiles in expXa on two MoE models: outputs unchanged, no measurable serving speedup.

    Adopted

  3. Switching CPU tile modes at runtime

    expXa can switch between per-row, hoisted and hoisted+VNNI CPU tiles inside one running process. Correctness checks and a three-way A/B on Qwen3-30B-A3B Q4_K_M.

    Adopted

  4. Dense models across three engines

    Qwen3.5-9B Q4_K_M and Qwen3.5-0.8B Q8_0 on custom PXA 3.1, PXA 4.0.1 and expXa: 54 matched requests on V100, with correctness and cancellation checks.

    No change

Other topics: Attention AVX-512 VNNI CPU kernels CUDA kernels MoE CPU offload PXA comparison Q6_K NVIDIA V100