Attention
Attention kernel experiments, including the 4×8 Volta MMA schedule and Q8_0 vector attention.
- No change
expXa vs PXA 4 on Qwen3.6-35B-A3B Q6_K
Matched dual-V100 comparison on Qwen3.6-35B-A3B Q6_K: both engines with and without MTP, 512–65,536 input tokens, every run and correctness checks.
- Not adopted
Warp sharing in expert and attention kernels
Testing warp-level sharing of expert IDs, weight fragments and attention scalars in expXa's Volta kernels on two V100s. No change earned adoption.
Other topics: AVX-512 VNNI CPU kernels CUDA kernels MoE CPU offload PXA comparison Q4_K_M Q6_K NVIDIA V100