Three tile modes.
One running process.
Idle-only tile control passed CUDA/VNNI host tests and a real-model A/B. The qualified runtime is now active for Rust, with its existing serving profile preserved.
What passed
The engine API accepted all three supported modes. Eighteen operator cases preserved bitwise output equality across original tiles, hoisted AVX2 tiles and hoisted VNNI tiles. A debugger breakpoint confirmed native VNNI execution; both V100s returned 1,024 exact matrix outputs each.
Disabled control stays absent. Invalid input, unsupported combinations and non-loopback clients are rejected. Switches during active and deferred work return 409 without changing the generation. Legacy and OAI completion/chat routes stamp both streamed and nonstreamed responses. A real two-model hot-swap process rejects GET and POST control with 501.
All 18 scored real-model outputs matched within each prompt length. Every scored request and its excluded warmup reported the expected arm, generation, unchanged state and positive Rust expert-operation count.
Production activation
After qualification, Rust moved to the exact tested 30cec2f engine. Its 40 CPU expert layers, 8/20 threads, context and no-speculation policy stayed unchanged; Kairo and the other model processes were not restarted. The new native-loopback control remains in hoisted + VNNI mode. A live completion carried the matching generation, unchanged mode and a positive Rust operation count.
The serving proxy now rejects operator paths, including encoded variants, so ordinary inference credentials cannot reach tile controls through that route. The rollout verified loaded binaries, libraries, wrapper source and model identity, then reopened admission. The old runtime and wrapper remain available through guarded rollback. Seventy-seven focused gateway and rollout tests passed. This activation adds controls and traceability; it is not a new performance result.
Real-model observations
Median rates below are tokens per second. Arm 0,0 uses per-row tiles; 1,0 hoists activation-column work; 1,1 adds VNNI. Throttling is the container's accumulated throttled milliseconds during the client-observed prefill window.
| Input tokens | Arm | Scored | Prefill t/s | Decode t/s | Prefill throttled ms |
|---|---|---|---|---|---|
| 512 | 0,0 | 4 | 178.59 | 28.47 | 2468.4 |
| 512 | 1,0 | 4 | 200.96 | 30.56 | 2179.0 |
| 512 | 1,1 | 4 | 234.96 | 30.58 | 1852.1 |
| 4,096 | 0,0 | 2 | 184.28 | 25.82 | 14521.5 |
| 4,096 | 1,0 | 2 | 194.50 | 26.03 | 14128.4 |
| 4,096 | 1,1 | 2 | 229.76 | 25.57 | 10237.2 |
512 tokens, 1,0 / 0,0: median 1.095×, observed paired range 1.068–1.184× (4 pairs).
512 tokens, 1,1 / 0,0: median 1.306×, observed paired range 1.199–1.340× (4 pairs).
4,096 tokens, 1,0 / 0,0: median 1.057×, observed paired range 1.014–1.100× (2 pairs).
4,096 tokens, 1,1 / 0,0: median 1.249×, observed paired range 1.187–1.312× (2 pairs).
These small samples are functional A/B observations, not a production speedup claim. The paired ranges are descriptive, not confidence intervals. No accepted slow run was discarded.
Configuration and limits
Xeon Gold 6248 with AVX-512 VNNI, two V100 SXM2 32GB GPUs, and a 19.2-core container quota. Huihui Qwen3-30B-A3B abliterated Q4_K_M, expXa source 30cec2f, 8 generation / 20 batch threads, 8,192 context, Q8 KV, no speculation, CUDA1 attention. All 48 expert layers stayed on CPU to preserve production GPU headroom. The live profile has 40 CPU expert layers; these rates are not directly comparable.
The test process ran at nice 10 on the shared host. Production remained active, so neighboring work, throttling and scheduling may affect results. Cgroup counters include other processes; per-process CPU and per-thread run-queue measurements accompany them. Exiting threads can make summed schedstat deltas decrease. The first nonempty streamed content approximates the phase boundary and includes transport/token buffering.
Within each pair, arm order was shuffled with seed 20261103. Each switch was followed by one excluded full-length warmup and one scored request. Exact 512/4,096-token native fixtures, 128 generated tokens, temperature zero, seed 42 and disabled prompt caching were held constant.
Build provenance and retained evidence
The CUDA-enabled candidate rebuilt CPU objects and all six server compilation units against the integrated API, with 207 cached link inputs verified by hash and the complete source snapshot verified. This was an incremental CUDA build, not a clean rebuild of every CUDA translation unit.
Integration review preserved atomic caches and the portable counter implementation. Regression tests corrected cgroup-root quota handling and malformed numeric stamp acceptance. The repository suite now passes 297 tests, alongside local numerical and native/non-AVX2 API checks. The first real-model attempt stopped when a thread exited during a schedstat read and Linux returned ESRCH. Three initial samples were excluded and retained. Harness e828f8d handles that race; the complete A/B schedule restarted with fresh warmups.
During qualification, production process identities remained unchanged. Test servers exited; activation happened afterward. Prior studies and their adverse outcomes remain available.