# Dense-model engine comparison — 2026-11-03

Requested comparison: custom PXA 3.1, official PXA 4.0.1 and expXa on the existing Qwen3.5 9B and 0.8B weights. This is an isolated benchmark, not authorization to switch either production route.

## Controls

- Use the existing production dual-V100-SXM2-32GB/NVLink host, with all five text backends loaded. Native `/slots` is sampled every 0.5 seconds; any observed production work terminates only the isolated candidate. Shared-host activity between samples remains a limitation.
- Exact production model files are hashed once. Each runtime binary and its mapped engine libraries are hashed. No model conversion, source patch, runtime replacement, or production restart.
- Custom build `0a1fe56`, official 4.0.1 runtime preserved from the Rust activation, expXa `30cec2f540bc`. Record actual runtime provenance with each arm.
- 9B: 8,192 context, both GPUs, equal layer split, main GPU1, 4 generation / 8 batch threads, batch4,096 / microbatch512. This intentionally differs from production's single-GPU 131,072 context to fit spare capacity. Results do not establish single-GPU or full-context deployment performance.
- 0.8B: 4,096 context on GPU1, 2 generation / 2 batch threads, batch256 / microbatch128, matching the live envelope.
- Both: all layers on GPU, one slot, Q8_0 K/V, Flash Attention enabled, speculative decoding disabled. No Rust CPU-expert or Kairo-specific expXa profile is enabled. PXA automatic sampler/context control disabled equally.

## Samples

Three rounds with rotating engine order, alternating model order and rotating prompt-length order. Each arm has an unscored 512-token warm-up. Fixed synthetic token fixtures must match exactly between engines for each model. Prompt cache is disabled on every speed sample; returned prompt and generation counts must equal the requested counts.

9B inputs: 512, 2,048 and 6,144 tokens. Helper inputs: 512, 2,048 and 3,584. Each timed request forces 128 output tokens at greedy sampling. Collect streamed time to first nonempty content, wall-clock request time, native prompt/generation throughput and output. Forced EOS suppression is only a speed fixture, not evidence of useful answer quality.

Four separate natural-EOS checks cover arithmetic, supplied-context grounding, field extraction and negation. Retain actual responses and finish reasons, including failures. After speed samples, cancel an isolated long request, check slot release, then repeat the arithmetic check twice to assess recovery and deterministic output. These are narrow sanity checks, not a broad model-quality evaluation.

## Guards and interpretation

Require three consecutive idle checks before launching. Enforce GPU free memory of at least768MiB per card, host memory headroom of at least4GiB, and a600-second candidate deadline. Preserve all interrupted/failed attempts. Retry traffic interruptions at most four times per arm; never include them in accepted results.

Report median and range across three rounds, separating each model and prompt size. Compare whole requests as well as decode rate. Preserve output differences, failures and context/placement limitations. Verify production PID/start-time/argv identity after every arm and after the whole study; confirm temporary listeners and processes exit.

Publish completed results and adverse outcomes under `expxa.com/benchmarks/`, with one dated update and downloadable evidence. No production engine promotion is part of this task.