Single-GPU MoE fit must bind quantization to runtime

A single-GPU MoE recommendation must bind the exact quantized checkpoint to its serving runtime. Total parameters determine weight residency; active parameters only approximate part of token compute. A memory fit is not proof of quality, efficient kernels, or concurrent serving capacity. Reserve context-cache and runtime memory, then compare quality-qualified completions under latency limits at realistic concurrency. This applies to GPU-resident autoregressive MoEs; offloading and speculative decoding change the performance model.

Evidence/example — 2026-09-22: Google lists Gemma 4 26B A4B load estimates of 57.7 GB BF16, 28.8 GB SFP8, and 14.4 GB Q4_0, excluding context cache (https://ai.google.dev/gemma/docs/core). An official QAT Q4_0 GGUF exists (https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-gguf). Separately, the vLLM Gemma4 recipe excludes this MoE from its QAT W4A16 path, citing sensitivity of 704-wide experts, and recommends int8 per-channel weight-only quantization (https://github.com/vllm-project/recipes/blob/main/Google/Gemma4.md). These format-specific statements should not be collapsed into either 'all 4-bit works equally' or 'all 4-bit is unusable'. Inference: a 32 GB GPU is a plausible Q4 GGUF deployment target, while 8-bit leaves substantially less serving headroom. No hardware benchmark or quality evaluation was performed; throughput and useful concurrency remain unmeasured.