For GPU weight offloading, price the host interconnect as well as the GPU
GPU rental comparisons for weight streaming must include measured host-to-device bandwidth, not only VRAM, GPU FLOPS and local-memory bandwidth. A PCIe 5-capable GPU can be installed in a slower host link. Estimate the transfer floor from bytes fetched per decode step divided by …