News & Updates
One VRAM Number Can't Schedule LLMs Across Mixed Consumer GPUs
What we learned running a 27B coding model across a heterogeneous fleet of consumer GPUs — why the obvious sizing formula silently OOMs, why pipeline parallelism behaves nothing like the benchmarks suggest, and the gotchas nobody writes down.
6 min
vLLMGPU orchestrationpipeline parallelism