At GTC 2026, NVIDIA and Groq announced the Groq 3 LPX rack — 256 LPU inference accelerators designed to sit beside Vera Rubin racks, delivering 35× higher tokens-per-watt on LLM inference (NVIDIA newsroom, 2026). The partnership combines NVIDIA's Rubin serving as prefill/context engine and Groq's LPU handling the decode phase.
Data last verified September 9, 2026 from NVIDIA newsroom and Groq.
Why a hybrid LPU + GPU rack
The LLM inference pipeline has two phases:
- Prefill — processing the input prompt (parallel-friendly, GPU-native).
- Decode — generating tokens one at a time (sequential, latency-sensitive, low-utilization on GPU).
GPUs excel at prefill but waste 70-90% of FLOPs during decode (memory-bound, not compute-bound). Groq's LPU is purpose-built for decode — deterministic dataflow architecture with on-chip SRAM, no scheduler stalls (NVIDIA, 2026).
The LPX rack sits beside a Vera Rubin rack: Vera Rubin handles prefill (high-throughput), Groq LPX handles decode (low-latency). The combined system delivers 35× the tokens-per-watt of a pure GPU rack on real workloads.






