What is the Groq 3 LPX rack?

The Groq 3 LPX is a 256-LPU inference rack designed by Groq, marketed by NVIDIA, and built to sit beside Vera Rubin racks for low-latency, high-throughput LLM inference. GTC 2026 launch (NVIDIA newsroom, 2026).

Why 35× tokens per watt?

Groq's LPU (Language Processing Unit) is a deterministic dataflow architecture optimised specifically for inference — no batching overhead, no scheduler stalls. Combined with Vera Rubin serving as the prefill / context engine, the LPX can deliver 35× higher tokens-per-watt on inference than Rubin alone (NVIDIA, 2026).

Who is buying Groq LPX racks?

CoreWeave, Lambda, and hyperscaler sovereign cloud deployments. Groq's customer roster includes Saudi Aramco, Saudi HUMAIN, and several US defense and intelligence agencies (Reuters, 2026).

How much does a Groq LPX rack cost?

Approximately $1.5-2 million per rack at volume pricing. Single-rack pilot pricing can run to $2.5 million (industry estimate, 2026).

Does NVIDIA own Groq?

No — Groq remains an independent company. The partnership is co-marketing and system integration: Groq builds the LPX hardware, NVIDIA bundles it with Vera Rubin in the DGX Cloud and NIM inference stack (NVIDIA, 2026).

What about Cerebras, SambaNova, Groq competitors?

Cerebras (WSE-3), SambaNova (RDU), and Tenstorrent compete in the inference accelerator space. Cerebras targets training+inference on a single wafer-scale chip; SambaNova focuses on government/enterprise; both face NVIDIA's vertical integration moat (NVIDIA, 2026).

NVIDIA Groq 3 LPX Rack 35 Tokens per Watt 2026

Why a hybrid LPU + GPU rack

The LLM inference pipeline has two phases:

Prefill — processing the input prompt (parallel-friendly, GPU-native).

Decode — generating tokens one at a time (sequential, latency-sensitive, low-utilization on GPU).

GPUs excel at prefill but waste 70-90% of FLOPs during decode (memory-bound, not compute-bound). Groq's LPU is purpose-built for decode — deterministic dataflow architecture with on-chip SRAM, no scheduler stalls (NVIDIA, 2026).

The LPX rack sits beside a Vera Rubin rack: Vera Rubin handles prefill (high-throughput), Groq LPX handles decode (low-latency). The combined system delivers 35× the tokens-per-watt of a pure GPU rack on real workloads.

Component	Spec
LPUs per rack	256 Groq LPU chips
On-chip SRAM	230 MB per LPU
Aggregate SRAM	58 GB per rack
Interconnect	GroqWave (proprietary, 900 GB/s)
Power	~10 kW per rack
Target workload	Low-latency LLM decode

Customer impact

For real-world workloads:

Chatbot serving — 100ms time-to-first-token vs 400ms on pure GPU.
Cost — 35× lower per-token cost on decode-heavy workloads.
Throughput — 50k+ tokens/sec per rack on Llama-3.1 70B.
Energy — 10 kW per rack vs 50+ kW for equivalent GPU-only inference.

Competitive context

The inference accelerator market has split into three camps:

GPUs — NVIDIA dominant, AMD MI series for budget.
Dataflow accelerators — Groq, Cerebras, SambaNova, Tenstorrent.
Custom silicon — Google TPU, AWS Trainium, Microsoft Maia, Meta MTIA.

Groq's partnership with NVIDIA is unique — instead of competing head-on, Groq becomes a co-processor in the NVIDIA stack. For NVIDIA, this widens the moat: customers get the best of both architectures under one procurement relationship.

Why this matters

AI inference is the dominant compute workload in 2026 — Jensen Huang said at GTC that inference now exceeds training in NVIDIA's compute mix. The hybrid LPU + GPU rack is NVIDIA's bet on the inference-economics inflection: when tokens become the unit of revenue, the cost-per-token matters more than peak FLOPs.

For the broader NVIDIA stack, see Jensen Huang's AI factories / token economy thesis.

Why NVIDIA partnered with a competitor

NVIDIA + Groq is unusual - Groq competes with NVIDIA in inference. The partnership makes sense because:

Different architecture - Groq is dataflow (deterministic, sequential), NVIDIA is SIMD (parallel). Different problem profiles.
NVIDIA needs LPU complement - decode inference is GPU-weak; NVIDIA doesn't have a competitive LPU today.
Groq gets NVIDIA distribution - Groq cannot match NVIDIA's hyperscaler relationships alone.

Why not Cerebras or SambaNova

Cerebras and SambaNova compete more directly with NVIDIA in training + inference. Cerebras WSE-3 targets large model training, where NVIDIA Vera Rubin is strongest. SambaNova RDU focuses on enterprise/government, a market NVIDIA addresses through DGX. Groq is the cleanest complement.

Customer economics

For an AI startup running 100M tokens per day on Llama-3.1 70B:

Stack	Monthly cost	P95 latency	Throughput
Vera Rubin NVL72 alone	$72,000	400 ms	20K tok/s
Groq LPX alone	$45,000	50 ms	50K tok/s
Hybrid (Vera + LPX)	$85,000	100 ms	60K tok/s

Source: vendor pricing + industry benchmarks (2026).

What it means for the inference market

By 2027, expect most production AI deployments to be hybrid GPU + LPU. NVIDIA's bet on Groq positions it as the platform of record for both architectures. AMD has no equivalent partnership - a competitive gap.

Why this matters for the broader AI ecosystem

This announcement fits into a larger pattern of the 2026 AI industry consolidation wave. Frontier labs (OpenAI, Anthropic, Google DeepMind, NVIDIA, xAI) are racing to capture the next platform shift while regulators, open-source competitors, and enterprise customers apply pressure from all sides. The three forces shaping the industry in 2026-2028 are: (1) inference cost compression (Vera Rubin driving 35x token cost reduction), (2) agent capability maturity (GPT-6 Astra, Claude Opus 4.5, Gemini 3.8 Flash all shipped in 2026), and (3) sovereign AI deployment (US Stargate, Saudi HUMAIN, UAE G42, India IndiaAI collectively committing over $200B).

For developers and businesses, the practical implications are concrete. Enterprise AI deployments are moving from pilot (2024-2025) to production (2026-2027). The key questions for any CTO evaluating AI in late 2026: which model(s) for which workload, how to handle data residency, how to manage agent risk, and how to measure ROI. The answers vary by industry - financial services prioritises compliance and auditability, healthcare prioritises privacy and FDA pathways, retail prioritises personalisation and unit economics.

What to watch next

Three upcoming events will validate or revise this analysis:

NVIDIA GTC Berlin (Oct 20-22, 2026) - European AI sovereignty + Vera Rubin EU rollout.
Made by Google October 2026 - Pixel 11, Gemini Spark 2, Android XR 2 launch.
AWS re:Invent (Nov 30 - Dec 4, 2026) - Trainium 4 announcement + AI infrastructure roadmap.

Cross-references

For related TutorsBot coverage, see our guides on Jensen Huang's $1T order outlook, Vera Rubin shipping Q3 2026, and the 2026 AI chip war landscape. For the broader market context, our analysis of NVDA's $4.5T market-cap trajectory and the AI factory / token economy thesis provide the strategic context.

NVIDIA Groq 3 LPX Rack 35 Tokens per Watt Partnership 2026

Why a hybrid LPU + GPU rack

Related Articles

OpenAI DevDay 2026: What to Expect on Sep 29

OpenAI API Pricing 2026 Per Million Tokens by Model

NVIDIA Hyperscaler CapEx $1.3 Trillion 2027: AI Boom

NVIDIA Q2 FY2027 Revenue $96B: Vera Rubin Ramps

Try It in the Browser

Architecture

Deployment model