Groq 3 LPX specification reference
NVIDIA Groq 3 LPX Specs: LPU, SRAM and Rack Architecture
The Groq 3 LPX rack combines 256 Groq 3 LPU accelerators into one low-latency inference system. NVIDIA publishes unusually large internal bandwidth figures because the architecture depends on local SRAM and direct chip-to-chip communication. This page separates per-LPU, per-tray and per-rack specifications so the numbers are not mixed together.
Interactive calculator
Groq 3 LPX Rack Specification Calculator
Enter your own workload or capacity assumptions. Public LPX benchmarks are workload-specific, and this calculator does not certify application performance, electrical design, cooling or OEM compatibility.
Live Amazon supporting hardware
Current supporting components and Price Options
Compare current Amazon listings relevant to this guide. Product availability and prices can change.
Quick answer
The headline LPX rack specs
NVIDIA lists 256 LPUs, 128 GB aggregate SRAM, 40 PB/s SRAM bandwidth, 640 TB/s scale-up bandwidth, 12 TB DDR5 and 315 PFLOPS of FP8 inference compute per LPX rack. A compute tray contains eight LP30 chips with 4 GB SRAM, 1.2 PB/s SRAM bandwidth and 20 TB/s scale-up bandwidth.
Use specs to validate architecture, not to predict your application TPS
Memory and bandwidth figures explain why LPX is built for deterministic low-latency inference, but they do not translate directly into tokens per second for an arbitrary model. Use the calculator to aggregate rack resources, then use model-specific benchmarks for service sizing.
Groq 3 LPX specifications by component level
Per-LPU, tray and rack numbers come from current NVIDIA product and technical material.
| Specification | Per LPU | Per compute tray | Per LPX rack |
|---|---|---|---|
| LPU count | 1 | 8 | 256 |
| SRAM capacity | 500 MB | 4 GB | 128 GB |
| SRAM bandwidth | 150 TB/s | 1.2 PB/s | 40 PB/s |
| Scale-up bandwidth | 2.5 TB/s | 20 TB/s | 640 TB/s |
| FP8 inference compute | Not stated here | 9.6 PFLOPS | 315 PFLOPS |
| DDR5 via rack architecture | Not a simple per-chip retail figure | Up to 256 GB via fabric expansion plus host paths described by NVIDIA | 12 TB aggregate rack memory |
Before you use the result for procurement
Benchmark the exact workload
Keep the model, context length, output length, concurrency, precision and serving-software version beside every TPS result.
Separate context from generation
LPX is positioned around low-latency token generation; Rubin and Rubin CPX handle different context-heavy work. Measure both phases.
Treat rack specs as architecture
LPUs, SRAM and internal bandwidth explain the system design but do not by themselves predict application throughput.
Verify the current OEM design
Use qualified system documentation for rack power, cooling, networking, serviceability and supported configuration before procurement.
Start with the system boundary before reading any specification
LPX numbers appear at several levels: one LPU, one compute tray and the complete rack. Confusion happens when a per-chip memory figure is compared with a rack figure or when internal bandwidth is treated as an external network rate. A specification is only meaningful when its physical boundary is clear.
The table above keeps those boundaries explicit. The calculator then scales only the rack-level quantities by whole racks. It does not infer unsupported per-chip values from rack totals when NVIDIA has published a separate figure.
A full rack contains 256 Groq 3 LPU accelerators
NVIDIA describes the rack as 256 interconnected LPU chips. Technical material identifies the chip as LP30 and shows eight chips per compute tray, which implies 32 compute trays of eight chips when counting 256 processors. System packaging may include other trays and infrastructure, so this arithmetic should not be mistaken for an OEM rack assembly diagram.
For software and capacity planning, the more important point is that the accelerators participate in a tightly coordinated system. The compiler and chip-to-chip fabric are designed around the complete scale-up domain, so raw processor count does not imply the same scheduling model as 256 independent PCIe devices.
Each LPU carries 500 MB of SRAM
NVIDIA publishes 500 MB of SRAM per LPU. SRAM is expensive in area but offers very fast, predictable access compared with large off-chip memory. The architecture uses that property for latency-sensitive inference where waiting on memory movement can directly affect token-generation time.
Multiplying 500 MB by 256 produces the published 128 GB rack total using decimal units. The number is intentionally much smaller than the multi-terabyte pools elsewhere in the system. LPX relies on a hierarchy rather than trying to put every model parameter in SRAM.
SRAM bandwidth reaches 150 TB/s per LPU and 40 PB/s per rack
The on-chip memory bandwidth figure is one of the most distinctive LPX specifications. NVIDIA lists 150 TB/s per LPU, 1.2 PB/s per eight-chip tray and 40 PB/s across the rack. Small differences from simple multiplication can reflect published rounding, so use official aggregate figures when citing the full system.
This bandwidth is local accelerator memory traffic. It is not a 40-PB/s Ethernet fabric and should never be used to size external switches. The purpose is to keep compute fed with predictable low-latency data access inside the inference engine.
The chip-to-chip fabric provides 640 TB/s of rack scale-up bandwidth
Each LPU is published with 2.5 TB/s of scale-up bandwidth. Across the rack, NVIDIA lists 640 TB/s. The long-context technical blog also describes 96 chip-to-chip links per chip at 112 Gbps each, giving the compiler detailed topology information it can use when scheduling transfers.
Scale-up bandwidth is about coordinated processor communication inside the LPX rack. It is separate from scale-out networking between racks or clusters. Procurement teams should keep those bill-of-materials layers distinct because one is integral to the LPX architecture while the other depends on the data-center network design.
The LPX rack includes 12 TB of DDR5 memory
NVIDIA's product page lists 12 TB of DDR5 per LPX rack alongside the 128 GB SRAM pool. The large DDR5 capacity supports models and workloads that cannot reside entirely in the low-latency on-chip memory. The combined design gives the compiler and serving software access to memory tiers with very different capacity and bandwidth characteristics.
Do not treat the 12-TB figure as a shopping list of standard retail DIMMs. Exact modules, topology, serviceability and supported configuration are system-vendor decisions. Cloudzat therefore does not label Amazon RDIMMs as LPX replacement parts.
Rack-level FP8 compute is published as 315 PFLOPS
NVIDIA lists 315 PFLOPS of FP8 AI inference compute for the LPX rack, while the technical blog lists 9.6 PFLOPS per compute tray. Peak arithmetic capability helps describe the platform but does not independently determine application performance. Model operations, communication and memory behavior all influence delivered tokens.
Precision also matters. A peak FP8 figure cannot be compared with a GPU number at another precision without understanding the model path and quality requirement. Use performance benchmarks that run the same model and output-quality settings.
Compute-tray specifications help explain the modular design
NVIDIA's technical deep dive describes a tray with eight LP30 chips, 4 GB SRAM, 1.2 PB/s SRAM bandwidth, up to 256 GB DRAM through fabric expansion logic, up to 128 GB via the host CPU, 9.6 PFLOPS FP8 and 20 TB/s scale-up bandwidth. Those values show how the rack is assembled from repeatable compute building blocks.
Tray-level information is useful for architecture education and service planning, but buyers should still procure and support the platform through approved system channels. A compute tray is not equivalent to a consumer add-in card that can be mixed into an arbitrary server.
Internal bandwidth does not remove the need for fast external networking
LPX can generate tokens quickly only if the rest of the service can deliver requests, context and results without creating a new bottleneck. Vera Rubin platforms use high-speed networking for scale-out communication between systems. ConnectX SuperNICs and Spectrum-X are part of that broader factory design, not part of the 640-TB/s internal LPU fabric.
When sizing a cluster, create separate bandwidth budgets for internal accelerator scale-up, host I/O, storage traffic and inter-rack model/service traffic. Combining them into one number makes it impossible to identify where congestion could occur.
Rack specifications should be paired with power and cooling data from the OEM
LPX is part of a liquid-cooled rack-scale generation. The public product page focuses on inference architecture and does not give enough information to engineer a facility from the headline performance table alone. Exact rack power, coolant conditions, CDUs, manifolds, branch circuits and protection equipment belong in the current vendor engineering package.
The rack calculator on this page deliberately avoids inventing electrical numbers. It aggregates only the published compute and memory resources. Facility planning should remain a separate signed-off engineering task.
Software version can change how much of the hardware is usable for a model
An impressive hardware specification becomes valuable only when the compiler and serving stack support the model efficiently. NVIDIA continues developing Dynamo and LPX software paths, so model coverage and execution strategies can evolve after the physical rack is installed.
Record software versions whenever you benchmark. If a later compiler release improves scheduling or model support, rerun the service tests before assuming the original capacity model still applies. This is one reason Cloudzat separates static architecture specs from dynamic performance sizing.
Use aggregate calculations for planning, but avoid fake precision
It is reasonable to multiply 256 LPUs, 128 GB SRAM or 640 TB/s scale-up bandwidth by the number of racks when describing total installed resources. It is not reasonable to multiply a single benchmark TPS result by rack count and call the result guaranteed fleet throughput without accounting for utilization, routing, redundancy and workload variance.
The calculator therefore shows installed resources and asks separately for a target TPS planning number. To turn that target into a deployment, use the dedicated rack-sizing calculator with measured sustained throughput and explicit headroom.
Recheck the official page before freezing a specification sheet
Groq 3 LPX is a new platform and NVIDIA states that features, pricing, availability and specifications are subject to change. OEM implementations can also expose different service, networking or facility details. A procurement document should therefore carry the source date and vendor configuration, not just a copied web table.
Cloudzat links the official product and technical pages below so a buyer can verify current figures. If a specification changes, treat the manufacturer page and signed OEM documentation as authoritative over this reference.
Methodology and sources
Specification values are taken from NVIDIA's LPX product page and technical deep dive. Cloudzat distinguishes per-LPU, per-tray and per-rack boundaries and avoids deriving facility or application-performance claims from unrelated peak figures.
- NVIDIA Groq 3 LPX product page
- NVIDIA technical blog: inside Groq 3 LPX
- NVIDIA technical blog: LPX long-context interactivity
- NVIDIA Groq 3 LPX full-production announcement
- NVIDIA Vera Rubin NVL72
- NVIDIA technical blog: Vera Rubin POD architecture
As an Amazon Associate, Cloudzat may earn from qualifying purchases. Marketplace listings on these pages cover adjacent networking, storage, memory or power hardware only. They are not represented as LP30 chips, LPX trays or qualified Vera Rubin rack components. Verify exact models, warranty, interfaces and current OEM documentation before purchase.
Frequently asked questions
How many LPUs are in a Groq 3 LPX rack?
256.
How much SRAM is in one Groq 3 LPU?
NVIDIA lists 500 MB per LPU.
How much SRAM is in the full LPX rack?
128 GB aggregate SRAM.
What is the SRAM bandwidth per LPX rack?
NVIDIA publishes 40 PB/s of aggregate on-chip SRAM bandwidth.
What is LPX scale-up bandwidth?
NVIDIA lists 2.5 TB/s per LPU, 20 TB/s per compute tray and 640 TB/s across the rack.
How much DDR5 memory is in an LPX rack?
The published rack figure is 12 TB.
What is the LPX rack FP8 compute figure?
NVIDIA lists 315 PFLOPS of FP8 AI inference compute.
How many LPUs are on a compute tray?
The technical deep dive lists eight LP30 chips per compute tray.
Does 40 PB/s mean the rack needs a 40-PB/s Ethernet network?
No. That is internal SRAM bandwidth, not external network bandwidth.
Can I use these specs to calculate exact tokens per second?
No. Delivered TPS depends on the model, context, serving software and system configuration. Use application benchmarks for capacity sizing.