Gaudi 2 Specifications
Gaudi 2 supports all popular data types required for deep learning: FP32, TF32, BF16, FP16 & FP8 (both E4M3 and E5M2).
96 GB
HBM2e Memory
Optimized capacity for FP8 models with large context window and batch.
2.4 Tbps
RoCE v2 Bandwidth
Fast GPU interconnect for training and multi-node inference.
865
TFLOPS FP8
Superior token cost and performance at FP8 precision.
2.8X
Faster Inference
vs A100 at FP8 performance of A100, and 1.4x at BF16.
NVIDIA B200 on Denvr AI Cloud
1T Parameter Training
Scale across 8-GPU NVLink nodes for 1,440 GB of total VRAM and 1.8 TB/s per-GPU interconnect.
LLM Training & Inference
Native support for PyTorch and Hugging Face Optimum. Train and serve popular open-weight models including Llama, Mixtral, and Qwen.
Alternate Silicon
Evaluate non-NVIDIA accelerators to maximize your AI compute budget. Train or operate models via vLLM serving engine.
Managed Storage
High-performance Weka filesystem and local NVMe available. No external storage to provision for datasets, checkpoints, or model artifacts.
Configurations
Per-minute billing with on-demand and reserved options. All configurations available as bare metal, VM, or model endpoints.
Platform
GPUs
On-Demand
VRAM
vCPUs
Memory
Local Storage
Interconnect
Intel Gaudi 2
—
—
—
96 GB
—
—
—
PLATFORM
GPU COUNT
GPU VRAM
vCPUs
MEMORY
LOCAL STORAGE
INTERCONNECT
GPU Count
GPU VRAM
vCPUs
Memory
Local Storage
Interconnect
GPU Count
Related GPUs
Compare Denvr GPU options by workload and performance requirements.
Intel Gaudi 2
VRAM
VRAM
VRAM
VRAM
Memory Bandwidth
VRAM
VRAM
VRAM
FP64/FP32
VRAM
VRAM
VRAM
FP16
VRAM
VRAM
VRAM
FP8
VRAM
VRAM
VRAM
NVLink
VRAM
VRAM
VRAM
On-Demand Pricing
VRAM
VRAM
VRAM
PLATFORM
VRAM
BANDWIDTH
FP64/FP32
FP16
FP8
NVLINK
GPU Count
GPU VRAM
vCPUs
Memory
Local Storage
Interconnect
GPU Count
PLATFORM
VRAM
BANDWIDTH
FP64/FP32
FP16
FP8
NVLINK
GPU Count
GPU VRAM
vCPUs
Memory
Local Storage
Interconnect
GPU Count
PLATFORM
VRAM
BANDWIDTH
FP64/FP32
FP16
FP8
NVLINK
GPU Count
GPU VRAM
vCPUs
Memory
Local Storage
Interconnect
GPU Count
Infrastructure you can trust at scale
As an Intel Partner Alliance member we build and operate AI clusters following vendor reference architectures. Your models and data are supported via strict privacy safeguards and SOC 2 Type 2 security practices.
