Gaudi 2 Specifications
Gaudi 2 supports all popular data types required for deep learning: FP32, TF32, BF16, FP16 & FP8 (both E4M3 and E5M2).
96 GB
HBM2e Memory
Optimized capacity for FP8 models with large context window and batch.
2.4 Tbps
RoCE v2 Bandwidth
Fast GPU interconnect for training and multi-node inference.
865
TFLOPS FP8
Superior token cost and performance at FP8 precision.
2.8X
Faster Inference
vs A100 at FP8 performance of A100, and 1.4x at BF16.
NVIDIA B200 on Denvr AI Cloud
1T Parameter Training
Scale across 8-GPU NVLink nodes for 1,440 GB of total VRAM and 1.8 TB/s per-GPU interconnect.
LLM Training & Inference
Native support for PyTorch and Hugging Face Optimum. Train and serve popular open-weight models including Llama, Mixtral, and Qwen.
Alternate Silicon
Evaluate non-NVIDIA accelerators to maximize your AI compute budget. Train or operate models via vLLM serving engine.
Managed Storage
High-performance Weka filesystem and local NVMe available. No external storage to provision for datasets, checkpoints, or model artifacts.
Configurations
Per-minute billing with on-demand and reserved options. All configurations available as bare metal, VM, or model endpoints.
Platform
GPUs
On-Demand
VRAM
vCPUs
Memory
Local Storage
Interconnect
Intel Gaudi 2
8
96 GB
160
1024 GB
4x 7.6TB NVMe
-
$1.25 / GPU