---## 1. 目標平台**等級：NVIDIA Vera Rubin NVL72／機架級 AI 超級電腦** **用途：LLM／MoE 訓練、後訓練、Agentic AI、大規模推論、HPC、長上下文、分散式 AI**
---## 2. 核心運算規格- **機櫃：1 組 Vera Rubin NVL72 Rack**- **運算抽屜：18 組 1U Liquid-Cooled Compute Tray**- **GPU：72 顆 NVIDIA Rubin GPU**- **CPU：36 顆 NVIDIA Vera CPU**- **GPU：每組 Compute Tray 4 顆 Rubin GPU**- **CPU：每組 Compute Tray 2 顆 Vera CPU**- **CPU Core：3,168 個 NVIDIA Olympus 自訂核心**- **CPU Threads：6,336 Threads**- **GPU：72 顆形成單一 NVLink Scale-Up Domain**- **機櫃級 AI 加速器：72-GPU 全互連架構**
---## 3. GPU- **GPU 架構：NVIDIA Rubin**- **GPU 數量：72 顆**- **單 GPU HBM4：288 GB**- **總 GPU HBM4：20.7 TB**- **單 GPU HBM4 頻寬：最高約 22 TB/s**- **全櫃 HBM4 頻寬：最高約 1.58 PB/s**- **第三代 NVIDIA Transformer Engine**- **NVFP4 Tensor Core 加速**- FP8- FP6- BF16- FP16- TF32- FP32- FP64
### GPU 峰值運算能力- **NVFP4 Inference：3,600 PFLOPS**- **NVFP4 Training：2,520 PFLOPS**- **FP8／FP6 Training：1,260 PFLOPS**- **INT8 Dense：18 POPS**- **FP16／BF16 Dense：288 PFLOPS**- **TF32 Dense：144 PFLOPS**- **FP32：9,360 TFLOPS**- **FP64：2,400 TFLOPS**- **FP32 SGEMM：28,800 TFLOPS**- **FP64 DGEMM：14,400 TFLOPS**
---## 4. CPU- **CPU 架構：NVIDIA Vera**- **CPU 數量：36 顆**- **CPU Core：每顆 88 個 NVIDIA Olympus Core**- **CPU Threads：每顆 176 Threads**- **總 Core：3,168**- **總 Threads：6,336**- **CPU 記憶體：LPDDR5X**- **單 CPU：最高 1.5 TB LPDDR5X**- **全櫃：最高 54 TB LPDDR5X**
---## 5. 全櫃高速記憶體- **HBM4：20.7 TB**- **LPDDR5X：54 TB**- **Total Fast Memory：75 TB**- GPU HBM4：主要承載模型權重、Activation、KV Cache、Tensor／Expert Parallel 資料- CPU LPDDR5X：Host／Orchestration／資料處理／I/O／Agent 工作負載- 支援超大型 Dense LLM- 支援 MoE- 支援長上下文- 支援大規模 KV Cache- 支援多模型共存
---## 6. 晶片內部互連### NVLink-C2C- **NVIDIA NVLink-C2C**- **最高 65 TB/s 級頻寬**- Vera CPU ↔ Rubin GPU 高速互連- CPU／GPU 協同記憶體資料交換
### NVIDIA NVLink 6- **第六代 NVIDIA NVLink**- **單 GPU：最高 3.6 TB/s 雙向 GPU-to-GPU 頻寬**- **全 NVL72：約 260 TB/s Aggregate Scale-Up Bandwidth**- 72 GPU 全互連- Non-Blocking All-to-All- Uniform GPU-to-GPU Connectivity- SHARP In-Network Computing- GPU Collective Operations Offload- All-Reduce- Reduce- Broadcast- Tensor Parallel- Expert Parallel- Pipeline Parallel- Sequence Parallel
---## 7. NVLink Switch- **9 組 NVIDIA 第六代 NVLink Switch Tray**- 每個 Switch Tray：4 顆 NVLink 6 Switch Chip- 全櫃 72 GPU 全互連- **Rack-Level NVLink：最高約 260 TB/s**- 支援 Switch Tray Hot-Swap／高可用設計- NVLink Spine 架構- 約 5,000 條級互連線纜／連接架構（依實際 OEM 實作）
---## 8. Scale-Out Networking### NVIDIA ConnectX-9- **144+ 個 NVIDIA ConnectX-9 SuperNIC 端點**- 每 GPU 約 1.6 Tb/s 級 Compute Fabric- **全櫃 Network Bandwidth：28.8 TB/s 雙向級**- 800 Gb/s- NVIDIA InfiniBand- NVIDIA Ethernet- RDMA- GPUDirect RDMA- GPU Direct Async- Multi-Rail Networking- GPU-Affinity Networking
### NVIDIA Quantum-X800- 800 Gb/s InfiniBand Fabric- NDR／XDR 世代級高效能互連架構- SHARP In-Network Computing- 大規模 All-Reduce 加速
### NVIDIA Spectrum-X- 800 Gb/s Ethernet- RoCE- AI Fabric- Lossless／低延遲 AI Networking- 多 Rack Scale-Out
---## 9. DPU- **18 顆 NVIDIA BlueField-4 DPU**- 每 Compute Tray 1 顆- **400 Gb/s 級介面**- Network Offload- Storage Offload- Security Offload- Infrastructure Isolation- Telemetry- Provisioning- Zero-Trust Infrastructure- Confidential Computing Infrastructure
---## 10. 本地儲存每個 Compute Tray：- **4 × E1.S Gen6 NVMe SSD bays**- **1 × E1.S Gen5 NVMe Boot SSD bay**
全櫃：- 最多 **72 × E1.S Gen6 NVMe**- **18 × E1.S Gen5 Boot NVMe**- Gen6：Dataset Cache／Model Cache／Checkpoint／Temporary Tensor Data- Gen5：OS／Boot／Recovery
---## 11. 外部高速儲存建議另建 Storage Fabric：- NVMe-oF- GPUDirect Storage- All-Flash NVMe Storage- Lustre- BeeGFS- WEKA- VAST- DDN- S3-compatible Object Storage- Distributed Checkpoint Storage- Model Registry- Dataset Repository- AI Context／KV Storage
### 建議容量- 熱資料層：100–500 TB NVMe- AI Dataset Tier：1–5 PB- Cold/Object Storage：10–100+ PB- Checkpoint Storage：至少 2–3 倍主要模型容量
---## 12. I/O 架構- NVIDIA NVLink 6- NVLink-C2C- PCIe Gen6 級 I/O- 800 Gb/s InfiniBand- 800 Gb/s Ethernet- RDMA- GPUDirect RDMA- GPUDirect Storage- NVMe-oF- Multi-Rail Fabric- SHARP
---## 13. 冷卻系統- **100% Direct Liquid Cooling**- GPU Cold Plate- CPU Cold Plate- NVLink／Switch 液冷- 高速網路／ASIC 液冷- DPU／NIC 液冷- Rack-Level Coolant Manifold- Blind-Mate Liquid Coupling- Flow Monitoring- Pressure Monitoring- Coolant Temperature Monitoring- Leak Detection- N+1 Pump Redundancy- In-Row CDU- 可選 In-Rack CDU- Facility Water System（FWS）- ASHRAE 液冷環境規格- **NVIDIA 公開參考入口水溫：45°C**
### OEM 參考實作- 4 × 110 kW Power Shelf- In-Row CDU- 部分 OEM 提供最高約 1.8 MW CDU 選項- 單 Rack 依完整實作可達 200 kW 級以上散熱能力
---## 14. 電力系統- **4 × 3RU 110 kW Power Shelf**- **最高約 440 kW 級 Power Shelf Capacity**- **54 V DC Rack Bus**- **5,000 A Bus Bar**- A/B 雙電源路徑- Redundant PSU- Hot-Swap PSU- Rack Power Telemetry- Dynamic Power Management- Power Capping- UPS- BBU- Facility-Level Power Distribution
---## 15. 機櫃結構- **48U Rack**- 19-inch 寬度等級- 約 600 mm × 1,200 mm × 2,300 mm 級尺寸（OEM 參考）- 18 × 1U Compute Tray- 9 × NVLink Switch Tray- 2 × OOB Management Switch- 4 × 3RU Power Shelf- 4 × NVLink Cable Cartridge- Front Service Access- Rear Liquid Manifold- Blind-Mate Service Architecture- Tool-Less／Rail Service Architecture
---## 16. 管理與 BMC- Host BMC- **Redfish API**- IPMI 相容管理能力- Out-of-Band Management- 1 GbE／10 GbE Management Ethernet- Remote Power Control- Serial-over-LAN- Hardware Inventory- Firmware Inventory- Firmware Lifecycle Management- GPU Health- CPU Health- DPU Health- NIC Health- NVLink Health- Thermal Telemetry- Power Telemetry- Error Telemetry- Remote Diagnostics
### 安全管理- Secure Boot- TPM 2.0- Firmware Secure Verification- Remote Attestation- GPU Confidential Computing- Rack-Level Security- Zero-Trust- RBAC- mTLS- Secrets Management- Firmware／Container Image Signing- SBOM- CVE Scanning
---## 17. 作業系統### NVIDIA DGX 路線- **NVIDIA DGX OS 7**- Ubuntu 24.04 LTS 基礎環境- NVIDIA 驗證 Driver／CUDA／NCCL／cuDNN 版本矩陣
### 通用 OEM 路線- Ubuntu 24.04 LTS- RHEL 相容企業 Linux- Rocky Linux／其他 Enterprise Linux（視 OEM／軟體支援矩陣）
---## 18. NVIDIA AI 軟體堆疊### CUDA／低階加速- CUDA Toolkit- CUDA-X- cuDNN- NCCL- cuBLAS- cuSPARSE- cuFFT- NVRTC- CUDA Graphs
### 大模型訓練- PyTorch- Megatron Core- NVIDIA NeMo- FSDP- Tensor Parallelism- Pipeline Parallelism- Expert Parallelism- Sequence Parallelism- Distributed Data Parallel- Activation Checkpointing- Mixed Precision- FP8／FP6／NVFP4
### 推論- TensorRT- TensorRT-LLM- NVIDIA NIM- NVIDIA Dynamo- Triton Inference Server- vLLM- SGLang- Continuous Batching- Dynamic Batching- Speculative Decoding- KV-Aware Routing- Prefill／Decode Disaggregation- Distributed KV Cache
### AI Agent- NVIDIA NeMo- NVIDIA NIM- NVIDIA Dynamo- Agentic AI Runtime- Tool Calling- Multi-Agent Orchestration- Long Context- Retrieval／RAG- KV Cache Management
---## 19. 容器與叢集- Docker／OCI- NVIDIA Container Toolkit- NVIDIA NGC- Kubernetes- NVIDIA GPU Operator- NVIDIA Network Operator- NVIDIA Device Plugin- DCGM Exporter- Slurm- Run:ai- GPU Quota- Multi-Tenant GPU Scheduling- GPU／NUMA／NIC／Storage Affinity
---## 20. 監控與可觀測性- NVIDIA DCGM- Prometheus- Grafana- OpenTelemetry- NVIDIA Mission Control- GPU Utilization- GPU Temperature- GPU Power- HBM Usage- HBM ECC- CPU Usage- CPU Memory Usage- NVLink Bandwidth- NVLink Error Counter- RDMA Counter- Network Throughput- Network Packet Error- DPU Health- Storage IOPS- Storage Latency- TTFT- ITL- Tokens/s- Requests/s- Queue Depth- KV Cache Utilization
---## 21. RAS／高可靠性- ECC HBM4- ECC System Memory- NVLink Error Detection- GPU Fault Detection- CPU Fault Detection- DPU Fault Isolation- NIC Fault Isolation- NVLink Switch Fault Isolation- PSU Redundancy- A/B Power Feed- N+1 Pump- Liquid Leak Detection- Thermal Protection- Predictive Failure Detection- Firmware Rollback- Component-Level Isolation- Rack-Level RAS- Health Check Automation- Graceful Degradation- Checkpoint／Restore
---# 22. 最終頂級規格清單- **1 × NVIDIA Vera Rubin NVL72 Rack**- **18 × 1U Liquid-Cooled Compute Tray**- **72 × NVIDIA Rubin GPU**- **36 × NVIDIA Vera CPU**- **3,168 CPU Cores**- **6,336 CPU Threads**- **20.7 TB HBM4**- **54 TB LPDDR5X**- **75 TB Total Fast Memory**- **最高約 1.58 PB/s HBM4 Bandwidth**- **65 TB/s NVLink-C2C 級頻寬**- **260 TB/s NVLink 6 Scale-Up Fabric**- **9 × NVLink 6 Switch Tray**- **144+ × ConnectX-9 SuperNIC**- **18 × BlueField-4 DPU**- **28.8 TB/s 級 Scale-Out Network Bandwidth**- **800 Gb/s InfiniBand／Ethernet Fabric**- **18 × Gen5 Boot NVMe**- **72 × Gen6 NVMe bays**- **100% Direct Liquid Cooling**- **4 × 110 kW Power Shelf**- **54 V DC／5,000 A Rack Bus**- **約 440 kW 級 Power Shelf Capacity**- **BMC + Redfish**- **Secure Boot + TPM 2.0**- **Confidential Computing**- **NVIDIA Mission Control**- **NVIDIA DGX OS 7／Ubuntu 24.04 LTS**- **CUDA／CUDA-X／NCCL／cuDNN**- **TensorRT／TensorRT-LLM**- **NeMo／Megatron Core**- **NIM／Dynamo／Triton**- **PyTorch／vLLM／SGLang**- **Kubernetes／Slurm／Run:ai**- **DCGM／Prometheus／Grafana／OpenTelemetry**- **NVMe-oF／GPUDirect Storage**- **Lustre／BeeGFS／WEKA／VAST／DDN**- **Zero-Trust／RBAC／mTLS／SBOM／CVE Scan**---