🚀 DataCenter GPU Resources & Compute Architecture

Balanced Pairing (General Workloads)

• CPU Configuration: 2x Intel Xeon 8480+ (56 cores) or AMD EPYC 9654 (96 cores)
• GPU Allocation: 4-8x NVIDIA L40S or A100 per dual-socket node
• Memory Ratio: 512GB-1TB system RAM : 48-80GB GPU memory per card
• PCIe Configuration: PCIe 5.0 x16 per GPU, NUMA-aware placement
• Use Cases: Mixed workloads, multi-user environments, development clusters

Compute-Dense Pairing (Training Workloads)

• CPU Configuration: 2x Intel Xeon 8490H (60 cores) or AMD EPYC 9684X (96 cores)
• GPU Allocation: 8x NVIDIA H100 SXM in NVLink domain
• Memory Configuration: 2TB system RAM, 640GB total GPU memory
• Interconnect: NVLink 4.0 at 900GB/s bidirectional between GPUs
• Use Cases: Large model training, distributed computing, HPC simulations

Inference-Optimized Pairing (Production Serving)

• CPU Configuration: Higher core count for request processing (2x AMD EPYC 9754)
• GPU Allocation: 16x NVIDIA L4 in 4U server for maximum density
• Memory Strategy: Lower GPU memory per card, higher system memory for caching
• Network: 200GbE+ for low-latency request handling
• Use Cases: Real-time inference, API serving, edge deployment

GPU Device Plugins Configuration

# NVIDIA GPU Operator Deployment
apiVersion: v1
kind: Namespace
metadata:
  name: gpu-operator
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin-daemonset
  namespace: gpu-operator
spec:
  selector:
    matchLabels:
      name: nvidia-device-plugin-ds
  template:
    metadata:
      labels:
        name: nvidia-device-plugin-ds
    spec:
      containers:
      - image: nvcr.io/nvidia/k8s-device-plugin:v0.14.1
        name: nvidia-device-plugin-ctr
        env:
        - name: MIG_STRATEGY
          value: "mixed"  # Support both MIG and full GPU allocation
        - name: FAIL_ON_INIT_ERROR
          value: "false"
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
        volumeMounts:
        - name: device-plugin
          mountPath: /var/lib/kubelet/device-plugins
      volumes:
      - name: device-plugin
        hostPath:
          path: /var/lib/kubelet/device-plugins
                    

Multi-Instance GPU (MIG) Configuration

# Enable MIG mode on H100 GPUs
nvidia-smi -i 0 -mig 1

# Create MIG instances (example for H100 80GB)
nvidia-smi mig -cgi 1g.10gb,2g.20gb,3g.40gb -C

# Verify MIG configuration
nvidia-smi mig -lgip

# Kubernetes MIG resource requests
apiVersion: v1
kind: Pod
metadata:
  name: training-job-small
spec:
  containers:
  - name: pytorch-training
    image: nvcr.io/nvidia/pytorch:23.08-py3
    resources:
      limits:
        nvidia.com/mig-1g.10gb: 1  # Request 1g.10gb MIG slice
        memory: "16Gi"
        cpu: "8"
      requests:
        nvidia.com/mig-1g.10gb: 1
        memory: "16Gi"
        cpu: "8"
  nodeSelector:
    accelerator: nvidia-h100-mig
                    

Advanced GPU Scheduling

# Node affinity for GPU topology awareness
apiVersion: v1
kind: Pod
metadata:
  name: distributed-training
spec:
  containers:
  - name: training-worker
    image: nvcr.io/nvidia/pytorch:23.08-py3
    resources:
      limits:
        nvidia.com/gpu: 8  # Full 8-GPU node
    env:
    - name: NCCL_IB_DISABLE
      value: "0"  # Enable InfiniBand for NCCL
    - name: NCCL_NET_GDR_LEVEL
      value: "PIX"  # Enable GPUDirect RDMA
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node.kubernetes.io/instance-type
            operator: In
            values: ["gpu-h100-8x"]
          - key: topology.kubernetes.io/zone
            operator: In
            values: ["zone-a"]  # Co-locate for low latency
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app: distributed-training
          topologyKey: kubernetes.io/hostname
                    

GPU Power Profiles

• NVIDIA H100 SXM: 700W TGP, dynamic power scaling based on workload
• NVIDIA A100 SXM: 400W TGP, enterprise efficiency optimizations
• NVIDIA L4: 72W TGP, inference-optimized power consumption
• AMD MI250X: 560W TGP, advanced power management features

Power Management Commands

# GPU power monitoring
nvidia-smi --query-gpu=power.draw,power.limit --format=csv -l 1

# Set power limits (requires admin privileges)
nvidia-smi -pl 400  # Set power limit to 400W

# Power efficiency monitoring
nvidia-smi --query-gpu=power.draw,utilization.gpu --format=csv -l 1

# Temperature and throttling monitoring
nvidia-smi --query-gpu=temperature.gpu,temperature.memory,clocks_throttle_reasons.hw_slowdown --format=csv -l 1
                    

Cooling System Design

• Liquid Cooling: Direct-to-chip liquid cooling for high-density GPU deployments
• Airflow Management: Hot-aisle/cold-aisle configuration with 35-40°C inlet temperature
• Thermal Throttling: Automatic frequency reduction at 83°C+ to prevent damage
• Fan Curves: Dynamic fan speed control based on GPU temperature and workload