TB
Tech Boss Interactive Tools Suite
🎛️ Multi-GPU Local AI

Dual-GPU VRAM Split Calculator (llama.cpp & Ollama)

⚡ Mismatched Dual-GPU Layer Balancing Enter both GPU VRAM sizes and your target GGUF model to compute combined usable VRAM, display-headroom safety margin, and exact llama.cpp --tensor-split weights.

Engineering Mismatched Dual-GPU Rigs for Local LLMs

You do not need two identical flagship cards to run 48GB+ local language models. Pairing a used 24GB RTX 3090 with an affordable 12GB RTX 3060 gives you 36GB of raw VRAM—enough to run 32B models at Q6_K/Q8_0 precision or 70B models at IQ3_XS/IQ4_XS quantization without touching slow system DDR5.

Why GPU 0 Needs Extra Headroom

Your primary display GPU holds the Windows Desktop Window Manager (DWM), browser GPU buffers, and CUDA context overhead (typically 1.5–2.0 GB). Using an even 50/50 split on mismatched GPUs—or ignoring display overhead—causes instant Out-Of-Memory (OOM) crashes on GPU 0 while GPU 1 still sits half-empty.

PCIe Bandwidth & Power Supply Notes

Under --split-mode layer, only compact activation tensors cross the PCIe bus between GPUs, meaning even a PCIe Gen 3 x4 chipset slot works reliably for inference. Always verify combined transient wattage with our PC Power Supply Calculator.

Explore More Free Interactive Tools

View All 17 Tools →