Can 9 year old GPU beats today's latest GPUs?

A 9-year-old NVIDIA Tesla V100 rig beat two brand-new DGX Sparks at local LLM token generation: 155 tok/s against about 60 on Qwen 3.8 Flash Next (NVFP4, vLLM). Here's why old HBM2 bandwidth wins, and why the speed really comes from community software. The build: four used V100 SXM2 32GB cards on an NVLink carrier board, 128GB of VRAM for about $3,500. The other side is a DGX Spark Founders Edition plus a Gigabyte AI TOP Atom, two GB10 Grace Blackwell boxes clustered over ConnectX-7 (the same GB10 Superchip behind the new RTX Spark laptops). We break down prompt processing vs token generation, 900 GB/s vs 273 GB/s, and why an 8x RTX Pro 6000 server still loses on decode. We also cover how 1CatAI's 1Cat-vLLM fork made the same four cards 310% faster in a single release, and whether a used-GPU home AI server is worth building in 2026. In this video: - Why a 2017 V100 beats the DGX Spark at token generation (900 GB/s HBM2 vs 273 GB/s LPDDR5x) - Prefill vs decode: prompt processing is compute-bound, token generation is memory-bound - 1Cat-vLLM v1.5.0 → v1.5.1: 49 → 201 tok/s on the same V100s - NVFP4 on Volta: running a 4-bit model on a chip with no FP4 hardware - 8x RTX Pro 6000 Blackwell over PCIe: 126 tok/s, and why multi-GPU over PCIe can get slower - v100-skinny: 4x V100 16GB (A$600) vs an RTX 5090, 219 vs 214.7 tok/s on Qwen 3.8 27B - GTX 1080 Ti vs RTX 5060: why bandwidth on paper isn't enough - Home AI server costs: build price, 600–800 W power draw, Qwen 3.8 Flash API at $0.47/M tokens, payback - CUDA 13, PyTorch 2.14 and the driver 580 deadline (mid-2028) - Who should build a V100 AI home lab, and who should just buy a DGX Spark Jun's benchmark also includes Alex Ziskind's numbers for Apple's M5 Ultra (the Mac Studio chip): 44 tok/s on llama.cpp and 125 tok/s with oMLX. Chapters 0:00 A 9-year-old GPU vs DGX Spark 3:13 The 4x V100 SXM2 NVLink build 6:31 Blackwell vs Volta on the spec sheet 9:18 Prefill vs decode 12:20 Memory bandwidth: 900 vs 273 GB/s 14:45 1Cat-vLLM: +310% in one release 18:57 8x RTX Pro 6000 vs 4x V100 22:39 v100-skinny vs RTX 5090 24:07 What about the GTX 1080 Ti? 27:48 Build cost, power and payback 32:39 Who should build one 37:28 Verdict Sources Jun Yamog, "9 year old GPU beats today's latest?": https://www.youtube.com/watch?v=s6981RLmH0Q Alex Ziskind, 8x RTX Pro 6000: https://www.youtube.com/watch?v=C-31AX2TZ9Y 1Cat-vLLM (V100 / SM70 fork of vLLM): https://github.com/1CatAI/1Cat-vLLM v100-skinny: https://github.com/dnv2003/v100-skinny v100-skinny on r/LocalLLaMA: https://www.reddit.com/r/LocalLLaMA/comments/1vsq3zg/ llama.cpp CUDA scoreboard: https://github.com/ggml-org/llama.cpp/discussions/15013 DGX Spark hardware specs: https://docs.nvidia.com/dgx/dgx-spark/hardware.html DGX Spark 64GB pricing: https://www.storagereview.com/news/nvidia-dgx-spark-64gb-october-23-4999-two-units-cluster-to-128gb NVIDIA on CUDA 13 and driver 580: https://developer.nvidia.com/blog/navigating-gpu-architecture-support-a-guide-for-nvidia-cuda-developers/ PyTorch Volta wheel notice: https://dev-discuss.pytorch.org/t/notice-cuda-12-6-wheels-will-no-longer-be-published-from-pytorch-2-15-drops-maxwell-pascal-volta/3432 Qwen 3.8 Flash API pricing: https://openrouter.ai/qwen/qwen3.8-flash #V100 #DGXSpark #LocalLLM #LocalAI #vLLM