A creative AI enthusiast has repurposed a noisy and obsolete enterprise GPU to build a powerful local large language model (LLM) inference machine for just $266. The core of the project is an Nvidia Tesla V100 SXM2 card with 16GB HBM2 memory, paired with an RTX 4080. The result is a system with 32GB total VRAM that can run a 27-billion-parameter model at 32 tokens per second, described as "fast enough for interactive use" and faster than many cloud APIs.
Cheap Tesla V100 SXM2 sourced on eBay for under $140
Oscar Molnar, the builder, shared details on a forum about purchasing a Tesla V100 SXM2 on eBay for a low price. These data-center GPUs are now affordable due to newer architectures. However, they are not plug-and-play: Molnar also bought an SXM2-to-PCIe adapter for about $66 to connect the card to a standard PC.
Sponsored Protocol
The "fan from hell" tamed with a motherboard PWM mod
The biggest challenge was noise. The stock cooler measured 82 dB, described as "somewhere between a garbage disposal and a lawnmower." Molnar solved this by rerouting the fan wires to the motherboard PWM header using a 2.54mm male to PH2.0 female jumper cable. At just 10% speed, the GPU stays under 50°C under full load, silent and cool.
32GB total VRAM working with NixOS and legacy Nvidia driver
To make the Volta-based Tesla and Ada-based RTX 4080 work together, Molnar used NixOS with a legacy Nvidia driver that supports both architectures. This gives the system 32GB VRAM, enough to load large models without expensive cloud servers. A test with a 27-billion-parameter LLM achieved 32 tokens per second, proving that older hardware can still be competitive for local AI.
Sponsored Protocol
This DIY approach shows how to achieve high LLM inference performance on a budget. For those concerned about AI security, it's worth noting that tools like autonomous hackers from OpenAI make local model control even more important. Additionally, projects like Massachusetts V2G pilot program show how technology evolves, but for local AI, a PC with a Tesla V100 remains a solid choice.
For further reading, check the Wikipedia page on Nvidia Tesla or the original article on Tom's Hardware.
