RTX 4080 for Local AI: A Year of Hands-On Verdicts

I bought an RTX 4080 in mid-2025 with one job: run AI locally without renting someone else’s GPU. A year later, after thousands of ComfyUI generations and more llama.cpp benchmark runs than I care to count, here’s the honest scorecard.

The VRAM reality check

16GB is the whole story of this card. It’s the most interesting number in the spec sheet because it decides everything you can and can’t run:

• LLMs: 7B and 8B models quantized to Q4 fit with room to spare. 13B Q4 fits. 70B Q4 does not — you’re in partial-offload territory and it hurts.
• Image models: SDXL runs comfortably. SD 3.5 and Flux are tight but workable at reduced resolutions. Video models are mostly out.
• The wall: the moment a model wants 17GB, the 4080 becomes a 3090 with worse cooling and more regret.

For 90% of local-AI hobbyist workloads, 16GB is the sweet spot. For the other 10%, it’s a cliff.

ComfyUI: where it shines

ComfyUI is where this card earns its keep. A year of image generation and the 4080 has been a workhorse:

• SDXL at 1024×1024: 6-8 iterations per second — a full 30-step generation in about 4 seconds.
• Flux.1 dev (fp8) at 1024: roughly 2-3 iterations per second with the right nodes; a 20-step image lands in 8-10 seconds.
• Video (Wan 2.1, short clips): doable but slow — this is where the VRAM ceiling bites hardest.

The 4080’s real party trick is parallel batches. Four SDXL generations at once run at nearly the same speed as one, because VRAM is the bottleneck, not compute. If you’re generating concept boards or dataset variations, it’s effectively a 4x speedup for free.

llama.cpp: the token numbers

For LLM inference, the numbers that matter (Q4_K_M quantizations, context around 8k):

Model	Tokens/second
Llama 3.1 8B / Qwen 2.5 7B	90-120
Llama 3.3 70B (offloaded)	8-12
DeepSeek-R1-Distill 14B	45-55

Interactive chat on 7-14B models is genuinely comfortable. The 70B numbers are why people keep saying VRAM is the only spec that matters — compute is never the bottleneck, the 4080 just runs out of memory first.

What I’d change

• Buy it for ComfyUI and 7-14B LLMs: it’s arguably the best value card for this exact workload.
• Skip it for serious LLM work: if your primary goal is running 70B+ models, save for a 3090’s 24GB or a used 4090. The offload experience is not it.
• The power draw is modest for the class: ~320W under load, and the card sips power at idle — a real advantage in a homelab that runs 24/7.

The verdict

8.5/10 — but with a catch. The RTX 4080 is the best GPU I’ve owned for local image generation and small-to-mid LLM inference, full stop. It’s been running nearly continuously for a year in my lab and has never once failed me. The catch is the same one every 16GB card has: the ceiling is real, and the day you hit it, you’ll be shopping for 24GB. Buy it knowing exactly what it can’t do, and it will never disappoint you.