Prices & benchmarks as of Nov 16, 2026 (US)

Running local LLMs:
what to actually buy

The amount of VRAM sets which models you can run. Memory bandwidth sets how fast they run. The connection (eGPU or desktop slot) barely matters once the model is loaded.

~2×

Used GPU prices since early 2026. An AI-driven memory shortage pushed a used 3090 from ~$800 to ~$1,600.

tok/s ≈ BW ÷ size

Each generated token reads every weight once. Real speeds land at ~60–75% of bandwidth ÷ model size.

0–5%

Generation loss over OCuLink when the model fits in VRAM. Loading and CPU offload are where an eGPU hurts.

The market right now

Data centers are buying up the world's memory chips, so cards with lots of VRAM cost much more than they did earlier this year. Guides that quote a "$700 used 3090" or a "$150 P40" are out of date.

Price increase, earlier 2026 or MSRP → Nov 2026

Used = eBay sold or lowest listings. New = Tom's Hardware lowest price (Nov 3). The RTX 50 Super refresh is on hold over memory costs, so don't wait for it.

What fits, and how fast

Pick a model size. Bars show generation speed for one user at Q4 (4-bit) quantization. Gray rows can't fit the model in VRAM. Hover over a row for details.

Model
Show
NVIDIA AMD Intel Unified-memory box Estimate (no direct benchmark)

Reading speed is ~5–10 tok/s. At 20+ tok/s a chat feels instant. Above ~60 tok/s you only notice it in agents and batch jobs. Prompt processing (how fast it reads a long document) is a separate number, driven by compute, where NVIDIA is far ahead.

Four tiers

One pick per tier, chosen for value per dollar at November prices with the fewest software headaches. Alternatives are listed under each.

Cheapest that works
RTX 3060 12GB
~$300 used · 170W
  • 8B at 55 tok/s, 14B at 31 tok/s
  • CUDA, so every tool just works
  • Runs on a 550W PSU or any dock
  • Can't fit 32B; two of them (24GB) can, slowly
Tinkerers: P100 16GB ~$265 (EOL, needs a fan shroud) · Arc B580 12GB $329 new. Skip the P40 at $400 and the M40.
Mid
RTX 5060 Ti 16GB
~$770 new · 180W
  • 8B at 69 tok/s, 14B ~37
  • New with warranty, one 8-pin plug
  • Good for 14B–24B and MoE models
  • No 32B dense on one card
Need 32GB? Radeon AI PRO R9700 (~$1,300) or Arc Pro B70 (~$1,200). You trade in AMD/Intel software friction and slower prompt processing.
High · best value
RTX 3090 24GB (used)
~$1,500–1,700 used · 350W
  • 32B at 35 tok/s, 8B at 115
  • 24GB + 936 GB/s + CUDA
  • Two of them run 70B at 32 tok/s
  • Check VRAM temps; re-pad if needed
A 4090 (~$3,100) is only 13% faster at 32B; you'd mostly be paying for prompt speed. The used 7900 XTX ($1,440) lost its discount.
Ultimate
RTX PRO 6000 Blackwell 96GB
$12–16k · 600W (Max-Q 300W)
  • 70B on one card at 32 tok/s
  • gpt-oss-120B at 210 tok/s
  • No multi-GPU hassle
  • Fastest prompt processing
RTX 5090 32GB is faster per token (61 tok/s at 32B) but sells for $6–7k, 3× MSRP, for a third of the memory. Huge models: Mac Studio M5 Ultra 512GB ($22k), slower but holds far more.

If it's a cluster

SetupVRAMCost (GPUs)RunsNote
2× RTX 3060 12GB24GB~$60032B, tight (~11 tok/s est.)Cheapest route to 32B
2× RTX 309048GB~$3,20070B at 31.9 tok/s · 32B at 59.8The classic. Needs a 1,200–1,500W PSU
2–4× R9700 / Arc Pro B7064–128GB$2,400–5,20070B+, big MoEBlower cards made to stack; weaker software
Strix Halo mini PC (no GPU)128GB shared$3,300–4,300gpt-oss-120B 31–56 tok/s · 70B ~5Cheapest 128GB; good for MoE, slow on dense
DGX Spark128GB shared$6,950Same class as Strix HaloSame bandwidth as Strix Halo at twice the price. Skip for inference
Multi-GPU basics: llama.cpp's default layer split works with mismatched cards and slow links. It adds memory but not speed. Tensor split (-sm tensor or vLLM TP) adds speed (70B on 2×3090: 19.5 → 31.9 tok/s) but wants identical cards and x8+ PCIe links. Never run tensor-parallel over eGPU links.

The two buyers

Example parts lists at November prices.

A · GPU + external dock laptop / mini PC

First check the computer's ports: it needs OCuLink, Thunderbolt 3/4/5, or USB4. A plain USB-C port won't work.

Budget
RTX 3060 12GB (used)$300
Aoostar AG01 (OCuLink, 400W PSU built in)$150–180
Runs 8B–14B fast≈ $470
Recommended
RTX 3090 24GB (used)$1,600
Aoostar AG02 (OCuLink + USB4, 800W PSU built in)~$230
Runs 32B at ~35 tok/s≈ $1,830

Thunderbolt 5 only (no OCuLink)? Minisforum DEG2 (~$250) + Corsair RM850e (~$100), or Razer Core X V2 ($350) + PSU.

B · AI cluster desktop

Use a desktop with two x16-size slots (x8/x8 on AM5 is fine) and room for two 3-slot cards, or use risers.

Recommended
2× RTX 3090 (used)$3,200
1,200–1,500W ATX 3.1 PSU$190–350
Host (AM5 board, CPU, 64GB RAM) if needed$900+
70B at ~32 tok/s, 32B at ~60≈ $3,400–4,500

Power-limit each card to 280W (nvidia-smi -pl 280): under 1% slower, much less heat. Two cards stay within one 15A household circuit; four need two circuits.

Mostly want big MoE models (gpt-oss-120B, Qwen3 235B-A22B at low quant)? A 128GB Strix Halo box does it in one quiet machine.

You already have a desktop tower: put the GPU in it. Check three things first: PSU wattage and connectors (a 3090 needs 2–3× 8-pin and 750W+), case length (~300–330mm for a 3090), and a free x16 slot. A $100 new PSU is cheaper than any dock.

eGPU vs desktop slot

Measured bandwidth from host to GPU. The model loads once; after that, single-GPU generation sends only a few KB per token over the link.

Link bandwidth (GB/s)

What the link actually changes

WorkloadOCuLinkThunderbolt
Generation (model in VRAM)−0–5%−5–10%*
Model load time~4× slower~6–8× slower
Partial CPU offloadbadprompt speed −54%
Tensor-parallel multi-GPUavoidavoid
Hot-plugno, power off firstyes

*Small fast models (100+ tok/s) suffer more from Thunderbolt latency. One uncontrolled 2024 test saw −38% on a 7B model.

4090 over OCuLink

−2%

171.5 vs 175.4 tok/s in a desktop (Notebookcheck, Minisforum DEG2).

70B on 2× 3090 eGPUs

17.96 vs 17.83

tok/s, OCuLink+USB4 vs 2× Thunderbolt 3. With layer split, the link type doesn't matter.

Offload over TB3

304 → 140

Prompt tok/s, x16 vs Thunderbolt 3, with the model partly in system RAM. Keep the model fully in VRAM.

Mac: since April 2026 a signed driver (tiny corp's TinyGPU) allows compute on AMD/NVIDIA eGPUs, but only through tinygrad, and it's slower than the Mac's own chip on 8B models. Don't buy an eGPU for a Mac.

Docks

"Open frame" docks are bare boards: the GPU and PSU sit in the open. They're cheaper and fit any card. OCuLink is faster than Thunderbolt, but few laptops have it.

DockConnectionPowerPriceVerdict
Aoostar AG01OCuLinkBuilt-in 400W$149–179Budget pick for cards up to ~250W
Aoostar AG02OCuLink + USB4Built-in 800W~$229Best value. No PSU to buy; fine for a 3090/4090
Minisforum DEG1OCuLinkBring ATX/SFX~$110Cheapest if you have a good PSU
Minisforum DEG2OCuLink + TB5Bring ATX/SFX$240–299Best reviewed; one dock for any computer
Aoostar AG03OCuLink + 2× TB5Built-in 800W (GPU ≤500W)$249–279Newer, fewer reviews; no 5090
Aoostar EG022× TB5 (+OCuLink)Bring ATX/SFX$219–299Tight for 3-slot cards
Razer Core X V2TB5Bring ATX$350Enclosed, fits 4-slot cards, charges laptop 140W
Sonnet Breakaway 850 T5TB5Built-in 850W$600Plug-and-play enclosed box; pricey
ADT-Link R43SG / F9GM.2 slot → PCIeBring ATX$50–107Hack: turns a free NVMe slot into a GPU port

Windows

Error 43: update the Thunderbolt driver and BIOS. Code 12: enable "Above 4G Decoding."

Linux

Authorize Thunderbolt devices with boltctl. OCuLink shows up as a normal PCIe card, with fewer problems.

Cables

Use the shortest OCuLink cable you can (0.5m, 1m max). TB5 passive cables reach ~2m.

Power supply

Docks without a built-in PSU take a standard ATX (or SFX) desktop PSU. The dock's 24-pin connector switches it on. Without a dock, a $5–10 "ATX jumper" plug does the same job.

GPUCard powerDock-only PSUIn a desktop
RTX 3060170W450–550W550–650W
RTX 5060 Ti180W450–550W650W
RTX 3090350W*750W+850W+
RTX 4090450W850W1000W
RTX 5090575W1000W1200W
2× RTX 3090700W*—1200–1500W

*3090s spike to ~550–580W for about 1ms. Buy ATX 3.0/3.1 units, which are rated for 2× surges.

Goodwill / used PSU?

  • OK only for a cheap card, if it's a known good model (Tier A/B on the SPL tier list), under ~5 years old, with all its original cables.
  • Never mix modular cables between brands or models. The pinouts differ, and mixed cables have killed drives and GPUs.
  • Capacitors age even when the unit sits unused, and the warranty doesn't transfer to you.
  • For a $1,500 GPU, buy new. A good 850W unit costs ~$90–110.

PSU picks

UseModelPrice
3060 / 5060 TiCorsair RM750e (ATX 3.1)$80–90
One 3090 / 4090Corsair RM850e or MSI MAG A850GL$90–110
5090, or a 3090 in a desktopMontech Century II 1050W (10-yr warranty) or Thermaltake GF A3 1050W$100–130
2× 3090Seasonic Vertex GX-1200 / Super Flower Leadex VII 1200W$190–250
3–4 GPUsCorsair HX1500i / Seasonic Prime TX-1600 (or a used HP 1200W server PSU + breakout board, ~$50: loud, 900W on 120V)$300–450
RTX 5090 connector: 12V-2x6 plugs are still melting in 2026, even with the card capped at 500W. Use the PSU's own native cable (no adapters), push it fully in, don't bend it near the plug, and power-limit the card. 3090s use ordinary 8-pin plugs and don't have this problem.
Household circuit: a 15A circuit at 120V gives 1,800W, or ~1,440W continuous. That covers two power-limited 3090s plus the host. Three or four GPUs need a second circuit.

Where to buy

Price and risk move in opposite directions. For anything over ~$500, pay for buyer protection.

SourcePrice vs eBayProtectionBest for
eBaybaselineMoney Back Guarantee, 30 days, even on "no returns" listingsUsed 3090/4090, Tesla & MI50 server pulls. Filter by Sold to see real prices
FB Marketplace / Craigslist / OfferUp10–20% cheapernone, cashLocal deals. Only buy after watching the card run a load test
r/hardwareswap≈ eBay ±10%PayPal Goods & Services onlyEnthusiast sellers. Check confirmed-trade flair and timestamped photos
Jawa.gg≈ eBayEscrow, 2-day inspectionUsed parts from vetted sellers
Micro Centeropen-box / refurb deals90-day refurb warranty3060 12GB, refurbished cards, new PSUs; in person
Amazon / Newegg / Best BuyhighestEasy returns; Renewed 90 daysNew cards, docks, PSUs
AliExpress / Alibabacheapest for MI50 & modded cardsweak48GB 4090 mods, MI50s, for those who accept the risk
GovDeals / university surplussometimes very cheapas-isWorkstation and Tesla cards; luck-dependent

Test it inside the return window

  • GPU-Z: correct device ID, BIOS, full VRAM reported.
  • OCCT VRAM test across all 24GB. Bad memory often shows up only when it's full.
  • FurMark 15+ min: watch for artifacts and clock drops.
  • 3090 memory junction temp (HWiNFO): under 95°C is good, 100°C+ means new thermal pads ($20–40).
  • Load an LLM that nearly fills VRAM and chat for a while.

Scams going around

  • Stripped 4090s: the GPU chip and memory removed and resold. Check the weight and look for disturbed screws.
  • Relabeled chips: a 30-series die marked as a 4090. Confirm with GPU-Z.
  • 4090D sold as 4090 (the slower China-market version).
  • Ex-mining cards are usually fine; fans and thermal pads are what wear out. Judge them by test results and use wear to negotiate.
  • Stock photos and prices far below the Sold median are red flags.

Sources

Benchmark figures marked as estimates are derived from bandwidth or from a related model. Prices change weekly; check eBay's Sold listings before you buy.