On this page 8
Every number below is ours: the same model files, the same prompts and the same llama.cpp build on every card, through our own benchmark harness, and the prices come from our own tracker. We’ve written two long pieces on this hardware already, one on the Mac Studio M5 Ultra and one on what happens when a model doesn’t fit the card. This one is the buying guide: which card, for which models, for how much.
Price per token: every card on one chart
The x axis is what the card costs today. The y axis is how fast it writes Qwen3.8-27B, a dense 27B model at 4-bit, which is the biggest common model that fits every card here, so it is the fairest single speed to compare. Datacenter cards and the mining card have no retail price, so their dots sit at reseller estimates and are greyed.
| Item | Price today, US | Writing speed, Qwen3.8-27B at 4-bit (tok/s) |
|---|---|---|
| GeForce RTX 5090 (32GB · 575W) | ¥4,800 (last known price) | 78 |
| Radeon AI PRO R9700 (32GB · 300W) | ¥2,000 (last known price) | 31.6 |
| RTX PRO 6000 Max-Q (96GB · 300W) | ¥3,887,025 | 72.5 |
| CMP 170HX (64GB) | ¥1,500 | 36.4 |
| NVIDIA H100 PCIe (80GB) | ¥30,000 | 55.1 |
| NVIDIA H200 NVL (141GB) | ¥35,000 | 75.5 |
| NVIDIA B200 (180GB) | ¥40,000 | 82.9 |
Our llama.cpp runs, 4K-token prompt. The 5090 sits at what it sold for before stock ran out; the datacenter cards at reseller quotes.
Read it left to right. The Radeon is the cheapest card that writes a 27B model at a usable speed. The 5090 is 2.5 times as fast for roughly twice the money, at the price it sold for until NVIDIA cut off the supply in late October; today it comes inside a pre-built PC or not at all. Then nothing happens for a long way to the right. The PRO 6000 is slower than the 5090 on this model, because it is the same silicon held to 300W, and the three datacenter cards are no faster than a card you can order from B&H. What all of them sell is memory, and this chart can’t show memory. The next one can.
Above 32GB you are not buying speed. You are buying the right to load the model at all.
What fits in a graphics card
Pick a card size. The bars are each model’s weights at 4-bit; the context cache, the runtime and your desktop come on top, which is why “tight” starts well before the wall. Weights-only download sizes, measured on our cards where we have them.
- Fits, with room for context
- Tight: short context, close other apps
- Doesn't fit at this quant
Weights only. Context (KV cache) comes on top, roughly 0.3 GB per 1,000 tokens for a typical big model. "~" marks sizes calculated from parameter count rather than a published download. GGUF download sizes. Flash-Next's 29GB lookup table lives in system RAM, so only 80GB of it needs the card.
Four things the map decides for you:
- 16GB is a 14B-dense or 30B-MoE card. gpt-oss-20b, the Gemma and Qwen 14Bs, LFM2 and Devstral fit with room for context. Qwen3.8-27B does not: its 16.5GB file is bigger than the card. That is exactly why our pair of 5070 Tis had to split it across both cards.
- 24GB opens the 27 to 31B dense class at 4-bit, with a 64K context beside it. That is the Gemma 4 31B and Qwen3.8-27B tier, the models people actually use for coding agents.
- 32GB adds the 8-bit 27B and all the 30 to 35B mixture-of-experts models with long contexts. It does not reach the 70B class at 4-bit, and nothing between 32GB and 96GB exists at retail today except the 48GB workstation cards, which cost 5090 money.
- 96GB holds the 120B class whole, gpt-oss-120b and Qwen3.5-122B with context, and Flash-Next thanks to its table trick. The next step up is DeepSeek V4 Flash at 155GB, and no single card you can buy holds it.
Anything bigger than your card is not the end of the road if it is a mixture-of-experts model; llama.cpp will park the experts in system RAM and you’ll get 12 to 35 tok/s on a desktop. We measured every version of that in its own article.
The cards we measured
GeForce RTX 5090, 32GB
Gigabyte GeForce RTX 5090 WINDFORCE 32GB GDDR7 Black
The quickest thing on this page for any model that fits in 32GB. Qwen3.8-27B at 78 tok/s and a 64K-token prompt read in 22 seconds; gpt-oss-20b at 329 tok/s; Qwen3.6-35B at 239. Writes 2 to 8% faster than the RTX PRO 6000 Max-Q on every model both hold and reads about 20% faster, because it runs at 575W to the Max-Q’s 300. In a PC with 96GB of RAM it also runs gpt-oss-120b spilled, at 35 tok/s. Sold for about $4,800 for most of 2026 and has been out of stock since NVIDIA cut off the chip in late October; the cards left are inside pre-built PCs.
Full specs, price history & owner sentiment →The 5090 is the card you buy when the models you run fit, you want them fast, and the price has stopped mattering. For a 27B coding model it is the difference between 22 seconds and over a minute before the first word on a big prompt, and that difference is the whole experience of an agent. It is also a 570W space heater at full tilt, and it draws nearly 50W doing nothing. Mind the power supply and the room.
Then there is finding one. In late October 2026 NVIDIA stopped supplying the 5090’s chip to board partners, and within a week there was no card in stock at any store we track. The people who wanted one bought the next card instead, and every price near it moved:
| Card | Before | November 8, 2026 |
|---|---|---|
| RTX PRO 6000 Max-Q, 96GB | $10,517 | $19,999 |
| RTX PRO 6000 Workstation, 96GB | $11,040 | $17,690 |
| GeForce RTX 5070 Ti, 16GB | $1,170 | $1,751 |
| Radeon AI PRO R9700, 32GB | $1,480 | $2,000 |
If you want a 5090 today, it comes inside a pre-built PC, and the right comparison is that whole machine against a Radeon R9700 in a PC you build, or a PRO 6000 Max-Q if your models are over 32GB.
Radeon AI PRO R9700, 32GB
ASUS Radeon AI PRO R9700 TURBO 32GB GDDR6 workstation Black
The same 32GB as the 5090 at a fraction of the price, on Vulkan. Qwen3.8-27B at 32 tok/s, gpt-oss-20b at 168, Qwen3.6-35B at 116. Reads prompts at a third of the 5090’s pace, so a 64K-token prompt is a 70-second wait. Draws 300W working and 33W idle. In a PC with 128GB of DDR4 it ran gpt-oss-120b spilled at 20 tok/s. The card to buy if you want 32GB and would rather spend the difference on RAM.
Full specs, price history & owner sentiment →Here is what the Radeon gives up. It writes at 40% of the 5090’s speed on dense models and at half on the small mixture-of-experts models, and it reads at roughly a third. Spread over a day of chat that is nothing you’d notice; spread over an agent that re-reads 30,000 tokens every turn it is a minute of waiting per turn against twenty seconds. What it doesn’t give up is the models themselves: everything that fits a 5090 fits here, at the same quality. It serves a crowd about as well, too: eight users got 69 tok/s out of it on the 27B, against 213 from the 5090, but 2.5 times what one user gets, which is a better scaling ratio than the Mac Studio’s.
Everything that fits a 5090 fits a Radeon R9700. You just wait longer, and pay a lot less to do it.
RTX PRO 6000 Blackwell, 96GB
PNY NVIDIA RTX PRO 6000 Blackwell 96GB GDDR7 workstation
The only retail card that holds the 120B class whole. gpt-oss-120b at 212 tok/s with a 64K prompt read in 12.5 seconds, Qwen3.5-122B at 119, Flash-Next at 101 (on the 600W edition). Two editions, same silicon and same 96GB: this Max-Q at 300W, and the Workstation Edition at 600W, which reads faster and draws twice the power. On models that fit a 5090, the Max-Q is a 5090 held to 300W, about 5% slower at writing and a fifth slower at reading.
Full specs, price history & owner sentiment →Price History
PNY NVIDIA RTX PRO 6000 Blackwell 96GB GDDR7 workstation
Same card, 600W power limit, bigger cooler. Faster at reading and the one to pick if the PC can feed it. Our Flash-Next, DeepSeek V4 Flash and MiMo runs were on this edition.
Full specs, price history & owner sentiment →This card launched in 2025 near $8,500 and has been listed at anything from a little above that to more than double since. Watch the live price, because the spread between stores on the same week has been wider than the price of a 5090. What you are paying for is 96GB. If your models fit in 32GB, this card is a slower 5090 at several times the price. If they are 60 to 90GB, there is nothing else: a 5090 runs gpt-oss-120b spilled at 35 tok/s, the PRO 6000 runs it whole at 212, and reads it 30 times faster. The Max-Q is the better buy for a desk. It is the same silicon at half the power, gives up 5% of the writing speed and a fifth of the reading, and runs in a normal case on a normal power supply.
Two GeForce RTX 5070 Tis, 16GB each
We measured a pair, split across both cards, behind a six-core Xeon from 2014. Qwen3.8-27B at 22 tok/s, gpt-oss-20b at 99, Qwen3.6-35B at 82, and gpt-oss-120b spilled at 12. Those are two-card numbers on an old PC, not a review of one card; read the section below for what one should do.
Full specs, price history & owner sentiment →A single 5070 Ti is the sensible 16GB card: 896GB/s of memory bandwidth, 300W, and gpt-oss-20b, the 14Bs and the 30B mixture-of-experts models all fit. We didn’t measure one on its own. What we had was two of them in a 2014 workstation, and that rig is worth a paragraph because it shows what not to do. Qwen3.8-27B doesn’t fit a 16GB card, so llama.cpp split it across both, and the pair wrote it at 22 tok/s, a little over half of what one card’s bandwidth would give if the model fit it. gpt-oss-20b, which does fit one card, ran at 99 tok/s where the bandwidth says about 165: the split across two cards over an old PCIe bus cost it 40%. Two mid-range cards are not one big one. If you need 32GB, buy a 32GB card.
The datacenter cards: H100, H200, B200
For one person, none of them is worth having. On Qwen3.8-27B the H100 PCIe wrote at 55 tok/s, slower than a 5090; the H200 at 76 and the B200 at 83, a hair above the PRO 6000’s 73. On gpt-oss-120b the H200 did 183, the B200 199, the PRO 6000 212. They idle at 92 and 173W, and they cost two to three times a PRO 6000 at reseller prices, when a reseller will sell you one at all. The memory bandwidth they are famous for, 4.8 and 8TB/s, goes unused by one user: writing a dense model the B200 moves about 17% of it, because a single token stream can’t keep the card busy. We put that chart in the Mac guide.
What they are for is crowds. With 64 people on the B200 at once it served 949 tok/s of Qwen3.8-27B and 1,646 of gpt-oss-120b. If you have 64 people, rent one by the hour. If you have one person and a big model, the PRO 6000 is the same speed and the only one you can put in a tower.
The mining card: CMP 170HX, 64GB
NVIDIA sold this one to miners in 2021: an A100’s silicon with no display outputs, 1.5TB/s of HBM2e, and 8 or 10GB of memory enabled. In 2026 a tool turned up that unlocks the memory the chips physically have, 64GB on most cards, 80 on some, and the used price went from a couple of hundred dollars past $1,000 within weeks. We ran one with 64GB unlocked. It writes Qwen3.8-27B at 36 tok/s and gpt-oss-20b at 160, within 15% of the Radeon R9700 on everything we ran, at 200W. The 64GB is the point, and we didn’t get to use it: gpt-oss-120b’s 63GB file doesn’t fit beside its context cache, and nothing in our model set sits between 32 and 60GB. It is an Ampere part, so no FP8 or FP4 hardware, no warranty, no monitor output, and a price that doubled on a rumour. Interesting. Not a recommendation.
The wait before the first word
Writing speed is what the spec sheets argue about. The wait is what you feel. Here is how long each card made us look at a blank screen before Qwen3.8-27B started answering, as the prompt grows from a question to a codebase.
| Prompt size | RTX 5090 | RTX PRO 6000 Max-Q | B200 | H200 NVL | H100 PCIe | Two RTX 5070 Tis | Radeon AI PRO R9700 | CMP 170HX |
|---|---|---|---|---|---|---|---|---|
| 1K tokens | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 |
| 4K | 1 | 1 | 2 | 2 | 2 | 4 | 4 | 5 |
| 16K | 4 | 6 | 6 | 7 | 8 | 13 | 15 | 18 |
| 32K | 10 | 12 | 11 | 14 | 17 | 26 | 31 | 38 |
| 64K | 22 | 27 | 24 | 29 | 38 | 62 | 70 | 81 |
Our llama.cpp runs, cold prompt. The 5070 Ti pair splits the model across two cards.
Two groups again. The Blackwell cards and the datacenter cards read a 64K prompt in 22 to 38 seconds. The Radeon, the mining card and the 5070 Ti pair take over a minute. If you run an agent that drags a big context around, the right half of this chart is the number to buy by, and it is where the 5090 earns its price over the Radeon. If you chat, the left half is all you’ll ever see, and every card here answers in under four seconds.
The prompt cache softens all of it. After the first turn, every card here started the follow-up within 0.3 to 0.7 seconds on the same 64K context, so you pay the long read once per session, not once per message.
Cards we didn’t measure
The cards people actually ask about at the low end, and what to expect from them. These are napkin figures from memory bandwidth, scaled from the card of the same architecture we did measure, on the one small model everything holds. Treat them as the right order of magnitude, not a benchmark.
The napkin says what the map already did: below 24GB, buy by memory first. A 5080 and a 5070 Ti have the same 16GB as a 5060 Ti and write two to four times faster, but none of the three loads a 27B dense model, so for local AI the question on all of them is whether the 30B mixture-of-experts tier is enough for you. It usually is: Qwen3.6-35B-A3B and Qwen3-Coder-30B write code well and fit a 16GB card with room for context.
Two of these are worth a live price:
Sapphire Radeon RX 9070 XT PULSE 16GB GDDR6 Black
Same Navi 48 chip and the same 640GB/s as the Radeon AI PRO R9700, with 16GB instead of 32. Everything the R9700 does on models under 12GB, it should do at this speed, for roughly half the money. Unmeasured by us.
Full specs, price history & owner sentiment →
The budget way into the 16GB tier. Half the 5070 Ti’s bandwidth, so expect about half its speed: the napkin says roughly 80 tok/s on gpt-oss-20b, which is still far faster than you read. 180W. Unmeasured by us.
Full specs, price history & owner sentiment →Power, and what a million tokens costs
We sampled every card’s power once a second while it answered one user. This is the card alone, as its driver reports it; the rest of the PC adds 60 to 120W on top.
| One user, Qwen3.8-27B | Idle | Answering | Writes | tok/s per watt | Power per million tokens |
|---|---|---|---|---|---|
| GeForce RTX 5090 | 48W | 569W | 78 | 0.14 | 2.0 kWh · $0.37 |
| RTX PRO 6000 Max-Q | 36W | 297W | 73 | 0.24 | 1.1 kWh · $0.21 |
| Radeon AI PRO R9700 | 33W | 296W | 32 | 0.11 | 2.6 kWh · $0.47 |
| Two RTX 5070 Tis | 65W | 149W | 22 | 0.15 | 1.9 kWh · $0.34 |
| CMP 170HX | 43W | 200W | 36 | 0.18 | 1.5 kWh · $0.28 |
| NVIDIA H100 PCIe | 83W | 290W | 55 | 0.19 | 1.5 kWh · $0.26 |
| NVIDIA H200 NVL | 92W | 395W | 76 | 0.19 | 1.5 kWh · $0.26 |
| NVIDIA B200 | 173W | 554W | 83 | 0.15 | 1.9 kWh · $0.33 |
Power per million tokens at the card’s own draw and writing speed, priced at the US residential average of about 18 cents per kilowatt-hour.
Electricity, then, is 20 to 50 cents per million tokens on every card here. The card is the cost. Run it eight hours a day for three years and a card that writes at 78 tok/s produces about 2.5 billion tokens; every $1,000 of its price adds $0.41 to the cost of a million. At 32 tok/s the same $1,000 adds a dollar. On the small mixture-of-experts models, where the 5090 writes at 329 tok/s, it adds ten cents. Multiply by the live prices above and compare with what a cloud API charges for the same open model, which for the 120B-class models is well under a dollar per million output tokens. The arithmetic says the cloud wins unless the card is busy most of the day. People buy the card anyway, for the same reasons they buy the Mac Studio: privacy, no meter, nobody retiring the model under you.
Which card to buy
| You | Buy |
|---|---|
| Trying local models, small budget | A 16GB card. The RTX 5060 Ti 16GB is the cheapest way in, the RX 9070 XT the fastest for the money. Run the 30B mixture-of-experts models and the 14B dense ones; skip the 27B class. |
| One person, 27 to 35B models, money matters | The Radeon AI PRO R9700. Everything a 5090 holds, at 40% of the speed and a fraction of the price. Put the savings into 128GB of RAM and gpt-oss-120b runs too, spilled, at 20 tok/s. |
| One person, 27 to 35B models, want it fast | The RTX 5090, which since late October 2026 means a pre-built PC. A 1,000W power supply, and 96GB of RAM beside it to keep gpt-oss-120b usable at 35 tok/s. |
| One person, 60 to 90GB models | The RTX PRO 6000 Max-Q. The only retail card that holds the 120B class, in a normal case at 300W. The 600W edition if you’ll run agents all day and have the PC for it. |
| Models over 96GB | Not a graphics card. A 256GB Mac Studio holds DeepSeek V4 Flash whole at 39 tok/s; a PRO 6000 with the experts in desktop RAM gets 22. The Mac guide and the spill guide have the rest. |
| A team of five or more | The twelve-channel EPYC build with one PRO 6000 we work out in the spill guide, or two DGX Sparks if everything fits in 128GB. |
| A product with real traffic | Rent an H200 or a B200 by the hour. For one person they are a PRO 6000 at three times the price. |
How we tested
- Engine: llama.cpp release b11146, the prebuilt binaries for each platform (CUDA for NVIDIA, Vulkan for the Radeon). Whole model on the card unless the text says it spilled. The 5070 Ti pair used llama.cpp’s default layer split across both cards.
- Files: the same GGUF on every card. Qwen3.8-27B is Unsloth’s UD-Q4_K_M (16.5GB), gpt-oss-20b and gpt-oss-120b the MXFP4 builds (12 and 63GB), Qwen3.6-35B-A3B Q4_K_M (20GB), Qwen3.5-122B-A10B Unsloth’s Q4_K_M (77GB), Qwen3.8-Flash-Next Unsloth’s UD-Q4_K_XL (111GB).
- Prompts: a synthetic log-analysis task at 1K, 4K, 16K, 32K and 64K tokens, 256 tokens generated, fresh cache each time; three runs per size, medians reported. A follow-up in the same conversation measures the prompt cache. Crowds are 1, 2, 4 and 8 simultaneous users with a 1K prompt each; the 64-user B200 figures are from llama.cpp’s batched benchmark.
- The PCs: the RTX 5090 behind a Core i9-14900KF with 96GB of DDR5; the Radeon AI PRO R9700 behind a Ryzen 9 5900X with 128GB of DDR4; the RTX PRO 6000 Max-Q behind a Core Ultra 7 with 188GB of DDR5, the 600W edition behind a Core Ultra 9 285K with 256GB of DDR5; the two RTX 5070 Tis behind a Xeon E5-2620 v3 from 2014 with 64GB of DDR4; the CMP 170HX behind an EPYC 7402 with 256GB. The H100 is the 80GB PCIe card, the H200 the 141GB NVL card, the B200 the 180GB SXM module, all in servers.
- Power: what the GPU driver reports, sampled once a second. The card, not the wall.
- Prices: live from our tracker at render time, lowest in-stock US listing, new condition. The H100, H200, B200 and CMP 170HX dots on the value map are not tracked; they sit at the middle of published reseller and used-market estimates, which are linked below and marked as such.
Sources
- Our own benchmarks (October-November 2026): llama.cpp on a GeForce RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000 Blackwell in both editions, two GeForce RTX 5070 Tis, a CMP 170HX, an H100 PCIe, an H200 NVL and a B200. Method in “How we tested” above.
- Prices: our tracker, lowest in-stock US listing; the before-and-after table is its daily lows.
- RTX 5090 supply: Notebookcheck and VideoCardz, late October 2026.
- Datacenter card price estimates: TRG Datacenters’ H200 price guide, Runpod on H200 pricing, GPUpoet’s H200 NVL card page, and IntuitionLabs’ datacenter GPU price guide for the H100 and B200 ranges. None of these is a list price; NVIDIA doesn’t publish one.
- CMP 170HX: LTT Labs on the memory unlock tool (Sept 12, 2026) and Digital Citizen on the price jump.
- Electricity: EIA Short-Term Energy Outlook, 2026 US residential average around 18 cents per kilowatt-hour.
- Cloud price reference: Context Studios, the per-token cost of a 120B-class open model through a cloud API.
- Model cards: Qwen3.8-Flash-Next (the 55B of tables), OpenAI’s gpt-oss cards, Qwen’s and Google’s cards for the parameter counts on the fit map.