When the Model Doesn't Fit on the GPU
A 155GB model on a 96GB card still runs, if it is a mixture-of-experts model and llama.cpp parks the experts in system RAM. We measured what that costs on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000, an H200 and a B200, why two desktop memory channels are the wall, and the twelve-channel EPYC build that gets around it.