> 1) the compute needed to get GPT4 performance will decrease. 2) the compute on consumer GPUs will increase. I’m assuming we’re just talking inference here…

> 1) the compute needed to get GPT4 performance will decrease. 2) the compute on consumer GPUs will increase. I’m assuming we’re just talking inference here… Sure compute abilities for consumers will increase but the original comment had a fixed GPU - the 4090. I can already eke out LLama3:8b on my MacBook Air, and Apple will sell you a laptop capable of running the full sized LLama. There is a direct correlation between parameters and “knowledge” for an LM. There’s some open questions as to density (LLaMa3 specifically challenged previous assumptions) but it seems implausible to fit an equivalent model as GPT4 into 24gb vram. Just like compression, you can’t shrink forever. GPT-4 and GPT-2 are pretty similar architecturally (I assume). So if abilities don’t matter, we can already run GPT-2 so we’re basically there for 4.