I admit I have not pushed them that far, though I am also using 40B and 65B models which may perform better than a 30B model. Nothing open approaches GPT-4 yet

I admit I have not pushed them that far, though I am also using 40B and 65B models which may perform better than a 30B model. Nothing open approaches GPT-4 yet, which I have heard is an overlay of multiple 220B models. The only way you will see that on even high end consumer hardware is advances in compression and pruning, but those are happening. There’s a lot of research in those areas. Of course then cloud models could use the same techniques to get even bigger, if the model and training will keep scaling. Edit: as AI matures you may also see lots of custom ASICs hit the market. I could see those getting cheap enough to run many hundred B models locally. The acceleration of an ASIC designed for a specific type of model over CPU and GPU could be dramatic, reducing the amount of silicon you need.