I think there is still lots room for improvement to reduce hardware requirements, such as 3-bit quantization or pruning weights from sparse models. But if you'

I think there is still lots room for improvement to reduce hardware requirements, such as 3-bit quantization or pruning weights from sparse models. But if you're willing to spend $1500 on two used RTX 3090, it's the sweet spot in terms of the ability to run large models right now. Everything beyond that is much more expensive.