The answer varies wildly. It's a trade-off between what constitutes overall VRAM on your system. For example I have a 3070 Ti and my total VRAM is ~8GB. This is

The answer varies wildly. It's a trade-off between what constitutes overall VRAM on your system. For example I have a 3070 Ti and my total VRAM is ~8GB. This is quite insufficient for working with 70b, for example. So, it depends on the size of the models that you _want_ (or _need_) to work with, the total VRAM on your current system, and other factors. A 4090, for example, would be massive overkill if you want to play around with just 7b models.