6 ms·
Related question: Does it make sense to buy a RTX 4090 or 3090 Ti for this? Or for finetuning existing open models.
by ofrzeta 3y ago
Related question: Does it make sense to buy a RTX 4090 or 3090 Ti for this? Or for finetuning existing open models.
- felsokning 3y agoThe answer varies wildly. It's a trade-off between what constitutes overall VRAM on your system. For example I have a 3070 Ti and my total VRAM is ~8GB. This is quite insufficient for working with 70b, for example. So, it depends on the size of the models that you _want_ (or _need_) to work with, the total VRAM on your current system, and other factors. A 4090, for example, would be massive overkill if you want to play around with just 7b models.