3 ms·
What amount of ram and GPU are people considering adequate for playing around with all the local inference possibilities over the next year?
by swader999 3y ago
What amount of ram and GPU are people considering adequate for playing around with all the local inference possibilities over the next year?
- nullsense 3y agoAs much as your budget allows for.
- nickthegreek 3y agoAgreed. But your choices are now the 4060 to 16gb for $500 or a way more expensive 3090/4090 with 24gb and the power draw to go with it.
- nullsense 3y agoI started out with a 1080ti that I had lying around. Within days upgraded to an RTX3060. Within weeks of that upgraded to an RTX 4090. In retrospect I wish I had of known that "buy once, cry once" might have been the way to go here. That's just me.
- washadjeffmad 3y ago3090s go for under $800 used, and you can set the power limit to whatever you want. I found that 290W keeps the fans from spinning up audibly under heavy CUDA workloads with ~2% lower performance.