I purchased a 3060 specifically for the 12gb of memory last November and I've been able to run llama, alpaca, stable diffusion out of the box for everything wit

I purchased a 3060 specifically for the 12gb of memory last November and I've been able to run llama, alpaca, stable diffusion out of the box for everything without ever having any memory issues. Training is usually overnight, and a stable diffusion will render in ~5 seconds, llama will do 20 tokens/second. I would say start with the 3060 for 250 bucks, and if you're still loving it after a couple months, drop 10x more on a quadro. My only word of advice is get docker setup and install the nvidia docker toolkit to passthrough your gpu to docker images -- the package management for all these python ai tools is a hell-scape, especially if you want to try a bunch of different things.