6 ms·
You can finetune whisper, stable diffusion, and LLM up to about 15B parameters with 24GB VRAM. Which leads you to what hardware to get. Best bang for the $ rig
by ttt3ts 3y ago
You can finetune whisper, stable diffusion, and LLM up to about 15B parameters with 24GB VRAM.
Which leads you to what hardware to get. Best bang for the $ right now is definitely a used 3090 at ~$700. If you want more than 24GB vram just rent the hardware as it will be cheaper.
If you're not willing to drop $700 don't buy anything just rent. I have had decent luck with vast.ai
- aci_12 3y agoCan we use 3090 or nvidia gpus in general with mac? do people generally have their windows dekstop for the gpus?
- ttt3ts 3y agoNo clue but if you want learn/finetuned ML use a Linux box otherwise you will spend all your time fighting your machine. If you just want to run models Mac might work.
- NBJack 3y agoThere is the world of used Nvidia Teslas, like the M40. Very cheap, but some assembly required.
- ttt3ts 3y agoI own a P100, P4, M40. They either lack ram or speed. Also, unless you are putting them in a server you have to cool them. If your goal is to learn ML don't tinker with very obsolete hardware. Rent or buy something modern.
- flangola7 3y agoWhat is their advantages?
- copperx 3y agoWhat do you mean by assembly needed? I looked them up and they look like normal gfx cards. Am I missing something?