4 ms·
I’m a math grad student and want to play with LLMs as well, so I have been thinking about getting a 3060 or 3090, due to budget. However, I’m using a MacBook Pr
by hedgehog0 3y ago
I’m a math grad student and want to play with LLMs as well, so I have been thinking about getting a 3060 or 3090, due to budget. However, I’m using a MacBook Pro from 2011, thus not very convenient for these kinds of things.
What do you think the lowest budget or spending that I should have, given my “requirement”? Or would you say that vast.ai or similar websites would suffice?
- neilv 3y agoYou might try to find forums where people are doing the kind of LLM work you want to do, and see what they're recommending for how to do that work. Personally, for my changing needs, my GPU server is already on its 4th GPU, after I kept finding I needed more VRAM for various purposes. The RTX 3090 seems to be a sweet spot right now, and my next upgrade right now would probably be a 2nd 3090 with NVLink, or some affordable older datacenter/workstation GPU with even more VRAM. Other than the GPU, you just need an ordinary PC with an unusually big PSU, maybe a big HDD/SSD (for data), and maybe a lot of RAM (for some model loading that can't just stream it). https://www.neilvandyke.org/machine-learning/ https://www.neilvandyke.org/machine-learning/