3 ms·
Out of curiosity, what is your benchmark here? I have a $2000 RTX card that is great for games, but pretty poor for LLMs. For LLM development, I'd be much happi
by imdsm 3y ago
Out of curiosity, what is your benchmark here? I have a $2000 RTX card that is great for games, but pretty poor for LLMs. For LLM development, I'd be much happier with a Studio and an M2 Ultra. How much would it cost me to get 192 GB in discreet cards I wonder?
- singhrac 3y agoI think the statement "I'd be much happier with a Studio" is a little hypothetical? Sorry if that's not true, but everywhere I've looked, it seems like these are not ML training chips, and people are just hoping they will handle LLMs well. You can absolutely build (with real support from the PyTorch folks) a 4x3090 deep learning workstation that has 96 GB of VRAM for roughly $7k. Or, more likely, you'll rent a A100 from AWS for ~$0.15/hr.