4 ms·
> Too small for ML. What do you mean by this - I assume you mean too small for SoTA LLMs? There are many ML applications where 12GB is more than enough. Even
by sangnoir 2y ago
> Too small for ML.
What do you mean by this - I assume you mean too small for SoTA LLMs? There are many ML applications where 12GB is more than enough.
Even w.r.t. LLMs, not everyone requires the latest & biggest LLM models. Some "small", distilled and/or quantized LLMs are perfectly usable with <24GB
- Havoc 2y agoIf you're aiming for usable, then sure that works. The gains in model ability from doubling size is quite noticable at that scale though. Still...tangibly cheaper than even a 2nd hand 3090 so there is perhaps a market for it
- tacticus 2y agocomparing the bottom of the market cards with a card that is 4+ times the power budget and 6 times the price is um....