4 ms·
> someone will come out with a high-ram neural network only card at some point, and that would end up being better than the 4090 anyway because 24 gigs of vram
by squeaky-clean 3y ago
> someone will come out with a high-ram neural network only card at some point, and that would end up being better than the 4090 anyway because 24 gigs of vram is still quite small
These already exist, they're just disproportionately expensive compared to consumer cards. You can get a Nvidia A100 with 80GB VRAM on eBay for about $14,000.
3.33x the memory for 9x the price.
- YetAnotherNick 3y agoOr you could get A6000 for $4k with 48 GB VRAM.
- moonchrome 3y agoIs 48GB enough to run 70B models ?
- sacred_numbers 3y agoYes, with 4 bit quantization.
- 7speter 3y agoYou can also buy used enterprise cards on ebay. There may not be many 32gb+ cards up yet, but they'll be on there eventually. The big issue is how to cool these cards though, both amd and nvidia make the cards with fan-less heat sinks because the servers they get packed into provide cooling via their loud, ultra fast, potentially digit severing fans, but people who are salvaging these cards have to find their own solution to cool them, and fit them in a given case.
- yrro 3y agoHow purchasers get drivers, don't they require a subscription, or are they pirated?
- 7speter 3y agoAs far as AMD drivers are concerned, you can at least reflash some cards. Nvidia cards... well I've only seen videos where people say they have a given driver so there might be underhanded means involved.
- muttled 3y agoI'm surprised nobody's yet making custom coolers for them (complete coolers with heatsink/fans, not gluing a 3d-printed fan shroud to the front) and re-selling them. Seems like a niche gap in the market.