5 ms·
High end consumer GPUs would be your 3090/4090 cards which both have 24GB.
by nightski 3y ago
High end consumer GPUs would be your 3090/4090 cards which both have 24GB.
- giobox 3y ago4080 is still a 16gb part, and still the 2nd fastest part Nvidia sell at consumer level (its a 1200 dollar part). The 3090 (for however long it remains on sale) and the 4090 are outliers. The obvious counter point is an M2 MacBook in 14/16 inch size, where wallet permitting you can have up to 96gb of unified memory too... I simply chose the Air in my example earlier because its incredible you can have that level of performance in ostensibly an entry level passively cooled machine.
- deleted 3y ago[deleted]
- nightski 3y agoYou don't have that level of performance though. Not even close really. RAM amount is only one piece of the puzzle.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- ajb117 3y agoWell at least in the U.S. a lot of used 3090s are out for less than $800 on eBay which is a steal for 24GB VRAM. The 4090 is about $1600 and it's still a consumer-level card; gamers are buying it more often than the 4080. For ML workloads, it's actually a pretty good deal considering how expensive Nvidia's higher-end, non-consumer offerings are. Lambda Labs did some benchmarking: https://lambdalabs.com/blog/nvidia-rtx-4090-vs-rtx-3090-deep-learning-benchmark https://lambdalabs.com/blog/nvidia-rtx-4090-vs-rtx-3090-deep... But yeah the M2 MacBooks are incredible for local LLMs for their price. Nvidia doesn't have any consumer-level priced accelerators with that much memory.
- fomine3 3y agoI don't know is 24GB Air qualify "entry level"
- duskwuff 3y agoA 4090 is $1500-2000 depending on the vendor. A M2 MacBook Air with 24GB RAM is in the same price range, and it's a whole-ass laptop.