5 ms·
That unified memory architecture makes them well suited in many cases - that's the one bit current x86 desktop platforms don't have a good answer for, where the
by giobox 3y ago
That unified memory architecture makes them well suited in many cases - that's the one bit current x86 desktop platforms don't have a good answer for, where the system design generally keeps GPU and system memory as entirely separate technologies.
This is why fanless M2 MacBook Airs are regularly loading models 5k dollar PCs might choke on - you can get 24gb of unified memory, while almost all high end consumer GPUs top out at 16gb.
- nightski 3y agoHigh end consumer GPUs would be your 3090/4090 cards which both have 24GB.
- giobox 3y ago4080 is still a 16gb part, and still the 2nd fastest part Nvidia sell at consumer level (its a 1200 dollar part). The 3090 (for however long it remains on sale) and the 4090 are outliers. The obvious counter point is an M2 MacBook in 14/16 inch size, where wallet permitting you can have up to 96gb of unified memory too... I simply chose the Air in my example earlier because its incredible you can have that level of performance in ostensibly an entry level passively cooled machine.
- deleted 3y ago[deleted]
- nightski 3y agoYou don't have that level of performance though. Not even close really. RAM amount is only one piece of the puzzle.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- ajb117 3y agoWell at least in the U.S. a lot of used 3090s are out for less than $800 on eBay which is a steal for 24GB VRAM. The 4090 is about $1600 and it's still a consumer-level card; gamers are buying it more often than the 4080. For ML workloads, it's actually a pretty good deal considering how expensive Nvidia's higher-end, non-consumer offerings are. Lambda Labs did some benchmarking: https://lambdalabs.com/blog/nvidia-rtx-4090-vs-rtx-3090-deep-learning-benchmark https://lambdalabs.com/blog/nvidia-rtx-4090-vs-rtx-3090-deep... But yeah the M2 MacBooks are incredible for local LLMs for their price. Nvidia doesn't have any consumer-level priced accelerators with that much memory.
- fomine3 3y agoI don't know is 24GB Air qualify "entry level"
- duskwuff 3y agoA 4090 is $1500-2000 depending on the vendor. A M2 MacBook Air with 24GB RAM is in the same price range, and it's a whole-ass laptop.
- ownagefool 3y agoDon't AMD APUs have unified memory?