3 ms·
So currently our consumer grade CPUs with DDR5 are limited to less than 100GB/s. Meanwhile Apple is shipping computers with multiples of that.
by whatever1 2y ago
So currently our consumer grade CPUs with DDR5 are limited to less than 100GB/s. Meanwhile Apple is shipping computers with multiples of that.
- oDot 2y agoStrix Halo is rumored to be about twice as fast but unfortunately not near Apple's speed.
- mananaysiempre 2y agoOn the other hand, I bought an 8C/16T Zen 4 laptop with 64GB RAM and an 4TB SSD for less than $2000 total including tax. I’ll take that trade.
- sroussey 2y agoHow are 70b LLMs running on that?
- jocaal 2y agoThe free version of chatgpt is better than your 70b LLM, whats the point?
- cma 2y agoQwen coder 32b instruct is the state of the art for local LLM coding and will run with a smallish context with that on a 64GB laptop with partial GPU offload. Probably around .8 tok/sec. With a quantization of it you can run larger contexts and go a bit faster. 1.4 tok/sec at 8b quant with offload to a 6GB laptop GPU. Speculative decoding has been being added to lots of the runtimes recently and can give a 20-30% boost with a 1 billion weight model running the speculative token stream.
- YetAnotherNick 2y agoWhy do you need 64GB RAM?
- theandrewbailey 2y agoSeveral Electron apps and 1000+ Chrome tabs. (just guessing)
- scheme271 2y agoMemory can get eaten up pretty quickly between IDEs, containers, and other dev tools. I have had a combination of a fairly small C++ application, clion, and a container use up more than 32GB when combined with my typical applications.
- evoke4908 2y agoI just built a new PC with 64GB for just this reason. With my workloads, the 32GB in my work laptop is getting cramped. For an extra $150 I can double that and not worry about memory for the next several years
- shakabrah 2y agoI have 128gb in PC (largely because I can) and android studio, a few containers and running emulators will take a sizable bite into that. My 18gb MacBook would be digging into swap and compressing to get there.
- mananaysiempre 2y agoPartly because I can, because unless you go absolutely wild with excess it’s the RAM equivalent of fuck-you money. (Note it’s unified though, so in some situations a desktop with 48GB main RAM and 16GB VRAM can be comparable, and from what I know about today’s desktops that could be a good machine but not a lavish one.) Partly because I need to do exploratory statistics to say ten- or twenty-gigabyte I/O traces, and being able to chuck the whole thing into Pandas and not agonize over cleaning up every temporary is just comfy.
- jchw 2y agoRunning 128 GiB of RAM on the box I am typing on. I could list a lot of things but if you really wanted a quick demonstration, compiling Chromium will eat 128 GiB of RAM happily.