6 ms·
As someone who mucks around with running LLMs and other large models on my computer the 32GB maximum RAM is a show-stopper for me. I'm on a M2 with 64GB at the
by simonw 2y ago
As someone who mucks around with running LLMs and other large models on my computer the 32GB maximum RAM is a show-stopper for me. I'm on a M2 with 64GB at the moment and I'm already regretting not going for 96 or even 128.
I want to be able to run a large model AND other apps at the same time.
- flemhans 2y agoWhen I first scanned your sentence I assumed you were gonna write that 32 GB is now the minimum, not the maximum.
- lynguist 2y agoBut that means the next Air will be upgrade worthy for those who want an Air and a memory uplift from 24 max to 32 max!
- klodolph 2y agoSure, but to be honest, I don’t think you should be shipping a high-spec computer with a built-in screen. Anything high-spec should be broken into at least a separate screen and computer. The screen is such a major point of failure. I’ve seen so many iMacs and MacBooks with broken screens where the end solution is to replace the entire device, which is a waste. It’s that much more of a waste if you are getting a high-spec version.
- simonw 2y agoYeah, that's my biggest complaint with iMacs: I want to be 100% certain I can repurpose them as monitors later in their lives. I have a ten year old iMac at the moment that would make an amazing second monitor... but it doesn't quite have the features I need to use it like that. An HDMI input would be great.
- lisper 2y ago> An HDMI input would be great. There is a reason that is difficult to do: the licensing terms for HDMI require manufacturers to put DRM in place to make it impossible (or at least not straightforward) to record from an HDMI source. It's nearly impossible to meet that requirement on a computer.
- tcoff91 2y agoDisplayPort doesn't have that same restriction, right?
- lisper 2y agoI don't know, but probably not. It's a legal requirement for HDMI, not a technical one. It's actually not hard to build an HDMI recorder, you just can't legally sell one. But bootleg recorders are easy to find.
- buccal 2y agoHP does HDMI-in in somo of its AIOs: https://www.hp.com/us-en/shop/pdp/hp-eliteone-840-g9-all-in-one-pc-wolf-pro-security-edition-p-a27jlua-aba-1 https://www.hp.com/us-en/shop/pdp/hp-eliteone-840-g9-all-in-...
- lisper 2y agoIt's not impossible, but it has to be a completely separate signal chain that bypasses the PC. (Well, OK, it's theoretically possible to do it other ways, but it would be really hard.)
- fwip 2y agoCouldn't you "just" wire the HDMI port to the panel control logic? That is, the HDMI-in doesn't connect to the computer part of the iMac, just to the display circuitry. Edit: nevermind, I see you addressed this in another comment.
- sleepybrett 2y agoThey could design it in such a way that the hdmi input port bypasses the whole 'computer' and goes straight into the display board. If it's plugged in the 'computer' could either not boot or just run headless, i'd prefer the former (just to save on energy costs).
- hultner 2y agoYou probably know this but it’s possible to buy a controller card for the panel on Ali Express and retro-fit into the case and use as a monitor if you are ready to retire the computer itself. I’m contemplating doing this myself at some point but my maxed out 2019 iMac upgraded to 128GB ram and extra SSD is still plenty fast for me, actually feels subjectively quicker than my M2 Pro MacBook Pro with significantly less ram feel. I was a bit surprised as I had read all the hype of the responsiveness of the Apple M-machines.
- eastbound 2y agoIf they could be used as monitors, then my iMac 2018 would still be worth 1700€ (the price of the Apple monitor)! Instead, the 2018 iMac is incredibly slow, and can be thrown away.
- seanmcdirmid 2y ago> The screen is such a major point of failure. The screen of an iMac (at least the 5K 27" one) is the part that still has value, so its annoying when you can't use it on new hardware.
- klodolph 2y agoYes, that is annoying. There’s also such a massive price difference between 4K and 5K monitors. I’ve decided just to accept 4K everywhere because it’s so much cheaper. But I would consider getting a 5K iMac.
- throw0101d 2y ago> Yes, that is annoying. There’s also such a massive price difference between 4K and 5K monitors. Perhaps check out the Asus PA27JCV: 27" at 5120x2880.
- vesrah 2y agoThe unfortunate part is that these aren't for sale yet and we don't know how they actually compare to the existing LG 5k or Apple Studio display. It is nice to see more options coming to market.
- throw0101d 2y ago> The unfortunate part is that these aren't for sale yet […] They were announced 'all the way' back in June, and I've heard rumours of October 31 being the ship date, so perhaps check back next week. In the US, B&H is taking pre-orders, so presumably it can't be that far off: * https://www.bhphotovideo.com/c/product/1850479-REG/asus_pa27jcv_27_proart_5k_usb_c.html https://www.bhphotovideo.com/c/product/1850479-REG/asus_pa27...
- msisk6 2y agoI have a MacBook Air on my desk plugged into a Apple 27-inch 5k Studio Display next to my work MacBook Pro plugged into a cheap AOC 27-inch 4k display via HDMI and frankly there's not much difference. The speakers and mic in the Apple display are nice, but if you're just concerned about the display itself save yourself the bucks and stay with the 4k.
- cududa 2y agoGreat. But plenty of consumers disagree with you, which is why this product exists. Not everything you see is designed to be useful for you specifically
- falcor84 2y ago> plenty of consumers disagree with you What do you mean? This was just announced, no? Are there any sales/pre-order figures already? Also, how would you even go about analyzing the counterfactual of whether the number of people who would buy this spec are "plenty" compared to the number who would have bought a spec with more RAM had it been available?
- cududa 2y agoWell by virtue of the fact that they've sold these un-repairable/ un-openable macs for 3 years now, and they're refreshing it instead of killing the product - all these things indicate consumers like the current approach
- GeekyBear 2y agoExactly. Someone who is looking to muck about with LLMs isn't looking to pay for a 4k screen that cannot be separated from a non-upgradeable PC. If you're in the Apple ecosystem, you're going to want eithet a Mac Mini Pro or a Mac Studio, depending on where the RAM configurations on the Mac Mini Pro tops out. .
- declan_roberts 2y agoWhich models are you running? I'm on a M2pro w/ 32gb and I can run meta llama 8B on lmstudio pretty decently while coding.
- simonw 2y agoYeah 8B is fine but I really want to run 70B (or even 405B but that's way outside my system at the moment). I can run 70B at the moment... but not if I also want Firefox and VS Code at the same time.
- gigatexal 2y agoM3 MAX here with 128GB ... and even that's not going to be enough one day. I bet the M4 Max goes to 256. Oh the envy I will have.
- a_wild_dandan 2y agoI'm so jazzed to comfortably run Llama 3 405b on my Mac with less quantization.
- gigatexal 2y agoIt only goes to 128 :/
- vineyardlabs 2y agoRAM aside, it would be silly for you to ever buy this model for your use case anyway. This is the base M4 that's also in ipads with no active cooling. If your running large LLMs locally, you aren't the target market for this product.
- wtallis 2y ago> I'm on a M2 with 64GB at the moment and I'm already regretting not going for 96 or even 128. You're using a M2 Max, not a M2. Two (big) steps up the chip product stack from what's used in the iMac.
- dagmx 2y agoYou have a higher tier M2. This is the base M4. The comparison is apples to oranges.
- notarealllama 2y agoIt's literally Apple to Apple
- sroussey 2y agoRyzen 3000G vs Instinct 300A — both literally comparing AMD to AMD. Sheesh
- turnsout 2y agoI'll be curious to see if the Mini has an M4 Pro option, and if so, what the RAM ceiling will be!
- sroussey 2y agoI guess we find out tomorrow.
- traceroute66 2y ago> As someone who mucks around with running LLMs and other large models on my computer the 32GB maximum RAM is a show-stopper for me. To be fair, I don't think people mucking around with large LLMs is the primary target market for the iMacs. The sort of people who muck around with LLMs almost certainly already have a monitor, keyboard and mouse. And so are more likely to pick up a Mac Studio which will no doubt be coming soon with M4 Ultra.
- whiplash451 2y agoOr separate concerns and shell out on a Studio display with the-mac-you-like on usb-c
- talldayo 2y agoOr buy a real computer, plug in an Nvidia GPU, and save your DRAM for compute while using your VRAM for inference.
- nickthegreek 2y agoSo now you just limited to 24gb unless you are running dual 3090s or leave the consumer market for a gpu.
- talldayo 2y agoI mean, model layering has been around for several years now: https://huggingface.co/blog/lyogavin/airllm https://huggingface.co/blog/lyogavin/airllm
- Philpax 2y agoMoving the weights between the CPU and the GPU significantly limits performance. It's not comparable to having the entire model resident within video/unified memory.
- deleted 2y ago[deleted]
- 015a 2y agoWe should do a "Most HackerNews HackerNews Comment Of The Year" award each year; in homage to the O.G. [1]. I'd like to nominate this comment. [1] https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
- skeaker 2y agoI agree with the premise but this isn't a great nomination, I think saying "this device won't appeal to the niche LLM gadgetry crowd" is valid in this case since yeah, it won't.
- theWreckluse 2y agoWhile it's not a great one, it certainly checks the tone deaf attitude some HN comments have.
- geodel 2y agoAgree. LLMs are really next generation requirements. Previously people would run a dozen docker containers in Kubernetes cluster on Mac or further before a dozen linux VMs etc and predictably but sadly macs never worked well for these requirements.
- Apocryphon 2y agoIt's also not like the comment was claiming the iMac isn't going to sell well. (And to be fair, it probably won't sell spectacularly well compared to than prior iMacs anyway, given how the MacBooks, Mac Minis, the Mac Studio, etc. have eaten that desktop's lunch.)
- djoldman 2y agoHrm. My OG vote goes to: https://news.ycombinator.com/item?id=35079 https://news.ycombinator.com/item?id=35079 > pg: That has to be the comeback of all time.
- ahofmann 2y agoThat was hilarious, thank you for posting this :-)
- Der_Einzige 2y agoI really wish that "elite" LLM folks like yourself or many others in the field would just abandon apple/mac. Yes, they make better laptops from a hardware perspective. No, M series chips are not actually competitive with Nvidia hardware on anything except cheap (and relatively slow) inference of big models. The fact that windows laptops are considered DOA despite the insane amount of inertia behind Nvidia GPUs is just sad. I want a world where Dell/Lenovo can actually convince folks like you or other "elites" to use their shit. The XPS should be a better laptop than the macbook pro. Yet, I have to watch as the technical elites fawn over a company which continues to sell a starting level un-upgradable laptop with 8gb of ram and a gimped "SSD" in 2024 (this was criminal to do back in 2018) all because other companies can't make a good keyboard or touchpad. I have a similar old-man yelling at cloud tier rant about the slow death of X86...
- cherioo 2y agoThey would run from Mac when nvidia offers a 128gb or 512gb laptop offering. Until then it’s meaningless to even consider windows (laptop)
- internetter 2y ago> The XPS should be a better laptop than the macbook pro. Except its.. not. I'm a longtime windows/linux user who recently switched to mac, and my mac somehow manages to get better performance out of the same specs while lasting 2x longer on battery. In my price range, there is no competition on price:performance:battery.
- leptons 2y agoThere are a lot of "power users" that refuse to realize that they simply will never be in Apple's target market. They love Apple so much for cosmetic reasons, for wanting to be in the "in-crowd" reasons, or for whatever the fanboy reason, that they can't see the writing on the wall. Apple is not making hardware for their use cases, and likely never will again.
- pram 2y agoYou pretty casually dismiss the value of being able to run big models, when it's literally impossible on a mobile Geforce in a lot of cases since they max out at 16GB!
- andrewmcwatters 2y agoI own a MacBook Pro (16-inch, 2021) Apple M1 Max 64 GB. I'm just adding to this that at the time of purchase, this was Apple's top-of-the-line MacBook Pro model, and it's just not that great, in my opinion. Apple's new hardware pales in comparison to the relative abilities of what features existed on a 27-inch iMac with Retina 5K Display 10 years ago. All of the power of modern computers come primarily from their GPUs, and Apple's aren't very good, and have been chronically underpowered compared to the competition for years now. I'm considering relegating Apple devices to being good for just design work. Edit: I agree with Der_Einzige's sentiments. It's beyond time for us to move past Apple hardware.
- _fw 2y agoCan you elaborate on this? It’s my understanding that the CPU alone on that machine even blows the iMac Pro out of the water in terms of performance. Can you share where you’re finding inadequate performance? Genuinely curious to know
- wordofx 2y agoUnless Op has a very special use case that requires an Nvidia card or something then he doesn’t know what he’s talking about. The M2 Max fully specced 14” is insane. Being able to take the laptop anywhere and not need to plug it in and still get all the same power I normally get if it’s plugged in without the worry of the battery dying in 30m is incredible.
- seec 2y agoYeah that's pretty much the problem with modern Apple hardware. If you just want a computer to do casual "home" stuff they feel very overpriced for what they will allow you to do, even though the design and build quality are good. And if you start to drop a lot of cash for a very powerful machine, they seem very underwhelming for the amount of cash you will need in comparison to a beastie PC. The MacBook Pros are in a class of their own but I suspect that most who buy the higher end ones do not actually use the power all that much (they would prefer a real workstation otherwise). When I used to work at an Apple Premium Service Provider, all the clients who had the high-end MacBook Pro were the executives and people like that who made little use of the power. Every worker, even in media agencies had a desktop workstation (iMac or Mac Pro). The MacBook Pro craze is mainly a Silicon Valley startup thing. And yes, Apple computers are mostly useful for design/media work, if you have more use cases you are very likely to hit a roadblock, which is sad for the price.
- insane_dreamer 2y agoI missed the "also great for mucking around with LLMs!" in the press release
- sroussey 2y agoI have a M2 with 64GB and find that it’s fine. Larger models that don’t fit are only moderately better, while much slower. I’d want higher memory bandwidth over more memory. The next five years will push for better models that are smaller. Smaller is faster and more useful. I feel like we are in the brute force phase of model development and that it will pass. Running 405B param model in 16bit on my laptop would be neat, but I’d stop after the novelty wore off.
- jollyllama 2y agoHuh, is that a step backwards for them? iMacs from over 6 years ago were upgradeable to 64
- tiltowait 2y agoThe M-series is a little more complicated. The M1 was max 16GB, with the Pro and Max going up to ... 64, I think? The latest models can go higher, but we're still a long ways off from the old Mac Pro's 1.5TB max.
- jollyllama 2y agoI'm talking about a regular iMac Retina circa 2016.
- sleepybrett 2y agowhy would you want an imac of all things for that? I suspect we are going to see m4 studios, which, given their current options, will support your rediculous ram requirements. iMacs are, generally speaking, targeted for home users, normies.
- grecy 2y agoIt sounds like you should buy products in the "pro" line.
- vid 2y agoI think the OP's point is that Apple is forcing people to pay a lot more for something that doesn't cost Apple that much (extra RAM) and is part of artificial product lines. One could claim the product lines are important for ultimate profitability, but Apple makes so much money it's hardly critical if they wanted to be a truly incredible consumer focused company. Apple has gotten far away from the idea of a home computer someone can hack with, where hacking includes local AI these days. In this period we know more memory is important for many applications of local AI, which they claim is a goal to provide for people, so it's hard not to say Apple's approach is optimized for shareholders, not end users.
- grecy 2y agoThen it sounds like OP ( and yourself) should spend your money elsewhere. I mean, I’m livid Ferrari don’t make a cheap commuter for my family and 9 dogs. So I shop elsewhere.
- vid 2y agoOh, I do take my money elsewhere. However, your livid-ness is a lot more misplaced compared to expecting a very massive, "user friendly," general computing company that "thinks different" to give the people what they want.
- slashdave 2y agoBut... new colors!
- zephyrfalcon 2y agoSo they say, but I'm really not sure what has changed... see https://support.apple.com/en-us/111895 https://support.apple.com/en-us/111895 ...?
- steve_adams_86 2y agoSimilar case here. I'm on a 32GB Mac Studio and constantly wishing I had 128. I didn't expect that when I chose 32... I'd been getting by fine with 16 for a decade. I was "future proofing", haha.
- pmarreck 2y agoCame here to post this. Although I'm looking for an M4 Macbook Pro with 128GB RAM for the same reason. This going to be announced? #OneOfUs #OneOfUs
- sroussey 2y agoMy guess is Wednesday. - iMac today, USB accessories - Mac Mini on Tuesday, likely debut M4 Pro - MacBooks on Wednesday, debut M4 Max Now in the “I wish” category (zero percent chance): - Thursday would be MacStudio with HBM memory for Max and Ultra - Friday would be macPro with 1 to 4 Ultra in NUMA configuration. Now I don’t believe those last two. But I do want to see the X-ray of the max chip to see if it has the UltraFusion part that allowed for combining two chips. It was missing from the M3 Max (and maybe all future odd numbered max chips). If it returns, then we know an Ultra is on the way for sure.
- beAbU 2y agoI'm pretty sure the iMac with no upgradeable RAM nor external GPU support is really targeting the hobbyist big-compute crowd anyway. So you don't have to worry, they are not trying to make you buy one.
- znpy 2y ago> As someone who mucks around with running LLMs and other large models on my computer the 32GB maximum RAM is a show-stopper for me. your specific use-case is probably irrelevant to apple. > I want to be able to run a large model AND other apps at the same time. you answered yourself... go for a machine with 96 or 128gb ram. btw for the money you'd be spending (6-7 k$) you might as well rent or buy a dedicated box with 128, 256gb or more ram and all the gpus you need.
- xyst 2y agoThis computer appeals to families that use their computers to browse social media. For grandparents that browse YouTube with reckless abandon. The emphasis on multiple color options is a strong indicator. Even has some “cutesy” appeal for college students to decorate their dorm. The “Apple Intelligence” branding is nothing more than a selling point. Sure it might run some very small LLM workloads , but don’t expect much especially in the locked down hell known as Apple ecosystem. This is not meant for heavy workloads.
- mark_l_watson 2y agoFor many use cases you are correct. That said, I bought a 32G M2 Mac mini in January, and mostly using Ollama, I use local LLMs for many useful local apps and many experiments. I augment running local models with Colab Pro, Grok APIs, OpenAI APIs, etc.
- neodymiumphish 2y agoThis is the chief reason I’m closely watching this week’s releases from Apple. I’ll hopefully be switching from UnRAID to a Mac Mini or Mac Studio and a multi-drive enclosure, but the Mac needs to support enough RAM for my various services (Plex and Immich primarily) as well as enough to test some large LLM models as a replacement to constant Claude/OpenAI API utilization.
- efficax 2y agoIt's an iMac, the target market is not running local LLMs or doing machine learning research
- anthk 2y agoAnd here I am with 1GB and 512MB as ZRAM on an Atom Netbook. With SBCL.
- lanza 2y agoWhen shopping for a new car to take to the race track on the weekends did you stop and point out that the Honda Odyssey's suspension is too soft?
- bonestamp2 2y agoSame here, I wish I went for 128. That said, I don't think this use case applies to 99.9999% of iMac buyers.
- deleted 2y ago[deleted]