4 ms·
It's absolutely mind boggling that such a powerful processor can run at a power envelope less than 10 watts!
by dd_xplore 3y ago
It's absolutely mind boggling that such a powerful processor can run at a power envelope less than 10 watts!
- crooked-v 3y agoIf the rumors about LLM stuff baked into Siri on the next iPhone are true, it will be interesting see how they make the most use of the hardware with that.
- Mistletoe 3y agoGood Lord this may the brain transplant she has needed for so long.
- lostlogin 3y agoI thought you were piling on a human and felt a bit outraged. Siri deserves more slander than you delivered.
- smcleod 3y agoI'm really hoping for a complete replacement. Talk about a product that went downhill over time.
- thejazzman 3y agoLiterally changes throughout the day from sort of working to insulting me by doing the opposite ("turn on the kitchen lights" -> all lights in the house turn off)
- brandall10 3y agoWhat's interesting is the A17 Neural Engine is about twice as powerful as it in on the M3 series - 35 TOPS vs. 18. It's rumored the difference is to aid in photography and possibly Face ID, but of course this will be helpful for any onboard inference.
- bernaferrari 3y agoeven if that's true, the A series neural engine is much worse than the M series, I'll say it wrong, but it can only do 32bit inference (or something like that) where M series can do 64bit, so A series can run LLM but has a series of limitations that M series doesn't.
- icyfox 3y agoPractically speaking, most models today infer at 8bit or 16bit (sometimes, rarely 32). You don't see an empirical lift at more bits of precision. Size of the memory is far more important.
- OJFord 3y agoIf we're talking about the results, is there any reason to think it should make a difference at all?
- icyfox 3y agoSometimes gradients are small but meaningful, if you constrain them to too few bits / degrees of freedom they'll be unable to backprop successfully. This can hamper training and therefore results quality. You can also think about it as compounding errors - at any one weight index the bit values might not be too meaningful, but cascaded over a lot of tensor multiplications they will be.
- OJFord 3y agoOh I was thinking we were talking about the same calculations on different hardware.
- yieldcrv 3y agoI think next year's refreshes across the board will have a different architecture so the neural engine has a lot more access to RAM so that models with larger parameters can be used and faster
- trvz 3y agoI don't believe that's true, regardless of what you mean by "can run at a power envelope". The M1/M2/M3 use 20-25 Watt at peak, but only 10% of that when idle. (But that's only in the notebooks – it goes beyond that in the Mac mini.) The M3 Pro is basically 50% more of everything than the M3, so I'd reuse that multiplier to guess its power levels.
- lend000 3y agoParent (and the article) are talking about power per core.
- hospitalJail 3y agoToo slow to be useful for anything interesting. Too expensive to be useful for any IOT or general computing. I'm sure its really useful that one time you couldn't take a flight to a location and didn't have a car so you took a bus and that bus was made in 1990 and didn't have 120V outlets retrofitted and you decided to work for 12 hours straight on things that never used GPU.
- dd_xplore 3y agoHaha true that. It's really limited in terms of how well it can be integrated, however I really wish Apple would make some socket version of this(in future )for general consumers or atleast OEMs , it would be hugely successful. The chances are close to impossible since it's Apple, and the architecture will need lot's of changes or the OEMs will have to change their motherboard design drastically to support a socketed M series chip ( storage controller, memory controller are all on the SOC)
- seec 3y agoPredictively you got massively downvoted but I don't understand how that is not the sentiment. People are really irrational around Apple stuff. Funniest thing is that I like macOS enough that I would buy their crap if it wasn't so absurdly expansive because of RAM pricing. I'm willing to let go of gaming and GPU apps (wasn't many in macOS anyway, because there were historically behind) but come on, not at this price... The battery life argument is a mystery to me. I wish they would figure out this would much better in a product like the Watch, instead of wasting power for useless computation, graphics or what have you...
- arcticbull 3y agoWithout taking away from how cool that is, the Threadripper 7980X manages to run at just 3.4W/core according to GamersNexus [1] It's pretty amazing where everyone (other than Intel) is on the perf/W scale. [1] https://www.youtube.com/watch?v=yDEUOoWTzGw https://www.youtube.com/watch?v=yDEUOoWTzGw
- pram 3y agoI don’t know but I’m curious: I’d imagine at that many cores you’re probably reaching the limits of how much power the socket can realistically get? I know some motherboards have that supplemental power port.
- Jonnax 3y agoYeah motherboards for threadripler have extra power supply connections. I believe there's a the option to connect two https://videocardz.com/newz/amd-threadripper-pro-7995wx-96-core-zen4-cpu-rips-201k-cinebench-r23-oc-record-with-peak-1-6kw-power https://videocardz.com/newz/amd-threadripper-pro-7995wx-96-c... World record on cinebench with liquid nitrogen pulled 1.6kW
- crote 3y agoAs I understand it, the main issue is cooling the damn thing. Socket SP5 can do up to 700W, and that's all emitted from a heatspreader 72mm x 75mm. Getting the power into it isn't trivial, but it's not rocket science either. When the socket has 6000 pins, you can spare a few hundred for power.