4 ms·
If the rumors about LLM stuff baked into Siri on the next iPhone are true, it will be interesting see how they make the most use of the hardware with that.
by crooked-v 3y ago
If the rumors about LLM stuff baked into Siri on the next iPhone are true, it will be interesting see how they make the most use of the hardware with that.
- Mistletoe 3y agoGood Lord this may the brain transplant she has needed for so long.
- lostlogin 3y agoI thought you were piling on a human and felt a bit outraged. Siri deserves more slander than you delivered.
- smcleod 3y agoI'm really hoping for a complete replacement. Talk about a product that went downhill over time.
- thejazzman 3y agoLiterally changes throughout the day from sort of working to insulting me by doing the opposite ("turn on the kitchen lights" -> all lights in the house turn off)
- brandall10 3y agoWhat's interesting is the A17 Neural Engine is about twice as powerful as it in on the M3 series - 35 TOPS vs. 18. It's rumored the difference is to aid in photography and possibly Face ID, but of course this will be helpful for any onboard inference.
- bernaferrari 3y agoeven if that's true, the A series neural engine is much worse than the M series, I'll say it wrong, but it can only do 32bit inference (or something like that) where M series can do 64bit, so A series can run LLM but has a series of limitations that M series doesn't.
- icyfox 3y agoPractically speaking, most models today infer at 8bit or 16bit (sometimes, rarely 32). You don't see an empirical lift at more bits of precision. Size of the memory is far more important.
- OJFord 3y agoIf we're talking about the results, is there any reason to think it should make a difference at all?
- icyfox 3y agoSometimes gradients are small but meaningful, if you constrain them to too few bits / degrees of freedom they'll be unable to backprop successfully. This can hamper training and therefore results quality. You can also think about it as compounding errors - at any one weight index the bit values might not be too meaningful, but cascaded over a lot of tensor multiplications they will be.
- OJFord 3y agoOh I was thinking we were talking about the same calculations on different hardware.
- yieldcrv 3y agoI think next year's refreshes across the board will have a different architecture so the neural engine has a lot more access to RAM so that models with larger parameters can be used and faster