5 ms·
So my phone and my laptop both have the capability to perform 15 trillion operations per second, just in the neural engine? What kind of things are taking adva
by simonw 4y ago
So my phone and my laptop both have the capability to perform 15 trillion operations per second, just in the neural engine?
What kind of things are taking advantage of this right now? It's gotta be more than just Face ID right?
What's my laptop likely to be doing with that?
- gedy 4y agoI can imagine some thing like Siri running on device much more effectively against local content. The cynic in me doesn’t want to hope too much for cloudless services like this, but one can hope.
- waboremo 4y agoScene analysis in photos, image captions, and machine translations are also done using ANE. CoreML also utilizes it when possible.
- fauigerzigerk 4y agoI don't know for sure, but things like text recognition (Live Text) or object recognition in Photos (Visual Look Up) are obvious candidates. I think neural engine is absolutely key to Apple's strategy. They want people to buy expensive devices and they don't want to process user data on their servers. Users get privacy. Apple gets money. It's a pretty coherent business model.
- flutas 4y ago> They want people to buy expensive devices and they don't want to process user data on their servers. > Users get privacy. Apple gets money. Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers.
- blululu 4y agoIt’s not a subsidy. It’s a pricing structure for a commercial transaction. Fundamentally a business can not just give out free compute. In the long run the user of computation needs to pay for it. It’s a question of whether people feel more satisfied paying for it in a lump sum bundled with a device or through a subscription plan on the cloud. For frequently, on demand, low latency applications I would suspect that people will always be happier running the computations locally.
- saagarjha 4y agoApple also runs an OS on that device, so they can't just offload infinite computation for it: it would use too much battery.
- GeekyBear 4y ago> Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers. Subsidizing the smart speaker hardware and running everything on their servers hasn't worked out well for Alexa and Google Assistant. > The Alexa division is part of the "Worldwide Digital" group along with Amazon Prime video, and Business Insider says that division lost $3 billion in just the first quarter of 2022, with "the vast majority" of the losses blamed on Alexa. That is apparently double the losses of any other division, and the report says the hardware team is on pace to lose $10 billion this year. It sounds like Amazon is tired of burning through all that cash. Google expressed basically identical problems with the Google Assistant business model last month. There's an inability to monetize the simple voice commands most consumers actually want to make, and all of Google's attempts to monetize assistants with display ads and company partnerships haven't worked. With the product sucking up server time and being a big money loser, Google responded just like Amazon by cutting resources to the division. https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-colossal-failure-on-pace-to-lose-10-billion-this-year/ https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...
- blululu 4y agoTo my knowledge this is mostly used by internal tools, though a number of common 3rd party apis (qr code scan) use hw acceleration under the hood. Internally there is a ton of ML running on the device. The most obvious is touch screen and inputs and the camera. 3rd party developers have acres to this via CoreML, but unless latency is critical it is usually easier to develop and run ml on the cloud. For camera apps using ml, this chip is going to be used either explicitly or implicitly.
- simonw 4y agoOh the touch screen! That's fascinating, is that definitely running stuff on the neural engine?
- blululu 4y agoIf you think about it a capacitive touch sensor provides a noisy Grayscale image and the goal is to detect and classify blobs as touch gestures as quickly and accurately as possible. Since it is running at all times and latency really burns the UX. Consequences this has always been done on a HW accelerator.
- nl 4y agoI'm pretty sure this isn't correct. Gesture recognition has been around since before the neural engine shipped and doesn't appear to be different on devices with or without it.
- GeekyBear 4y agoFor example: >Machine learning is used to help the iPad’s software distinguish between a user accidentally pressing their palm against the screen while drawing with the Apple Pencil, and an intentional press meant to provide an input. https://yugalchoubisa.medium.com/how-machine-learning-and-artificial-intelligence-usedapple-a7e61153557e https://yugalchoubisa.medium.com/how-machine-learning-and-ar...
- 4y ago
- iamgopal 4y ago15 trillion operation per second ? Of what kind ? Addition ? Isn’t that mind blowing ?
- selectodude 4y agoMatrix multiplication
- sebzim4500 4y agoI know you probably didn't mean this, but in case anyone is confused ANE is not doing 15 trillion matrix multiplications per second. It is doing 15 trillion scalar operations in order to multiply a much smaller number of matrices.
- amelius 4y agoOf what size??
- conradev 4y agoIt’s used for a variety of things: - Biometrics (Face ID and Touch ID) - Image analysis (face matching, aesthetic evaluations, etc) - Text to speech and speech to text (smaller models on device, used for privacy/latency/reliability) - Small ad-hoc models like Raise to Speak on Apple Watch, the Hey Siri detector (https://machinelearning.apple.com/research/hey-siri https://machinelearning.apple.com/research/hey-siri) These things have been in phones for 5 years now and have been used from day one
- simonw 4y agoRight, but do any of those things really need 15 trillion operations for second? Have they been getting noticeably better with upgraded phone models?
- jamiek88 4y agoYes definitely. I could only find a blurry YouTube video of the instruction manual for an old old heater in my house. I paused the video on the bit I needed the guy had zoomed into and was able to copy and paste the text that I could barely read into a notes doc. There’s no one splashy thing just lots of little quality of life improvements.
- blululu 4y agoNo but the first party users should not consume all the compute on the chip. The bigger the margin the better for the device. The other aspect of this is speed and power consumption (battery life is a top 3 phone feature across pretty much all consumers).
- burnished 4y agoI got one recently and generally think the phone is garbage, but the OCR built into pictures is really something else. I took a photo of a label for a barcode when I couldnt see it myself but could get my hand nearby, it was at an odd angle, but when I pressed my finger to the text I was interested in the phone captured it immediately, highlighted it, and I copied it nice as you please.
- 4y ago
- k_bx 4y agoThey're putting it everywhere they can. From Notes to pressing pause on Video in QuickTime or Safari and copying text from a frame instantly.
- deleted 4y ago[deleted]
- ManuelKiessling 4y agoRunning Microsoft Teams. Barely.
- IIAOPSW 4y agoSiri wasn't a product. She was an emergent feature they couldn't extinguish.
- secretsatan 4y agoArkit makes use of it on the phone, there’s plane detection and classification, image and object detection, segmentation for people occlusion, probably more behind the scenes. I find it a little frustrating we aren’t using the built in capabilities of iphones more in our company, i still kinda think apple tech is kind of a pariah in some circles, so we have to run with stuff that runs on cloud that costs us money over, heaven forbid something you could run on an iphone
- kmeisthax 4y agoThere's an app called Draw Things, for iOS/iPadOS/macOS/etcOS, that uses the ANE to run Stable Diffusion on your phone/tablet/laptop.
- nl 4y agoYou can run large language models locally using it: https://machinelearning.apple.com/research/neural-engine-transformers https://machinelearning.apple.com/research/neural-engine-tra... I know the Swift version of Stable Diffusion from Hugging Face uses this too.
- Our_Benefactors 4y agoWhile it’s true you can run some parts of a machine learning stack, Apple still lags way behind nvidia due to not having CUDA support. It makes a huge difference in both speed and quality of output.
- zamnos 4y agohttps://www.rewind.ai/ https://www.rewind.ai/ is one that I know of. it only works on apple Silicon but their software records the shit out of everything you do on your computer, so you never have that moment when you half remember something you did the past. just pull up rewind and find it!