11 ms·
Apple Is Working on a Dedicated Chip to Power AI on Devices
- wyldfire 9y ago> . The chip, known internally as the Apple Neural Engine, Is this a real IC/processor for arbitrary software or an abstraction of an underlying GPU/DSP?
- mtgx 9y agoMost likely some kind of dedicated deep learning accelerator. This is coming with or without Apple: > Exynos 8895 features VPU (Vision Processing Unit) which is designed for machine vision technology. This technology improves the recognition of an item or its movements by analyzing the visual information coming through the camera. Furthermore, it enables advanced features such as corner detection that is frequently used in motion detection, image registration, video tracking and object recognition. http://www.samsung.com/semiconductor/minisite/Exynos/w/solution/mod_ap/8895/ http://www.samsung.com/semiconductor/minisite/Exynos/w/solut... > New Vision Processing Unit (VPU) paired to the Image Signal Processors (ISP) that provides a dedicated processing platform for numerous camera features, freeing up the CPU and GPU and saving power. https://www.mediatek.com/products/smartphones/mediatek-helio-x30 https://www.mediatek.com/products/smartphones/mediatek-helio... I was also hoping that with Google's high-efficiency for the TPU, they would make a version for mobile as well, at least for their Pixel phones. I guess they still might, but the fact that the TPU2 does both training and inference makes me think they won't do that anytime soon anymore. The biggest reasons why I like this "mobile AI chips" trend is that they can give you back some privacy, if the data can be analyzed locally without going to the vendors' servers, and I think they will also boost the capabilities of computational photography. No more spying toys for kids, etc.
- protomyth 9y agoWhat does the instruction set look like on that? Is there another example out of this type of chip?
- MBCook 9y agoGoogle has their Tensor Flow. chips, I would imagine it is similar.
- protomyth 9y agoIs there released documentation on the instruction set of the Google Cloud TPU?
- pjscott 9y agoThey list the main ones on pages 3-4 of this paper: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk... It's mainly moving memory around, matrix multiplication, convolution, and applying activation functions (sigmoid, tanh, relu, etc.). Very simple, high-level stuff. This has the handy side-effect of making timing very predictable, which makes the latency a lot more deterministic.
- binocarlos 9y agoNo idea what instruction set the Apple device uses but Google just announced alpha access to their Tensorflow Processing Unit: https://cloud.google.com/tpu/ https://cloud.google.com/tpu/ on Google cloud
- indy 9y ago> I was also hoping that with Google's high-efficiency for the TPU, they would make a version for mobile as well At Google IO there was a slide during the keynote that they're working with Mediatek to produce a mobile TPU
- samstave 9y agoI wish they would have called it Apple Neural Technology, so we could start referring to the devices as ANTs and Hives and Colonies as we build out Richard Hendrick's new decentralized internet.
- samfisher83 9y agoWhy not use a gpu. A lot of AI stuff is linear algebra: Multiply accumulate etc.
- obstinate 9y agoI am 100% positive that they considered that before starting to work on this ASIC. Some possible reasons: GPUs are insufficiently specialized, or use too much energy on the subset of work that Apple wants to enable with this chip. GPUs are too large. GPUs are busy doing other things, etc.
- nsxwolf 9y agoThey considered it sure - but why did they conclude they should go with an ASIC? That's what grandparent asked and it was a reasonable question. "They considered that" isn't a suitable answer.
- obstinate 9y agoI think when you add "why try to reinvent the wheel?" to the end, it is less of a question and more of a statement. Similar to saying, "Why would you do that?" after someone does something silly. You aren't actually asking them why they'd do the thing. You're saying they ought not have.
- cortesoft 9y agoBut they didn't say that?
- stephencanon 9y agoThey did, but they've since edited the comment.
- obstinate 9y agoThat is exactly what they said.
- strin 9y agoNeural Engine = General Matrix Multiplier?
- highd 9y agoThis is probably going to be a hyper parallel fixed point / integer engine like TPU gen1. Doing fast matrix multiply over really small fields is very subpar on CPUs and GPUs. That was the initial reasoning behind TPU gen1 - improving runtime performance. One question is if it will architecturally be closer to a GPU or an FPGA. The field moves so fast that it might make sense to "future-proof" a bit with a live-reconfigurable FPGA.
- imh 9y agoHow do FPGAs compare on space and power? Would differences be enough to matter for mobile?
- highd 9y agoI don't think space is an issue, but an ASIC designed exactly for a workload will always beat an FPGA on power. But if you don't know the workload exactly or don't have the money to fab an ASIC then an FPGA will be superior if the workload is a bad fit for CPUs or GPUs. So if you can save (2-10)x power on some unknown ML workload in the future that might be preferable to (10-20)x on some fixed workload with a fixed-point ASIC. I.e. Bitcoin mining went GPU->FPGA->ASIC, each with more investment required to design but higher overall performance in Hash/W. But that workload is known exactly.
- blackguardx 9y agoSpace is only an issue because it is directly correlated with unit cost.
- seanp2k2 9y agoI doubt they'll do an FPGA. Devices are too concerned with battery life to be running that, plus their margins would suffer or it'd be even more expensive.
- aswanson 9y agoProbably worse on power consumption as well as price per MIP.
- cft 9y agoI have been worried about this trend: if they don't open it up, things like this introduce a disparity between startups that can only have access to GPUs and big companies that make their own proprietary ASICs for their proprietary software, such that startups cannot complete.
- spott 9y agoThe flip side is that there is pretty obviously a market for such a product. If it isn't released by google or apple, it will be released by someone else. If it isn't, then that is a pretty good idea for a startup.
- cft 9y agoOnly well funded startups will make ASICs.And most of them will fail. This is very different from many small startups programming general purpose computers.
- asimpletune 9y agoSo then maybe the key is a start up that's in the business of raising the chance of success that other players having in this endeavor?
- deleted 9y ago[deleted]
- elefanten 9y agoWon't it likely be in their interest to make those capabilities open to 3rd parties? My guess: the platforms that don't will suffer in experience.
- cft 9y agoWhat about the server side? Google executing Tensorflow on ASICs versus a startup restricted to GPUs or Google Cloud/TPUs?
- 9y ago
- shauder 9y agoKnowing them this will be pretty good. The A10 is a beast.
- johansch 9y agoKnowing Apple's (software) prowess in AI the end-result will still likely be shit compared to Google. (I think what we are seeing here is the usual thing where Apple's software/product/design people decide the iPhone hardware roadmap, rather than the hardware people.)
- yalogin 9y agoWhat is your basis for this? Am curious.
- DigitalJack 9y agoOS X has degraded over the last several years in terms of bugginess. Anecdotally. It has gone from "it just works" to a source of annoyance. Although nothing can top my work computer (Windows 7) for irritation. Siri is a bag of hurt. I bet Apple has a very useful library of recordings of people saying "damn it, that's not what I said." Just thinking about how terrible Siri is makes me hope and pray that they really aren't working on any self-driving car software.
- midhunsezhi 9y agoSiri vs Google assistant. Google assistant being way better in understanding voice and performing tasks. I've used both iPhone and Pixel and I can confirm that Google assistant is way more smart and intelligent than Siri at this point.
- mathieuh 9y agoDon't know why you're being downvoted, this has been my experience too. Siri feels more like a toy with its infuriating jokes and stunted capabilities, Google Assistant feels much more polished. I have confidence that attempting a command I've never tried before with Google Assistant will work, with Siri it's potluck.
- JumpCrisscross 9y agoCould this mean on-device ANI? My deal breaker with Amazon, Google, Microsoft and even Siri is their role in normalising the hoovering up of sensitive data.
- whatisani 9y agowhat is ANI?
- georgespencer 9y agoArtificial Narrow Intelligence or "weak" AI. Like Siri.
- yellow_postit 9y agoMore work needs to be done on training models with less data, differential privacy, and unsupervised learning, but so long as supervised learning continues to be the main path forward for the current set of "AI" centralizing the data into ginormous data sets will continue to be the norm.
- halflings 9y agoI don't see how unsupervised learning makes this any better? That data you're training on in an unsupervised manner is still collected somewhere, and could contain as much private information as a labeled dataset.
- visarga 9y agoYep, whether it's supervised or not has no bearing on privacy. What counts is where the data is processed and who has it.
- yellow_postit 9y agoLabels for supervised training tend to come from humans in the loop. I think many would consider another human looking at their photos, searches, etc. To be a loss of privacy albeit with a small surface area.
- deepnotderp 9y agoWell.... This is interesting indeed, although I suppose it was somewhat inevitable. I'm definitely interested in the architectural details of the chip, but I doubt Apple will open up. Apple has control of the software stack and by extension, what models will run on this chip, so I expect that it will be a little bit more special purpose than general purpose.
- khana 9y agoRemember that video clip where the ocean whale submerges the human marine biologist just long enough for her to get the message. Like scarily submerged to about 75ft. I don't want an AI chip in my phone Apple!
- hackuser 9y agoMy assumption has been that Google, Amazon, and Microsoft run the heavy-duty AI in the cloud when possible, benefiting from huge scale and easier updates. Maybe that assumption is wrong? If it's right, is Apple adopting a more decentralized model, with AI (or more AI) running locally? Could that compete with cloud-based AI's advantages? Obviously it would be better for offline usage, for responsiveness when transmission to the cloud is a significant part the latency, and for confidentiality.
- happycube 9y agoGoogle's been working on distributed training as well.
- hackuser 9y agoWhy? What is the benefit to Google? Also, are they doing training for the local user or for Google's 'general' systems or for both?
- bitmapbrother 9y agoSo they can do on device AI/ML with TensorFlow Lite, through the use of specialized neural network DSP's, as discussed during the keynote at I/O 2017. https://youtu.be/Y2VF8tmLFHw?t=1h22m8s https://youtu.be/Y2VF8tmLFHw?t=1h22m8s
- jchw 9y agoWhy is Bloomberg not mentioning that Google announced it was working on the same thing? They mentioned vaguely that Amazon and Google both were working on AI, but nothing about the seemingly similar TPU and how Google announced they were going to bring it to phones at I/O just a bit ago. Am I wrong to be thinking that's pretty relevant here?
- ex3ndr 9y agoBecause Google's TPU is for servers and not for Mobile.
- pg_is_a_butt 9y agoBecause Google announced they were going to bring it to phones at I/O just a bit ago.
- jchw 9y agoPretty sure they announced during I/O that they were working on making a TPU for mobile.
- hn_throwaway_99 9y agoDo you have a link? I don't remember this, and I didn't find anything through googling.
- TechRomancer 9y agoI'm definitely doubting myself now. I remember watching the keynote and thinking how they didn't make a bigger deal out of the on device chip. The only thing similar I can find is at about 1:22:00 in the keynote here https://youtu.be/Y2VF8tmLFHw https://youtu.be/Y2VF8tmLFHw but all he actually says is "silicon specific accelerators". So honestly at this point it could mean anything.
- xxgreg 9y ago
- philplckthun 9y agoIt's nice that the article is trying to deliver an intro that explains that Apple clearly has some catching up to do. Except that now I'm pretty baffled, since I've seen an article a few months earlier, that says Apple is massively investing in AI, and already using it in several places in their products. So what am I supposed to believe now? :/
- H4CK3RM4N 9y agoAs I understand it, Apple simply hasn't been able to provide the same quantity of training data that other companies use.
- luhn 9y agoBelieve them both. Apple is investing massively into AI and is using it in their products. However, Google has been working with AI longer and has much more experience. (The article praises Amazon's AI chops as well, I dunno about that one.)
- ClammyMantis488 9y agoVery interesting...
- gigatexal 9y agoGood. But Google beat them to it already.
- npgatech 9y agoHow so? Google developed a server class TPU for Datacenters. Apple is trying to build on-device low powered custom chip.
- bitmapbrother 9y agoGoogle announced it at I/O. They're using DSP's, specialized for neural network processing, on the SoC with TensorFlow Lite on the device. https://youtu.be/Y2VF8tmLFHw?t=1h22m8s https://youtu.be/Y2VF8tmLFHw?t=1h22m8s
- npgatech 9y agoOh wow, thanks. I feel like Apple is a step behind in pretty much all things AI.
- gigatexal 9y agoYeah SIRI has been lame compared to Google's hey Google for a long time.
- GeekyBear 9y agoApple added a neural networking API that can leverage the GPU or CPU of a given device in the last version of iOS. https://www.wired.com/2016/06/apple-bringing-ai-revolution-iphone/ https://www.wired.com/2016/06/apple-bringing-ai-revolution-i... There was an article last year discussing customizations Apple had made to Imagination's GPU that made it more suitable for this purpose. http://www.realworldtech.com/apple-custom-gpu/ http://www.realworldtech.com/apple-custom-gpu/
- Geee 9y agoAre these kind of chips used to accelerate the NN training process only?
- ge96 9y agoThat's what we need! A bunch of GPU's and computers carrying a car battery haha. Man that would be crazy. Pre trained before it leaves the factory. (don't know what I'm saying) but I do imagine a man-sized humanoid robot with a bunch of GPU's, hard drives.
- faragon 9y agoApple partially offloading Siri from the cloud to client devices in order to reduce datacenter costs?
- ojosilva 9y agoI think the general availability of TPUs is an important inflection point in the path to AI popularization, abd, who knows, some type of singularity event. Definitely a milestone in 21st century history. But I can't resist making a parallel between evolving TPUs and how the CPU found in the arm of the T800 changed history (negatively) forever in the Terminator universe.
- bonoetmalo 9y agoI'm not too familiar with the concept of ML specific chip designs, but isn't most AI done on servers and the results returned to the device? What kind of applications involve local ML code execution?
- msh 9y agoIf you want to protect user privacy it's a very relevant selling point to be able to say that the data never leaves the device.