4 ms·

You don't put an FPGA in a device you're going to sell 200M+ of. The cost per unit would be way higher than an asic, and your just going to come out with a bett

You don't put an FPGA in a device you're going to sell 200M+ of. The cost per unit would be way higher than an asic, and your just going to come out with a better version next year anyway so why bother? I foresee it as similar to their M series co-processors - the first one was pretty basic, and more sensors and jobs have been given to the newer ones each year.

rch 9y ago
IIRC there's already an FPGA in the iPhone.
- deleted 9y ago
  [deleted]
- Retric 9y ago
  There is a tiny one in the iPhone 7. But, that's for flexibility on current tasks not future proofing. In terms of AI there is little reason to run it on the phone unless it's heavily used or needs or be low latency. Consider if they add a 100$ of computing power to a phone that sits unused 99% of the time they can just build a server using those same 100$ worth of parts that can then serve 100 phones saving 90+$ per phone including upkeep etc. PS: This is the same logic why Amazon Echo is so cheap, it simply does not need much hardware.
  - panic 9y ago
    Privacy is another reason to keep the computation local.
    - themihai 9y ago
      I believe that's actually the main reason to choose a "local" AI.
    - taneq 9y ago
      Good luck getting any of the big players to acknowledge that. >.<
      - djrogers 9y ago
        That's exactly why Apple does things like analyze your photo library locally on the phone - for privacy.
        DannyBee 9y ago
        Yes, surely this has no relation to their inability to build reliable cloud products ....
        mercer 9y ago
        It could be both. Perhaps Apple concluded that 1) they're subpar with cloud services and will have difficulty competing, 2) there's a growing need/demand for more privacy and less 'cloud', and 3) Apple's products are already, on the whole, recommended when it comes to privacy. And based on that they figured privacy was a good thing to aim for. Play to their strengths and differentiate based on that.
      - dgacmu 9y ago
        I believe many of them do. Google has TensorFlow Lite: https://techcrunch.com/2017/05/17/googles-tensorflow-lite-brings-machine-learning-to-android-devices/ https://techcrunch.com/2017/05/17/googles-tensorflow-lite-br... Facebook has Cafe2Go. Apple is working on this (and already has bindings optimized to use the ARM vector unit for DNN evaluation). Running on device, if it can be done with reasonable power, is a win for everyone. Better privacy, better latency, and more robust operation in the face of intermittent connectivity.
- djrogers 9y ago
  You are correct, but we're talking about an entirely different magnitude of fpga for AI purposes than the tiny little one in the iPhone 7.
amirhirsch 9y ago
I work on FPGA embedded vision. I think a lot of people in this thread are making incorrect assumptions about FPGAs implementations of neural network applications. (1) forward networks are constant multiplications, i.e. Fixed shift and add. FPGAs are very nearly optimal architecture for programmable constant-shift-and-add (2) individual neurons in a network can be bitwidth optimized and huffman encoded for bit-serial execution, FPGAs are a very nearly optimal architure for variable bit-width operations in a bit serial architecture with programmable huffman decoders [edited: huffman encoding, not hamming] (3) running a forward network requires multiple channels of parallel memory with separate but deterministic access patterns. Most fpga architurs are designed with onboard ram specifically to be used this way. (4) fpga architectures can be designed inherently fault and defect tolerant, like gpus disabling cores, but with finer granularity. Especially if the compilation is done in the cloud, the particular defect / yield profile can be stored for placement optimization. (5) anything optimized for ASIC design will be necessarily so close to an FPGA that it may as well benefit from the existing programmable logic ecosystem to be flexibly optimized for a particular trained network. You can't just tape out an asic for every trained network, but based my previous points, you most likely can optimize the logic for a specific forward network to run on an FPGA better than any asic designed to run arbitrary networks
- fauigerzigerk 9y ago
  But what about unit cost?

rch 9y ago

IIRC there's already an FPGA in the iPhone.

deleted 9y ago
[deleted]
Retric 9y ago
There is a tiny one in the iPhone 7. But, that's for flexibility on current tasks not future proofing. In terms of AI there is little reason to run it on the phone unless it's heavily used or needs or be low latency. Consider if they add a 100$ of computing power to a phone that sits unused 99% of the time they can just build a server using those same 100$ worth of parts that can then serve 100 phones saving 90+$ per phone including upkeep etc. PS: This is the same logic why Amazon Echo is so cheap, it simply does not need much hardware.
- panic 9y ago
  Privacy is another reason to keep the computation local.
  - themihai 9y ago
    I believe that's actually the main reason to choose a "local" AI.
  - taneq 9y ago
    Good luck getting any of the big players to acknowledge that. >.<
    - djrogers 9y ago
      That's exactly why Apple does things like analyze your photo library locally on the phone - for privacy.
      - DannyBee 9y ago
        Yes, surely this has no relation to their inability to build reliable cloud products ....
        mercer 9y ago
        It could be both. Perhaps Apple concluded that 1) they're subpar with cloud services and will have difficulty competing, 2) there's a growing need/demand for more privacy and less 'cloud', and 3) Apple's products are already, on the whole, recommended when it comes to privacy. And based on that they figured privacy was a good thing to aim for. Play to their strengths and differentiate based on that.
    - dgacmu 9y ago
      I believe many of them do. Google has TensorFlow Lite: https://techcrunch.com/2017/05/17/googles-tensorflow-lite-brings-machine-learning-to-android-devices/ https://techcrunch.com/2017/05/17/googles-tensorflow-lite-br... Facebook has Cafe2Go. Apple is working on this (and already has bindings optimized to use the ARM vector unit for DNN evaluation). Running on device, if it can be done with reasonable power, is a win for everyone. Better privacy, better latency, and more robust operation in the face of intermittent connectivity.
djrogers 9y ago
You are correct, but we're talking about an entirely different magnitude of fpga for AI purposes than the tiny little one in the iPhone 7.

amirhirsch 9y ago

I work on FPGA embedded vision. I think a lot of people in this thread are making incorrect assumptions about FPGAs implementations of neural network applications. (1) forward networks are constant multiplications, i.e. Fixed shift and add. FPGAs are very nearly optimal architecture for programmable constant-shift-and-add (2) individual neurons in a network can be bitwidth optimized and huffman encoded for bit-serial execution, FPGAs are a very nearly optimal architure for variable bit-width operations in a bit serial architecture with programmable huffman decoders [edited: huffman encoding, not hamming] (3) running a forward network requires multiple channels of parallel memory with separate but deterministic access patterns. Most fpga architurs are designed with onboard ram specifically to be used this way. (4) fpga architectures can be designed inherently fault and defect tolerant, like gpus disabling cores, but with finer granularity. Especially if the compilation is done in the cloud, the particular defect / yield profile can be stored for placement optimization. (5) anything optimized for ASIC design will be necessarily so close to an FPGA that it may as well benefit from the existing programmable logic ecosystem to be flexibly optimized for a particular trained network. You can't just tape out an asic for every trained network, but based my previous points, you most likely can optimize the logic for a specific forward network to run on an FPGA better than any asic designed to run arbitrary networks

fauigerzigerk 9y ago
But what about unit cost?