4 ms·

I'm a hardware engineer and I downvoted you because your comment is shallow and uninformative. The "Von Neumann bottleneck" has nothing to do with digital vs an

by kevinnk 10y ago

I'm a hardware engineer and I downvoted you because your comment is shallow and uninformative. The "Von Neumann bottleneck" has nothing to do with digital vs analog hardware (rather it's about the separation between storage and computation) and the idea that analog computation is the way forward requires a much longer argument before it's even surface level plausible.

lend000 10y ago
Unfortunately, analog computation is not considered an important addition to the electrical engineering curriculum, even in computer architecture concentrations. I'll gladly explain what I know if you're genuinely interested; in general, you have to hope that people have some context when you make a comment in HN, but there are very few analog-focused people here. But anyway, that's where DARPA is putting their money: http://www.artificialbrains.com/darpa-synapse-program http://www.artificialbrains.com/darpa-synapse-program I definitely see how my language would convey that whereas I mean to refer to digital computers, not all hardware, when I mentioned the Von Neumann bottleneck. There are different arguments against digital hardware in general, even though FPGA's can be about an order of magnitude more efficient than microprocessors. Modern analog goes much further, although we're still not very good at 'programming' it, which is why I think AGI is a ways out.
- kevinnk 10y ago
  The main problem with analog computation is lack of reliability (analog is not always deterministic and manufacturing defects are more difficult to eliminate) and density (analog components are usually much larger, in part to increase reliability). That's a huge blocker for faster chips. How do you think we will get around those problems? Also, perhaps I'm misreading their paper[1], but that DARPA project is still using digital logic. In fact, they specifically say that they decided against using analog computation due to the increased density of digitial (one of the main problems with analog computing, as I mentioned). Regardless, it's pretty clear that the project doesn't have the goal of replacing general purpose computing, so I'm curious why you consider this evidence that analog computing will replace digital computing? Finally, in what way are FPGAs much closer to analog computing than microprocessors? The advantages of FPGAs (configurability) are orthogonal to analog vs digital. [1] http://www.modha.org/papers/013.CICC2.pdf http://www.modha.org/papers/013.CICC2.pdf
  - lend000 10y ago
    Who said anything about replacing digital? You are clearly more intelligent than my calculator, and yet I could program my calculator to do computations that would be practically impossible for you (unless you happen to be a savant). Strong AI will rely on analog, but through future integration with digital, it can achieve the best of both worlds. The topic is AI, where efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. Those are best described as specialized FPAA's and they certainly do rely on digital for programming, and special (although still solid state circuits) for pseudo-analog properties. However, the most interesting parts are yet to come, and are being developed by partners/founders of the program: http://knowm.org/about/ http://knowm.org/about/ The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. This will obviously need to interface with/through digital circuits to be of any use.
    - kevinnk 10y ago
      The way I read your second post was that analog computing is more efficient in general; it sounds like you don't actually believe that. Fair enough. However, I still take issue with your overall point. > Efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. I think it's at least plausible that we'll see gains from fundamentally different architectures (namely combining memory and computation) but where's the evidence that we need to switch from digital to analog? Where's the evidence that analog is even better for an architecture like that? Again, the DARPA project you linked actually gave evidence of the opposite. > The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. You keep talking as if there's some obvious reason that analog is more efficient/faster/better for the types of architectures you're talking about. I'm not seeing that reason.
      - lend000 10y ago

by kevinnk 10y ago

lend000 10y ago

Unfortunately, analog computation is not considered an important addition to the electrical engineering curriculum, even in computer architecture concentrations. I'll gladly explain what I know if you're genuinely interested; in general, you have to hope that people have some context when you make a comment in HN, but there are very few analog-focused people here. But anyway, that's where DARPA is putting their money: http://www.artificialbrains.com/darpa-synapse-program http://www.artificialbrains.com/darpa-synapse-program I definitely see how my language would convey that whereas I mean to refer to digital computers, not all hardware, when I mentioned the Von Neumann bottleneck. There are different arguments against digital hardware in general, even though FPGA's can be about an order of magnitude more efficient than microprocessors. Modern analog goes much further, although we're still not very good at 'programming' it, which is why I think AGI is a ways out.

kevinnk 10y ago
The main problem with analog computation is lack of reliability (analog is not always deterministic and manufacturing defects are more difficult to eliminate) and density (analog components are usually much larger, in part to increase reliability). That's a huge blocker for faster chips. How do you think we will get around those problems? Also, perhaps I'm misreading their paper[1], but that DARPA project is still using digital logic. In fact, they specifically say that they decided against using analog computation due to the increased density of digitial (one of the main problems with analog computing, as I mentioned). Regardless, it's pretty clear that the project doesn't have the goal of replacing general purpose computing, so I'm curious why you consider this evidence that analog computing will replace digital computing? Finally, in what way are FPGAs much closer to analog computing than microprocessors? The advantages of FPGAs (configurability) are orthogonal to analog vs digital. [1] http://www.modha.org/papers/013.CICC2.pdf http://www.modha.org/papers/013.CICC2.pdf
- lend000 10y ago
  Who said anything about replacing digital? You are clearly more intelligent than my calculator, and yet I could program my calculator to do computations that would be practically impossible for you (unless you happen to be a savant). Strong AI will rely on analog, but through future integration with digital, it can achieve the best of both worlds. The topic is AI, where efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. Those are best described as specialized FPAA's and they certainly do rely on digital for programming, and special (although still solid state circuits) for pseudo-analog properties. However, the most interesting parts are yet to come, and are being developed by partners/founders of the program: http://knowm.org/about/ http://knowm.org/about/ The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. This will obviously need to interface with/through digital circuits to be of any use.
  - kevinnk 10y ago
    The way I read your second post was that analog computing is more efficient in general; it sounds like you don't actually believe that. Fair enough. However, I still take issue with your overall point. > Efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. I think it's at least plausible that we'll see gains from fundamentally different architectures (namely combining memory and computation) but where's the evidence that we need to switch from digital to analog? Where's the evidence that analog is even better for an architecture like that? Again, the DARPA project you linked actually gave evidence of the opposite. > The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. You keep talking as if there's some obvious reason that analog is more efficient/faster/better for the types of architectures you're talking about. I'm not seeing that reason.
    - lend000 10y ago