3 ms·

The way I read your second post was that analog computing is more efficient in general; it sounds like you don't actually believe that. Fair enough. However, I

by kevinnk 10y ago

The way I read your second post was that analog computing is more efficient in general; it sounds like you don't actually believe that. Fair enough. However, I still take issue with your overall point. > Efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. I think it's at least plausible that we'll see gains from fundamentally different architectures (namely combining memory and computation) but where's the evidence that we need to switch from digital to analog? Where's the evidence that analog is even better for an architecture like that? Again, the DARPA project you linked actually gave evidence of the opposite. > The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. You keep talking as if there's some obvious reason that analog is more efficient/faster/better for the types of architectures you're talking about. I'm not seeing that reason.

lend000 10y ago
Here's a better attempt to explain it: we are using digital circuits to basically simulate analog circuits with analog properties. It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures). Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations. It's no surprise that some of the most successful models have shifted away from the perceptron-style representation and use more mathematical models to better leverage digital hardware for the node simulations, but they still rely on the basic models of weights and feedback cycles which require clock cycles and controlled computation on digital circuits. Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor). It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).
- kevinnk 10y ago
  > It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures). That's not necessary true; again, see the DARPA project you listed. > Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations. The analog hardware doesn't necessarily behave like the desired components either (for example, due to manufacturing errors). With digital you get to choose your accuracy and with much greater control. > Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor). Unless you don't care about noise at all, your design will be much more complicated than a single transistor. Either way, it's actually much easier to do this sort of "approximate computing" in the digital domain. > It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains). A bird has feathers yet our most efficient airplanes don't use them; there's no reason at all to believe that just because the brain is analog, our design should be as well.

by kevinnk 10y ago

lend000 10y ago

Here's a better attempt to explain it: we are using digital circuits to basically simulate analog circuits with analog properties. It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures). Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations. It's no surprise that some of the most successful models have shifted away from the perceptron-style representation and use more mathematical models to better leverage digital hardware for the node simulations, but they still rely on the basic models of weights and feedback cycles which require clock cycles and controlled computation on digital circuits. Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor). It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).

kevinnk 10y ago
> It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures). That's not necessary true; again, see the DARPA project you listed. > Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations. The analog hardware doesn't necessarily behave like the desired components either (for example, due to manufacturing errors). With digital you get to choose your accuracy and with much greater control. > Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor). Unless you don't care about noise at all, your design will be much more complicated than a single transistor. Either way, it's actually much easier to do this sort of "approximate computing" in the digital domain. > It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains). A bird has feathers yet our most efficient airplanes don't use them; there's no reason at all to believe that just because the brain is analog, our design should be as well.