4 ms·

How computationally complex is a transistor? I mean, all the ways that even a micro transistor interacts with voltages and heat make it potentially more comple

How computationally complex is a transistor? I mean, all the ways that even a micro transistor interacts with voltages and heat make it potentially more complex than an AND/OR gate. But that's often what a given transistor is used for. I'm not saying all the complexity of a neuron doesn't matter or that neuron isn't much more complex than a physical chip piece - just that treating parts in isolation, if you don't understand the system, can very easily result in wrong conclusions.

parsecs 5y ago
Another part of the consideration is that transistors are abstracted away into functions like logic gates partially because it makes it easier for the designers. Neurons on the other hand can likely take advantage of all sorts of complex interactions and behaviors as the "maximum local complexity" is not an issue.
- biofox 5y ago
  They certainly do take advantage of complex interactions. The perfect example of that is ephaptic coupling, where neurons interact with each other through the local electrical fields they induce -- and stuff like this is lost in modelling a single neuron: https://en.wikipedia.org/wiki/Ephaptic_coupling https://en.wikipedia.org/wiki/Ephaptic_coupling
moron4hire 5y ago
I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts to create circuits that achieved some task while minimizing transistor count. I think it might have been NASA trying to develop smaller, more sensitive radios, but I'm not sure. IIRC, they found that they could evolve novel circuits that could perform the task with far fewer transistors than traditional methods. But, they couldn't figure out how to transcribe it to an IC outside of the FPGA, as the circuits would stop working. There were also disconnected sections of the FPGA that the GA had evolved that didn't appear to contribute to the output, but couldn't be removed without again breaking the functionality. Obviously, there were some kind of crosstalk issues going on that the GA could minimax to a solution, but it was extremely brittle to change. EDIT: I was completely wrong about the source and application. Can't believe I was able to find it, because I was way off. This was it: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.9691 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.... Other reading indicates that this technique has been successfully applied in electronics design in the time since 1996. https://en.m.wikipedia.org/wiki/Evolvable_hardware https://en.m.wikipedia.org/wiki/Evolvable_hardware The NASA thing I was thinking of was this: https://en.m.wikipedia.org/wiki/Evolved_antenna https://en.m.wikipedia.org/wiki/Evolved_antenna
- bigiain 5y ago
  > I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts You have seen today's xkcd, right? (2510 - for you people from the future...)
- drmemory 5y ago
  Thank you! I’ve been looking for this forever. I think I first read about it in discover magazine as a teenager. I remember the phrase “overexploiting the physics” was used to describe the unusual behavior they saw.
- ajdude 5y ago
  Thank you so much for finding this, I have been trying to find it again for over a decade! I was just telling someone about the study a couple days ago.
- robotresearcher 5y ago
  Adrian went on to evolve circuits with the IC in a freezer, on the bench, and on a hot plate to try to avoid temperature-dependent effects. The GA didn’t know it was supposed to be temperature invariant, or that it was supposed to use the gates as digital devices. Often, the evolved FPGA program would not work on another chip of the same device, because it overfit to the exact chip(s) it evolved on. I was in this group under Phil Husbands at Sussex as a youngster and it was the most stimulating intellectual environment. We had GAs optimize recurrent NNs for robot control, and then had to figure out how they worked, on SPARC stations in 1994. But this ‘nouveau AI’ suffered the AI winter along with the GOFAI we rejected, partially because of the interesting difficulties this team discovered. It was cool seeing these ideas revived 25 years later, with a million times more compute.
deleted 5y ago
[deleted]

parsecs 5y ago

Another part of the consideration is that transistors are abstracted away into functions like logic gates partially because it makes it easier for the designers. Neurons on the other hand can likely take advantage of all sorts of complex interactions and behaviors as the "maximum local complexity" is not an issue.

biofox 5y ago
They certainly do take advantage of complex interactions. The perfect example of that is ephaptic coupling, where neurons interact with each other through the local electrical fields they induce -- and stuff like this is lost in modelling a single neuron: https://en.wikipedia.org/wiki/Ephaptic_coupling https://en.wikipedia.org/wiki/Ephaptic_coupling

moron4hire 5y ago

I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts to create circuits that achieved some task while minimizing transistor count. I think it might have been NASA trying to develop smaller, more sensitive radios, but I'm not sure. IIRC, they found that they could evolve novel circuits that could perform the task with far fewer transistors than traditional methods. But, they couldn't figure out how to transcribe it to an IC outside of the FPGA, as the circuits would stop working. There were also disconnected sections of the FPGA that the GA had evolved that didn't appear to contribute to the output, but couldn't be removed without again breaking the functionality. Obviously, there were some kind of crosstalk issues going on that the GA could minimax to a solution, but it was extremely brittle to change. EDIT: I was completely wrong about the source and application. Can't believe I was able to find it, because I was way off. This was it: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.9691 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.... Other reading indicates that this technique has been successfully applied in electronics design in the time since 1996. https://en.m.wikipedia.org/wiki/Evolvable_hardware https://en.m.wikipedia.org/wiki/Evolvable_hardware The NASA thing I was thinking of was this: https://en.m.wikipedia.org/wiki/Evolved_antenna https://en.m.wikipedia.org/wiki/Evolved_antenna

bigiain 5y ago
> I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts You have seen today's xkcd, right? (2510 - for you people from the future...)
drmemory 5y ago
Thank you! I’ve been looking for this forever. I think I first read about it in discover magazine as a teenager. I remember the phrase “overexploiting the physics” was used to describe the unusual behavior they saw.
ajdude 5y ago
Thank you so much for finding this, I have been trying to find it again for over a decade! I was just telling someone about the study a couple days ago.
robotresearcher 5y ago
Adrian went on to evolve circuits with the IC in a freezer, on the bench, and on a hot plate to try to avoid temperature-dependent effects. The GA didn’t know it was supposed to be temperature invariant, or that it was supposed to use the gates as digital devices. Often, the evolved FPGA program would not work on another chip of the same device, because it overfit to the exact chip(s) it evolved on. I was in this group under Phil Husbands at Sussex as a youngster and it was the most stimulating intellectual environment. We had GAs optimize recurrent NNs for robot control, and then had to figure out how they worked, on SPARC stations in 1994. But this ‘nouveau AI’ suffered the AI winter along with the GOFAI we rejected, partially because of the interesting difficulties this team discovered. It was cool seeing these ideas revived 25 years later, with a million times more compute.

deleted 5y ago

[deleted]