4 ms·

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microproce

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

yarcob 5y ago
But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)
- adrianN 5y ago
  Yes, but you wouldn't know that if the best knowledge you had about the inner workings of microchips is essentially looking which parts get hot while they perform different tasks.
- titzer 5y ago
  Wouldn't it suck if brains are just reservoirs used by the internet that can store roughly 3 memes at a time?
  - taneq 5y ago
    holds up a Duracell battery "You think thermodynamics means using humans as batteries is a dumb idea? Where do you think you learned thermodynamics?" faint nyan cat music in the background
    - cwillu 5y ago
      "May I have a physics textbook?" "No. The real world doesn't run on math."
zetalyrae 5y ago
True, but transistors are designed by humans (for now), so you have an upper bound on complexity as well as a certain degree of modularity. Whereas biology has no such limitations because evolution doesn't care about the understandability of its designs. So I tend to err on the side of biology being more complex than not. But it's absolutely possible that the high-level behaviour is simple while the underlying implementation is complex and chaotic, as in the gas laws.
- roenxi 5y ago
  Biology hasn't had time to figure out that intelligence is a good idea, up to humans the payoffs have been relatively low. The complexity in human intelligence probably focuses more on doing basic things extremely energy efficiently rather than being effective at thinking. Humans can attempt to multiply numbers and frequently get the wrong result. That doesn't scream "pushing the limits of intelligence".
- ben_w 5y ago
  Transistors were invented by humans, but chip layouts themselves involve a lot of automation and at more than one level — Verilog and VHDL both appeared in the 1980s, and there’s tools at both higher and lower levels of abstraction than those two.
- zarzavat 5y ago
  Complexity cuts both ways. Complexity, as every programmer knows, is no friend, and can easily create more problems than it solves. Often the most efficient solution to a problem ends up being a simple one. It would be very surprising if the human brain's Rube Goldberg machine was anywhere near close to a mathematically optimal implementation of intelligence. It's possible that much of the complexity in a biological neuron is simply working around other complexity introduced by biology, solving problems that we programmers do not even have to think about because we can simply directly use matrix multiplication.
- jerf 5y ago
  Probably more to the point is that humans are using transistors in their designs, so we deliberately confine them to their simple modes. It is an oversimplification to talk about transistors being either "on" or "off" because they technically have all sorts of intermediate states, but with a few exceptions, human designs avoid those intermediate states like the plague, because A: they defy our ability to build logic with them and B: depending on those exact behaviors means we can't mass-manufacture chips because the variance of the exact behaviors will be too high. In principle one could imagine a processor design that works on these intermediate states that somehow vastly exceeds the computational power of a modern system despite using the same base transistors; in practice we have no idea how to build such a thing, and if we did, we wouldn't know how to build a second one of the same thing reliably either. Biology lacks this restriction. That doesn't mean it's pure and utter chaos, either, there's bounds on that because it still needs systems to at least be metastable. But where humans engineer almost exclusively with stable systems, biology freely uses metastable systems all over the place. And then, even more remarkably, it deals with the question of how to replicate such a strange system in a way that no modern human engineer ever would by making every instance unique, and still somehow functional. It's a tough act to follow.
throwaway316943 5y ago
I think even dramatically simpler might be an understatement. Neurons are cells, living organisms capable of growth, movement, and some fairly intelligent interaction with their environment. A transistor doesn’t even approach that, it’s more on the level of a single protein within a neuron and not a very complex one.
tMcGrath 5y ago
The paper is measuring I/O behaviour, rather than the complexity of the mechanisms generating that behaviour. Transistors might have quite complex physics, but are designed to have relatively simple I/O behaviour.
mrfusion 5y ago
Don’t forget that transistors are millions of times faster too.
- qayxc 5y ago
  Could indeed be a trade-off between speed vs complexity and efficiency. Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires.
  - short_sells_poo 5y ago
    > Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires Given that entire datacenters don't come close to an AGI, let alone a human level one, I fear "several kilowatts of power" is lowballing it by a significant number of orders of magnitude. More to the point, we don't even know how or what gives rise to a general intelligence, and even defining it is basically a philosophical question. To me, the optimism of some AI enthusiasts (and I don't mean specifically the parent) feels like cavemen contemplating an expedition to the stars shortly after they invented the sling. I think the chances of us arriving to anything close to an AGI iteratively based on our current capabilities is a pipe dream.
  - ben_w 5y ago
    The lowest lowball estimate I’ve seen for the computational capacity of the human brain is 20e15 FLOPS (Kurzweil, 1999), and that would require 2/3 of a megawatt with the current best ranked supercomputer on the Green500 list. There are already several computers more powerful than this that don’t appear close to a working full brain emulation, and we definitely don’t understand intelligence well enough yet to engineer something like ourselves, so it’s reasonable to be skeptical of estimates saying we’ll be at the kilowatts level “soon” even if it turns out we’re just missing a step which will be obvious in hindsight. (Unless by “soon” you mean 15 years; I don’t want to bet on anything on that timescale).
    - 5y ago
taneq 5y ago
Also true - witness the weird circuits generated by evolutionary algorithms on FPGAs etc. where they use chip-specific nonlinear dynamics and capacitative / inductive coupling which only works on the specific chip used to evolve the circuit.

yarcob 5y ago

But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)

adrianN 5y ago
Yes, but you wouldn't know that if the best knowledge you had about the inner workings of microchips is essentially looking which parts get hot while they perform different tasks.
titzer 5y ago
Wouldn't it suck if brains are just reservoirs used by the internet that can store roughly 3 memes at a time?
- taneq 5y ago
  holds up a Duracell battery "You think thermodynamics means using humans as batteries is a dumb idea? Where do you think you learned thermodynamics?" faint nyan cat music in the background
  - cwillu 5y ago
    "May I have a physics textbook?" "No. The real world doesn't run on math."

zetalyrae 5y ago

True, but transistors are designed by humans (for now), so you have an upper bound on complexity as well as a certain degree of modularity. Whereas biology has no such limitations because evolution doesn't care about the understandability of its designs. So I tend to err on the side of biology being more complex than not. But it's absolutely possible that the high-level behaviour is simple while the underlying implementation is complex and chaotic, as in the gas laws.

roenxi 5y ago
Biology hasn't had time to figure out that intelligence is a good idea, up to humans the payoffs have been relatively low. The complexity in human intelligence probably focuses more on doing basic things extremely energy efficiently rather than being effective at thinking. Humans can attempt to multiply numbers and frequently get the wrong result. That doesn't scream "pushing the limits of intelligence".
ben_w 5y ago
Transistors were invented by humans, but chip layouts themselves involve a lot of automation and at more than one level — Verilog and VHDL both appeared in the 1980s, and there’s tools at both higher and lower levels of abstraction than those two.
zarzavat 5y ago
Complexity cuts both ways. Complexity, as every programmer knows, is no friend, and can easily create more problems than it solves. Often the most efficient solution to a problem ends up being a simple one. It would be very surprising if the human brain's Rube Goldberg machine was anywhere near close to a mathematically optimal implementation of intelligence. It's possible that much of the complexity in a biological neuron is simply working around other complexity introduced by biology, solving problems that we programmers do not even have to think about because we can simply directly use matrix multiplication.
jerf 5y ago
Probably more to the point is that humans are using transistors in their designs, so we deliberately confine them to their simple modes. It is an oversimplification to talk about transistors being either "on" or "off" because they technically have all sorts of intermediate states, but with a few exceptions, human designs avoid those intermediate states like the plague, because A: they defy our ability to build logic with them and B: depending on those exact behaviors means we can't mass-manufacture chips because the variance of the exact behaviors will be too high. In principle one could imagine a processor design that works on these intermediate states that somehow vastly exceeds the computational power of a modern system despite using the same base transistors; in practice we have no idea how to build such a thing, and if we did, we wouldn't know how to build a second one of the same thing reliably either. Biology lacks this restriction. That doesn't mean it's pure and utter chaos, either, there's bounds on that because it still needs systems to at least be metastable. But where humans engineer almost exclusively with stable systems, biology freely uses metastable systems all over the place. And then, even more remarkably, it deals with the question of how to replicate such a strange system in a way that no modern human engineer ever would by making every instance unique, and still somehow functional. It's a tough act to follow.

throwaway316943 5y ago

I think even dramatically simpler might be an understatement. Neurons are cells, living organisms capable of growth, movement, and some fairly intelligent interaction with their environment. A transistor doesn’t even approach that, it’s more on the level of a single protein within a neuron and not a very complex one.

tMcGrath 5y ago

The paper is measuring I/O behaviour, rather than the complexity of the mechanisms generating that behaviour. Transistors might have quite complex physics, but are designed to have relatively simple I/O behaviour.

mrfusion 5y ago

Don’t forget that transistors are millions of times faster too.

qayxc 5y ago
Could indeed be a trade-off between speed vs complexity and efficiency. Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires.
- short_sells_poo 5y ago
  > Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires Given that entire datacenters don't come close to an AGI, let alone a human level one, I fear "several kilowatts of power" is lowballing it by a significant number of orders of magnitude. More to the point, we don't even know how or what gives rise to a general intelligence, and even defining it is basically a philosophical question. To me, the optimism of some AI enthusiasts (and I don't mean specifically the parent) feels like cavemen contemplating an expedition to the stars shortly after they invented the sling. I think the chances of us arriving to anything close to an AGI iteratively based on our current capabilities is a pipe dream.
- ben_w 5y ago
  The lowest lowball estimate I’ve seen for the computational capacity of the human brain is 20e15 FLOPS (Kurzweil, 1999), and that would require 2/3 of a megawatt with the current best ranked supercomputer on the Green500 list. There are already several computers more powerful than this that don’t appear close to a working full brain emulation, and we definitely don’t understand intelligence well enough yet to engineer something like ourselves, so it’s reasonable to be skeptical of estimates saying we’ll be at the kilowatts level “soon” even if it turns out we’re just missing a step which will be obvious in hindsight. (Unless by “soon” you mean 15 years; I don’t want to bet on anything on that timescale).
  - 5y ago

taneq 5y ago

Also true - witness the weird circuits generated by evolutionary algorithms on FPGAs etc. where they use chip-specific nonlinear dynamics and capacitative / inductive coupling which only works on the specific chip used to evolve the circuit.