39 ms·

I haven’t seen a perceptron circuit done in less than 4 transistors plus a few diodes. However these chips are almost certainly digital and will use a lot more

I haven’t seen a perceptron circuit done in less than 4 transistors plus a few diodes. However these chips are almost certainly digital and will use a lot more to perform the floating point math. Single transistors can’t even compute our simplified model of a neuron let alone the complexities of the real thing.

mlyle 7y ago
If we're going to talk about net computational power per unit (neuron/transistor), we also need to note that single transistors do stuff a whole lot faster than biological neurons, too. If we had neurons that were 1,000x faster, could we use fewer of them for the same task?
- jacquesm 7y ago
  Well, we can turn that argument around: if an artificial brain could be faster than a real one, does that mean that right now we have slow artificial brains that are just as good as the real ones? The answer being an obvious 'no' may help answer your question.
  - mlyle 7y ago
    Just because we don't know a program that creates a decent artificial brain doesn't mean that such a program doesn't exist for the hardware we have. We're almost certainly going to be able to make speed/size trade-offs in AGI, because we're used to making them in all other computing. Even biological systems make these types of trade-offs. We're beings optimized for the environment we evolved in, not optimized for "smarts". There's many "choices" in energy balance, reaction time, etc, that have been made. Transistors are really powerful and fast computing devices; really our use of them in digital logic in some ways greatly under-utilizes them for ease in design and predictability. Saying that it takes 4 transistors to make a really simple neuron analog in a circuit undersells that those 4 transistors can operate many, many orders of magnitude faster.
    - jcims 7y ago
      Faster than what? There are a hundred trillion atoms in a neuron operating in a coordinated fashion to create its global behavior in the context of the brain. How many interactions between those components occur in a billionth of a second? Many.
  - tlb 7y ago
    It would be very hard to tell if an AGI running 1000x slower than a brain were intelligent. Humans babies don’t talk for about a year, but nobody has tried firing up a putative slow AI and waiting for 1000 years.
    - jacquesm 7y ago
      Then let's run 1000 of them in parallel, right? You're on to the exact point I'm trying to make. The whole idea that 'just' complexity or 'just' speed is enough to make something intelligent that is not yet intelligent is off-base to a degree that resorting to metaphors to try to explain why also breaks down. I liken intelligence to life: once it exists it is obvious and self-perpetrating but before it exists it is non-obvious and no amount of fantasizing about what it would take to make it will get you incrementally closer until one day you've got it. But that day will start out just like the thousands of days before it. It's a quantum leap, a difference in quality, not an incremental one. Yes, enough order of magnitude change can make a quantitative change into a qualitative one. But so far the proof that this holds for intelligence is eluding us. Maybe we're just not intelligent enough, in the same way that you can't 100% describe a system from within the system.
      - mlyle 7y ago
        If it's possible to implement AGI on a computer, a 4004 with enough RAM attached can do it... very slowly. So the speed/size trade-off is implicit unless we take a pseudo-spiritualist view that computing machines just can't be intelligent. There's two big options out there: either it requires much, much more computation than we can readily employ even to do it relatively "slowly", or we don't know how. The former has a little bearing on the balance between a transistor's computational capabilities and a neuron's, but even so is largely orthogonal. As to parallelism, it doesn't work that way. If you get nine women pregnant, you don't get a baby per month. If we've made a working but far-too-slow-for-us-to-realize-it AGI, making more of them doesn't help us understand the problem.
        jacquesm 7y ago
        > we don't know how Exactly.
- trott 7y ago
  > If we had neurons that were 1,000x faster, could we use fewer of them for the same task? Absolutely. In fact, we do: in batched processing (pervasive in deep learning today), CNNs and other weight sharing schemes (e.g. transformers) the same "neurons" get re-used many times.
  - mkl 7y ago
    Those aren't neurons, though. Biology is different, synaptic connections have to physically fit, etc. Individual neurons serve multiple purposes, but I don't think we know if their speed is a limiting factor in that.

mlyle 7y ago

If we're going to talk about net computational power per unit (neuron/transistor), we also need to note that single transistors do stuff a whole lot faster than biological neurons, too. If we had neurons that were 1,000x faster, could we use fewer of them for the same task?

jacquesm 7y ago
Well, we can turn that argument around: if an artificial brain could be faster than a real one, does that mean that right now we have slow artificial brains that are just as good as the real ones? The answer being an obvious 'no' may help answer your question.
- mlyle 7y ago
  Just because we don't know a program that creates a decent artificial brain doesn't mean that such a program doesn't exist for the hardware we have. We're almost certainly going to be able to make speed/size trade-offs in AGI, because we're used to making them in all other computing. Even biological systems make these types of trade-offs. We're beings optimized for the environment we evolved in, not optimized for "smarts". There's many "choices" in energy balance, reaction time, etc, that have been made. Transistors are really powerful and fast computing devices; really our use of them in digital logic in some ways greatly under-utilizes them for ease in design and predictability. Saying that it takes 4 transistors to make a really simple neuron analog in a circuit undersells that those 4 transistors can operate many, many orders of magnitude faster.
  - jcims 7y ago
    Faster than what? There are a hundred trillion atoms in a neuron operating in a coordinated fashion to create its global behavior in the context of the brain. How many interactions between those components occur in a billionth of a second? Many.
- tlb 7y ago
  It would be very hard to tell if an AGI running 1000x slower than a brain were intelligent. Humans babies don’t talk for about a year, but nobody has tried firing up a putative slow AI and waiting for 1000 years.
  - jacquesm 7y ago
    Then let's run 1000 of them in parallel, right? You're on to the exact point I'm trying to make. The whole idea that 'just' complexity or 'just' speed is enough to make something intelligent that is not yet intelligent is off-base to a degree that resorting to metaphors to try to explain why also breaks down. I liken intelligence to life: once it exists it is obvious and self-perpetrating but before it exists it is non-obvious and no amount of fantasizing about what it would take to make it will get you incrementally closer until one day you've got it. But that day will start out just like the thousands of days before it. It's a quantum leap, a difference in quality, not an incremental one. Yes, enough order of magnitude change can make a quantitative change into a qualitative one. But so far the proof that this holds for intelligence is eluding us. Maybe we're just not intelligent enough, in the same way that you can't 100% describe a system from within the system.
    - mlyle 7y ago
      If it's possible to implement AGI on a computer, a 4004 with enough RAM attached can do it... very slowly. So the speed/size trade-off is implicit unless we take a pseudo-spiritualist view that computing machines just can't be intelligent. There's two big options out there: either it requires much, much more computation than we can readily employ even to do it relatively "slowly", or we don't know how. The former has a little bearing on the balance between a transistor's computational capabilities and a neuron's, but even so is largely orthogonal. As to parallelism, it doesn't work that way. If you get nine women pregnant, you don't get a baby per month. If we've made a working but far-too-slow-for-us-to-realize-it AGI, making more of them doesn't help us understand the problem.
      - jacquesm 7y ago
        > we don't know how Exactly.
trott 7y ago
> If we had neurons that were 1,000x faster, could we use fewer of them for the same task? Absolutely. In fact, we do: in batched processing (pervasive in deep learning today), CNNs and other weight sharing schemes (e.g. transformers) the same "neurons" get re-used many times.
- mkl 7y ago
  Those aren't neurons, though. Biology is different, synaptic connections have to physically fit, etc. Individual neurons serve multiple purposes, but I don't think we know if their speed is a limiting factor in that.