13 ms·

While I agree, one of the biggest stumbling blocks is the engineering challenge. The architectures we're using nowadays for machine learning are terribly ineffi

While I agree, one of the biggest stumbling blocks is the engineering challenge. The architectures we're using nowadays for machine learning are terribly inefficient when compared to human brain (energy consumption- and performance-wise). And we already hitting the wall with the size we could go down to and frequency, so a completely novel approach would be needed if we want to use the resultant AI outside of datacenters. Given the recent developments, I don't think it's impossible to use laboratory-grown neurons for such development, although it's hard to imaging something like this appearing between 2030 and 2040.

ikrenji 6y ago
We also don't have the right computing machine. Each neuron integrates signals from hundreds of other neurons and it then propagates this signal to another set of hundreds of neurons. I'm not sure how it compares to what a computer does, but I'm fairly certain it's not that
- dvfjsdhgfv 6y ago
  Yes, I very much hope we develop in this direction, maybe also using synaptic transistors etc., and both neuroscience and computer science could greatly benefit from this.
chriswarbo 6y ago
I don't think this is too bad, since we care much more about prototyping at this stage. If some particular AI approach/architecture becomes 'good enough' for many applications, it wouldn't be long until ASICs for that calculation would arrive; probably not as good as the brain, but certainly orders of magnitude better than general-purpose CPUs/GPUs. There's also ongoing work involving ML with low-precision floats, etc. which can make such circuitry even more efficient. The reason we don't really see this today is that recouping the investment for designing and fabbing such ASICs may take a few years, and they would probably be obsolete by then, given today's rapid changes. For example, a few years ago there might have been a clear business proposition for putting ASICs dedicated to convolutional neural networks into cameras (even camera phones); yet CNNs now seem to be phasing out in favour of transformers; and it's not at all clear what the "best" transformer is yet (e.g. look at all the different approaches to making them O(n) memory!)

ikrenji 6y ago

We also don't have the right computing machine. Each neuron integrates signals from hundreds of other neurons and it then propagates this signal to another set of hundreds of neurons. I'm not sure how it compares to what a computer does, but I'm fairly certain it's not that

dvfjsdhgfv 6y ago
Yes, I very much hope we develop in this direction, maybe also using synaptic transistors etc., and both neuroscience and computer science could greatly benefit from this.

chriswarbo 6y ago

I don't think this is too bad, since we care much more about prototyping at this stage. If some particular AI approach/architecture becomes 'good enough' for many applications, it wouldn't be long until ASICs for that calculation would arrive; probably not as good as the brain, but certainly orders of magnitude better than general-purpose CPUs/GPUs. There's also ongoing work involving ML with low-precision floats, etc. which can make such circuitry even more efficient. The reason we don't really see this today is that recouping the investment for designing and fabbing such ASICs may take a few years, and they would probably be obsolete by then, given today's rapid changes. For example, a few years ago there might have been a clear business proposition for putting ASICs dedicated to convolutional neural networks into cameras (even camera phones); yet CNNs now seem to be phasing out in favour of transformers; and it's not at all clear what the "best" transformer is yet (e.g. look at all the different approaches to making them O(n) memory!)