4 ms·
"AI is harder than we think" of course depends a bit on who "we" is. The paper seems to focus a bit on the types who thought it's a bit of symbol manipulation
by tim333 6y ago
"AI is harder than we think"
of course depends a bit on who "we" is. The paper seems to focus a bit on the types who thought it's a bit of symbol manipulation you can do in lisp which has always seemed a bit dumb. Then the "it'll never happen" crowd seems dumb as well given people do think and seem to be built from atoms and stuff obeying normal physics.
The more sensible view it seems to me is to compare neural computation to what we can do with silicon and then project on a Moore's law type assumption as to when it would be likely, in the manner of Moravec and Kurzweil which has always put the date around 2030- 2040. And obviously it'll be hard like doing a moon landing is hard but probably not impossible like that also.
Moravec's 1998 paper had "it is predicted that the required hardware will be available in cheap machines in the 2020s" and then I guess we need the software.
- dvfjsdhgfv 6y agoWhile I agree, one of the biggest stumbling blocks is the engineering challenge. The architectures we're using nowadays for machine learning are terribly inefficient when compared to human brain (energy consumption- and performance-wise). And we already hitting the wall with the size we could go down to and frequency, so a completely novel approach would be needed if we want to use the resultant AI outside of datacenters. Given the recent developments, I don't think it's impossible to use laboratory-grown neurons for such development, although it's hard to imaging something like this appearing between 2030 and 2040.
- ikrenji 6y agoWe also don't have the right computing machine. Each neuron integrates signals from hundreds of other neurons and it then propagates this signal to another set of hundreds of neurons. I'm not sure how it compares to what a computer does, but I'm fairly certain it's not that
- dvfjsdhgfv 6y agoYes, I very much hope we develop in this direction, maybe also using synaptic transistors etc., and both neuroscience and computer science could greatly benefit from this.
- chriswarbo 6y agoI don't think this is too bad, since we care much more about prototyping at this stage. If some particular AI approach/architecture becomes 'good enough' for many applications, it wouldn't be long until ASICs for that calculation would arrive; probably not as good as the brain, but certainly orders of magnitude better than general-purpose CPUs/GPUs. There's also ongoing work involving ML with low-precision floats, etc. which can make such circuitry even more efficient. The reason we don't really see this today is that recouping the investment for designing and fabbing such ASICs may take a few years, and they would probably be obsolete by then, given today's rapid changes. For example, a few years ago there might have been a clear business proposition for putting ASICs dedicated to convolutional neural networks into cameras (even camera phones); yet CNNs now seem to be phasing out in favour of transformers; and it's not at all clear what the "best" transformer is yet (e.g. look at all the different approaches to making them O(n) memory!)
- YeGoblynQueenne 6y ago>> The paper seems to focus a bit on the types who thought it's a bit of symbol manipulation you can do in lisp which has always seemed a bit dumb. About half of the paper is about modern deep neural networks, but can you please give a few examples of the kind of AI you say is "a bit of symbol manipulation you can do in lisp"? Because I would say this is an extreme oversimplification borne by terminal unfamiliarity with the subject, beyond what may be commented on twitter. For instance, remember that Deep Blue, despite its name, had nothing to do with deep learning or machine learning of any kind, and was a manually programmed system, yet I doubt anyone would sensibly describe it as "a bit of symbol manipulation you can do in lisp". But please give examples of what you mean.
- teddyh 6y agoThey are referring to “classic AI”, as exemplified by SHRDLU¹ and Cyc². 1. https://en.wikipedia.org/wiki/SHRDLU https://en.wikipedia.org/wiki/SHRDLU 2. https://en.wikipedia.org/wiki/Cyc https://en.wikipedia.org/wiki/Cyc
- YeGoblynQueenne 6y agoThe capabilities of SHRDLU are still unsurpassed by modern NLP systems, unfortunately. Cyc is a different matter. I don't really know how close the project is to achieveing its goals and it always seemed like a bit of a futile moonshot to me, but it's certainly not "a bit of symbolic manipulation you can do in lisp". If nothing else, the sheer scale of a project that has been going on for almost 40 years, should cause a reduction of the arrogance and brashness of proclamations about it.
- kthejoker2 6y agoSymbolic AI is a well known topic. https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence?wprov=sfla1 https://en.wikipedia.org/wiki/Symbolic_artificial_intelligen... As the article notes > Symbolic AI was the dominant paradigm of AI research from the mid-1950s until the late 1980s. And, yes, Deep Blue was just "a bit of symbolic manipulation you can do in Lisp" at a very large hardware scale. It simply brute force calculated the value of as many position trees as it could up to ply 8, and the "value calculator" was a set of rules based on human expert inputs. Creating Deep Blue's hardware in 1997 was the most impressive achievement. Anyway, symbolic AI still potentially may be the path to AGI, but obviously ML/DL techniques utilizing very large datasets snd very large architectures bore a lot of useful fruit.
- andrepd 6y agoIt's true that those two approaches are "a bit dumb", but the statement > The more sensible view it seems to me is to compare neural computation to what we can do with silicon and then project on a Moore's law type assumption as to when it would be likely is no less dumb imo. What makes you say this? What's the evidence? What neural network system has achieved something even close to being on a path to AI?
- hyperpallium2 6y ago1. Moore's law isn't on track. 2. Maybe neurons do internal computation, meaning brain complexity is under-estimated.