5 ms·
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and t
by Animats 4y ago
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive."
That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a major screwup. At, at least, the competence level of the average squirrel. This is why robots are so limited.
If we could build a decent squirrel brain, something "higher level" could give it tasks to do. That would be enough to handle many basic jobs in unstructured spaces, such as store stocking, janitorial, and such. It's not the "high level reasoning" that's the problem. It's the low-level stuff.
A squirrel has around 10 million neurons. Even if neurons are complicated [1], somebody ought to be able to build something with 10 million of them. Current hardware is easily up to the task.
The AI field is fundamentally missing something. I don't know what it is. I took a few shots at this problem back in the 1990s and got nowhere. Others have beaten their head against the wall on this. The Rethink Robotics failure is a notable example.
The real surprise to me is how much progress has been made on vision without manipulation improving much. I'd expected that real-world object recognition would lead to much better manipulation, but it didn't. Even Amazon warehouse bin-picking isn't fully automated yet. Nor is phone manufacturing. Google had a big collection of robots trying to machine-learn basic manual tasks, and they failed at that.
That's the real problem.
[1] https://www.sciencedirect.com/science/article/pii/S0896627321005018 https://www.sciencedirect.com/science/article/pii/S089662732...
- weavejester 4y agoBiological brains have had a few billion years to optimize. Over the past decade or two, it's been increasingly apparent that the structure and algorithms that govern a particular neural net's behaviour are extremely important to its efficacy. We likely have a very warped view of what intelligence is, because the most prominent examples of it have been aggressively honed over an extremely long period of time to be good at tasks crucial to their survival, such as effectively navigating a 3D environment. We consider art to be a difficult and complex task, and making a sandwich to be a simple one, but that's because our particular brand of intelligence is optimized toward the latter.
- TuringTest 4y ago> Biological brains have had a few billion years to optimize. Not just that, but it's grown on a body which has been optimized for survival during a few billion years; and that body is built on cells that have evolved to survive hostile environments, and those cells are built with self-replicating molecules, evolving from complex chemical reactions in several changing environments, that competed with and displaced other less-successful self-replicating molecules that disappeared. Each of those layers provides a degree of adaptability and self-healing that is extremely hard to replicate. And if we managed to reverse-engineer and replicate one of those layers, it would still be missing all the layers below. Our best hope to create fully independent agents will come from re-adapting and controlling biological entities, not from tools built from the ground up with current engineering techniques.
- nikbackm 4y agoMulti-cellular life has only been around for 600 million years or so.
- jhoechtl 4y ago> The AI field is fundamentally missing something. I don't know what it is. I can only speculate but it certainly is for a reason that certain parts of our body are vegetativelly controlled whereas others are under the active control of our consciousness. If you step on a nail, the first reaction comes from vegetative stimulus, later your consciousness processes that information. A squirrels neuronal network is also separated in that way. That may be a reason. And second, AFAIK AI still doesn't 'think' in concepts, it has no notion of the 'world'. And third: The capability of reproduction and acting accordingly may be another thing.
- bravura 4y agoImagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas things we think are complicated, like playing Go or picking stocks or computer programming, are actually quite simple to learn. This would be counter-intuitive but---as you observed, and taking my argument recursively---common sense might be much more difficult to get right than obscene pathological thinking. Anyway, I've always thought a good startup would be to automate away Silicon Valley using AI. It's so punk rock that a lot of disillusioned smart techies would join under this banner. A collaborator of mine has already used AI to do high-level bug finding in blockchain code.
- te_chris 4y agoI think there's a lot of truth to this. I'm new to ML, still going through the ropes on some online courses, but already I can see that, once I get a bit of muscle memory in setting up models etc, there's a whole lot of power and efficiency to be unlocked by using simple models - specifically in CS/X and Marketing. Obviously model quality matters, so you have to have proper monitoring etc, but this stuff is low hanging fruit and should enable teams to be so much more efficient.
- pydry 4y agoIn a lot of these cases - maybe most, in fact - the sludge in the data pipeline makes makes the low hanging fruit hang high. I've worked on a number of projects where it looked simple to automate from the outset and impossible in retrospect.
- te_chris 4y agoAgree entirely, I wouldn’t want to build a saas startup around it due to data quality, but if you have control of your pipeline it’s easier.
- data_maan 4y ago> Current hardware is easily up to the task. I don't think so. If you want to model a single synapse in full to capture all effects that might lead to "learning", you have a system of ordinary differential equations. Solving that is very hard, and solving that for 10 million neurons is impossible. On current hardware can only implement but a poor caricature of a real neuron.
- carlmr 4y agoWhile this is true, the complexity perspective misses something more fundamental. 1) Our brains, and moreso those of animals, come with a really good pretraining at birth. This is collective genetic knowledge of millions of generations distilled into your brain. 2) Our brains have a lot of sensors and actuators to interact with the world. We only learn by reading as adults when our brains can already do the synesthesia of translating words into thought. But even as adults, most of us learn better if we do something, write something, engage in dialog, instead of passively listening, reading, or watching. Passive data can never replicate the rich environment our brains grow up in.
- beisner 4y agoWhile true, there’s a relatively small upper bound on how many bits of information are in this pre-training. Specifically, in the form of how much information is contained in DNA, which is only a couple gigabytes.
- lossolo 4y agoStable diffusion model is around 4 gigabytes, inside that 4 gigabytes you have understanding of the whole english language model and mapping to billions of objects, people, concepts etc capable of generating from just a single sentence almost any picture in any style you imagine. Seems like a few gigabytes can hold a lot of information.
- mkaic 4y agoI think this requires the assumption that modeling the complexity of biological synapses is required for general intelligence, when we don't know that to be the case. Personally, I believe that it's not a requirement at all and that the first AGI will be strikingly non-neuromorphic. Just my two cents though.
- ilaksh 4y agoThey need well-factored accurate multimodal world models. Things like transformers and stable diffusion are promising such as the 3d video stable diffusion paper or DeepMind's multimodal transformer. One thing that has held back progress is the way putting knowledge directly into the system has become taboo. So much so that they often fail to even guide the training towards really core aspects of the world model. Or even deliberately going about it with the assumption that everything from start to finish must be determined from the barest input data such as pixels. Then being surprised when it learns random inaccurate and overfit models that miss the underlying hierarchical structures.
- WeylandYutani 4y agoAutomation only happens when it's cheaper than the human counterpart. In the US there are plenty of immigrants whom cost peanuts to employ. I don't expect the robotic future to come from North America.
- geysersam 4y ago> A squirrel has around 10 million neurons. I don't believe this is correct. It's too low. It's more like 400 million. > somebody ought to be able to build something with 10 million of them Build them, sure, but they need to be connected in the right way.
- otikik 4y agoThere’s not a single missing something, there’s at least 2. One of them is physical structure. You can get 10M somethings, sure, but how do you wire them together is probably more important than how many there are. And there’s many possible combinations. The other missing part is that we have not figured out the high level software. A squirrel brain is a “desktop PC running windows”-level of utility. A bunch of neurons interlinked is some fashion is equivalent to a blank CPU. We know how the individual transistors work, but the BIOS, and OS are still unknown. It’s quite possible that problem 1 and problem 2 are related, because evolution doesn’t care about making things easy to understand for us with clear delimitations.