4 ms·

> the fact that a machine learning model trained in an unsupervised fashion can succeed that much at anything non-trivial is mind-boggling to me, and makes me h

> the fact that a machine learning model trained in an unsupervised fashion can succeed that much at anything non-trivial is mind-boggling to me, and makes me hopeful that other tasks will also be within reach relatively soon. To me, on the contrary, GPT-3 is quite a saddening thing. With every example I come across, I get a more and more clear idea of the limits of current mainstream approaches in AI. When it comes to predicting or achieving human-level general behavior it seems to me that the SOTA is reaching a hard limit that lies far below actual AGI. I'm not a researcher in this field, so this opinion should be taken with more than a grain of salt, but if I'm right, 10 years from now we're still gonna be stuck at GPT-3-ish levels of clever mimicry.

m12k 6y ago
Back when I took my cs degree I learned about the AI winter - the decades that came after we came up with effective planning, reasoning and problem solving algorithms, but realized we couldn't use them on anything in the real world, because it is fuzzy and imprecise and we don't have an interface to put it into the clean mathematical world of those algorithms. I think basically, we realized that the higher reasoning and planning capabilities of a human mind that we were emulating were worth nothing without the fuzzy pattern recognition capabilities of our reptile brain to recognize and categorize objects for us to even reason about. The way I see it, the deep learning revolution of recent years is basically providing us with that missing link, the fuzzy recognition engine that serves as a foundation for higher reasoning. It still remains to be seen if this approach will scale to AGI on its own or it needs to be combined with others. But saying we are running up against a wall with that is like saying we have run up against the wall of what a visual cortex can do - the real fun begins when we combine it with other components, and those other components are still being developed.

m12k 6y ago

Back when I took my cs degree I learned about the AI winter - the decades that came after we came up with effective planning, reasoning and problem solving algorithms, but realized we couldn't use them on anything in the real world, because it is fuzzy and imprecise and we don't have an interface to put it into the clean mathematical world of those algorithms. I think basically, we realized that the higher reasoning and planning capabilities of a human mind that we were emulating were worth nothing without the fuzzy pattern recognition capabilities of our reptile brain to recognize and categorize objects for us to even reason about. The way I see it, the deep learning revolution of recent years is basically providing us with that missing link, the fuzzy recognition engine that serves as a foundation for higher reasoning. It still remains to be seen if this approach will scale to AGI on its own or it needs to be combined with others. But saying we are running up against a wall with that is like saying we have run up against the wall of what a visual cortex can do - the real fun begins when we combine it with other components, and those other components are still being developed.