3 ms·
This is very informal and the blog post is systematic and probably on its way to a peer-reviewed paper. > How can the author believe that his examples won't b
by cscurmudgeon 4y ago
This is very informal and the blog post is systematic and probably on its way to a peer-reviewed paper.
> How can the author believe that his examples won't be solved in the near future?
OpenAI folks are not that rosy lol (https://futurism.com/the-byte/openai-ceo-people-will-be-disappointed-gpt4 https://futurism.com/the-byte/openai-ceo-people-will-be-disa...)
Established researchers think LLMs won't get to us human-level AI (Yann LeCun https://twitter.com/ylecun/status/1621810424910958592 https://twitter.com/ylecun/status/1621810424910958592), but let us assume they know nothing.
Let us discount all that. Let us assume LLMs will do that. What sized model will solve logic at or near the precision formal logic gives us? A 10 times bigger model? A 100 times bigger model? A 1e9 times bigger model? Can formal logic + deep learning give us a shorter path?
> Regarding AI conferences. Researchers often continue to ride dead horses if they find them interesting. As far as I know, symbolic approaches have never lead to any major success in AI. All the big AI achievements in recent years are based on machine learning, on deep neural networks specifically.
SAT solvers (a small part of symbolic approaches) are used in a lot of places:
https://ai.stackexchange.com/questions/67/what-are-the-real-world-uses-for-sat-solvers https://ai.stackexchange.com/questions/67/what-are-the-real-...
I remember in 2010 that deep learning was a dead horse. Deep learning was practically useless pre 2010 and a joke too (unlike formal logic which has almost all of its current uses outside of AI).
Sorry, this position is very short sighted, and I am young person and was a kid during the 90s but I have read the history of AI. This is like claiming in 1990s and early 2000s all the advances in the past decade were from non deep learning methods and saying we have to stop research in that. Imagine how bad that would have been.
Interestingly, I see experienced researchers (or complete novices and lay folks) taking your position and younger ones in AI being more balanced. I guess the experienced folks have a moat that needs protection and novices fall prey to popular press.
Imagine thinking Newtonian mechanics is all that is needed as it stood for centuries. Thankfully, physicists were not short sighted. Hopefully, AI researchers resist the pressure to be short sighted and fashionable.
Penrose wrote about this very same thing:
"Fashion, Faith, and Fantasy in the New Physics of the Universe"
https://press.princeton.edu/books/hardcover/9780691119793/fashion-faith-and-fantasy-in-the-new-physics-of-the-universe https://press.princeton.edu/books/hardcover/9780691119793/fa...
- cubefox 4y agoIn general, there seems to be a big difference between a dead horse and something being unfashionable. > I remember in 2010 that deep learning was a dead horse. Deep learning was practically useless pre 2010 and a joke too (unlike formal logic which has almost all of its current uses outside of AI By 2010 digit and text recognition was already long in use. It used hidden layers, so it was a form of deep learning.
- cscurmudgeon 4y ago> In general, there seems to be a big difference between a dead horse and something being unfashionable. That is subjective and shifts with time as we have seen multiple times in physics and AI. You don't get to decide what is dead. AI researchers as a group decide that and nature agrees/disagrees. > By 2010 digit and text recognition was already long in use. It used hidden layers, so it was a form of deep learning. False. I worked in that field and deployed production systems on mobile devices. SVMs with hand crafted were the rage with tons of hand crafted systems. No one serious used deep learning. Even if it was used in digit recognition, that is just one use case compared to dozens of uses cases for symbolic systems (e.g. hardware verification) in 2023. I don't blame folks for thinking that LLMs are the answer for everything (they maybe but we don't know yet). Folks felt the same about 1950s systems all the way till the AI winter.
- cubefox 4y ago2010 was just two years before AlexNet's spectacular success at the ImageNet challenge. Which used deep learning to categorize 2D photographs of 3D objects, not merely 2D scans of 2D digits and letters. Hard to believe just two years earlier they didn't even use it for the latter task.
- deleted 4y ago[deleted]