3 ms·
Seems like there is always push back on LLM's that they don't learn to do proofs and reasoning. Deepmind just placed pretty high at International Mathematical
by FrustratedMonky 3y ago
Seems like there is always push back on LLM's that they don't learn to do proofs and reasoning.
Deepmind just placed pretty high at International Mathematical Olympiad . Here it does have to present reasoning.
https://arstechnica.com/ai/2024/01/deepmind-ai-rivals-the-worlds-smartest-high-schoolers-at-geometry/ https://arstechnica.com/ai/2024/01/deepmind-ai-rivals-the-wo...
And it's couple years old, but AlphaFold was pretty impressive.
EDIT: Sorry, I said LLM. But meant AI/ML/NN generally, people say a computer can't reason, but DeepMind is doing it.
- imtringued 3y ago>To overcome this difficulty, DeepMind paired a language model with a more traditional symbolic deduction engine that performs algebraic and geometric reasoning. I couldn't think of a better way to demonstrate that LLMs are poor at reasoning than using this crutch.
- fennecfoxy 3y agoI suppose it's because LLM training data uses text that can contain reasoning within it, but without any specific context to specifically learn reasoning. I feel like the little reasoning an LLM can do is a byproduct of the training data. Does seem more realistic to train something not on text but on actual reasoning/logic concepts and use that along with other models for something more general purpose. LLMs should really only be used to turn "thoughts" into text and to receive instructions, not to do the actual reasoning.
- FrustratedMonky 3y agoI wouldn't say 'crutch' but component. Eventually LLMs will be plugged into Vision Systems, and Symbolic Systems, and Motion Systems, etc... etc... The LLM wont be the main 'thing'. But the text interface. Even human brain is bit segmented with different faculties being 'processed' in different areas with different architectures.