3 ms·

I’m not an AI zealot, but this is such a strange debate to me. “AI only _appears_ to be reasoning” is silly, because appearing to reason is our only sane defin

I’m not an AI zealot, but this is such a strange debate to me. “AI only _appears_ to be reasoning” is silly, because appearing to reason is our only sane definition of reasoning. If we can’t say exactly when our random statistical noise emerges into some vague concept of intelligence, why does our standard suddenly become rock solid when applied to other intelligences?

sgt101 3y ago
My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if it were an approximate retrieval system with an enormous knowledge base. this is a good paper: https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288
- famouswaffles 3y ago
  The "reversal curse" has nothing todo with inference or the reasoning abilities of a pre-trained model. GPT-4 or whatever has no problem telling you the right deduction with information presented in the context. This is a memory retrieval from training issue and is completely unsurprising.
ren_engineer 3y ago
I'd say the main issue is why does it take ingesting the entire collective knowledge of humanity for LLMs to work if they are actually reasoning and understand things from first principles? Humans don't require that volume of training data to learn how to write or do basic math
- Detrytus 3y ago
  Well, I think you underestimate the amount of "training data" that human gets in the early stages of their life. Just learning to speak takes a few years, during which all people around you talk to you for most of your waking hours. Learning to write and doing basic math typically does not start until age of 6.
  - asadotzler 3y ago
    No chance it comes close to the content of the internet.
- hnfong 3y ago
  You (humanity) don't learn from first principles either. The 3-year-old toddler doesn't learn the rules of English grammar before they learn to speak. Sure, we don't know how humans manage to learn with so "little" data yet, but it's definitely not "understand things from first principles". In fact, if you randomly pick an average person from the streets, they wouldn't be able to explain most things from first principles. (eg. How does their car work?)
alienicecream 3y ago
Reasoning is a process of apprehending facts about objects and inferring how they will relate with other facts. It has nothing to with statistical distributions of words, except in the special case where that distribution has been constructed by humans in a way that reflects their knowledge and inference of those facts. If you feed an LLM garbage, it will happily give you garbage. It doesn't know what garbage is and it doesn't even know that it doesn't know.

sgt101 3y ago

My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if it were an approximate retrieval system with an enormous knowledge base. this is a good paper: https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288

famouswaffles 3y ago
The "reversal curse" has nothing todo with inference or the reasoning abilities of a pre-trained model. GPT-4 or whatever has no problem telling you the right deduction with information presented in the context. This is a memory retrieval from training issue and is completely unsurprising.

ren_engineer 3y ago

I'd say the main issue is why does it take ingesting the entire collective knowledge of humanity for LLMs to work if they are actually reasoning and understand things from first principles? Humans don't require that volume of training data to learn how to write or do basic math

Detrytus 3y ago
Well, I think you underestimate the amount of "training data" that human gets in the early stages of their life. Just learning to speak takes a few years, during which all people around you talk to you for most of your waking hours. Learning to write and doing basic math typically does not start until age of 6.
- asadotzler 3y ago
  No chance it comes close to the content of the internet.
hnfong 3y ago
You (humanity) don't learn from first principles either. The 3-year-old toddler doesn't learn the rules of English grammar before they learn to speak. Sure, we don't know how humans manage to learn with so "little" data yet, but it's definitely not "understand things from first principles". In fact, if you randomly pick an average person from the streets, they wouldn't be able to explain most things from first principles. (eg. How does their car work?)

alienicecream 3y ago

Reasoning is a process of apprehending facts about objects and inferring how they will relate with other facts. It has nothing to with statistical distributions of words, except in the special case where that distribution has been constructed by humans in a way that reflects their knowledge and inference of those facts. If you feed an LLM garbage, it will happily give you garbage. It doesn't know what garbage is and it doesn't even know that it doesn't know.