2 ms·

> It might at least currently be on the cognitive level of an overconfident 6yo in many regards, but a 6yo with more knowledge than any single person alive. I'

by Closi 4y ago

> It might at least currently be on the cognitive level of an overconfident 6yo in many regards, but a 6yo with more knowledge than any single person alive. I'm not sure I agree entirely with this - It's very rare to see 6 year olds to be capable of writing valid code, summarising academic literature, or doing diagnostics on a dogs bloodwork. Even if I gave a 6 year old access to all the information and literature, they just wouldn't really be able to do these things. Maybe if we trained the 6 year old for 10 years then they could do these things, but then they aren't 6 anymore they are 16. At age 6 the developmental milestone is really "Can count to and understand the concept of "10."

prox 4y ago
6yo in terms of cognitive performance, not knowledge performance I assume parent comment means. Ultimately comparing LLM or MLS against humans is not a good idea.
- Closi 4y ago
  What do we mean by cognition, how do we measure it, and what aspect of cognitive performance are 6 year olds better at though? Because GPT4 would destroy a 6 year old at a verbal reasoning, quantative reasoning or an abstract reasoning test.
  - prox 4y ago
    Indeed. Lots of noise in communicating exactly what we mean. That is why it is futile to compare to a human. Most people don’t even know how to describe the qualities.
iforgotpassword 4y ago
True, it's not a great analogy because it doesn't apply in every regard as you point out. But I guess if you've used it you might understand what I mean... It's sometimes so hilariously wrong or self-contradicting, which reminds me of how children are always super convinced everything they picked up somewhere has to be true, or maybe did even pick up the right thing but just interpreted it wrong. I tested chatgpt a while ago by giving it some real world bugs I encountered in code in the past. It was really hit or miss, sometimes it confidently reasoned in hilariously nonsensical directions, other times it almost nailed it on first try. What's even more interesting, and what I don't see mentioned that often when people show impressive results, is that often times if you just regenerate an answer with the exact same input, you get a completely dumb and wrong reply the next time, or vice versa. So consistency is another problem.

by Closi 4y ago

prox 4y ago

6yo in terms of cognitive performance, not knowledge performance I assume parent comment means. Ultimately comparing LLM or MLS against humans is not a good idea.

Closi 4y ago
What do we mean by cognition, how do we measure it, and what aspect of cognitive performance are 6 year olds better at though? Because GPT4 would destroy a 6 year old at a verbal reasoning, quantative reasoning or an abstract reasoning test.
- prox 4y ago
  Indeed. Lots of noise in communicating exactly what we mean. That is why it is futile to compare to a human. Most people don’t even know how to describe the qualities.

iforgotpassword 4y ago

True, it's not a great analogy because it doesn't apply in every regard as you point out. But I guess if you've used it you might understand what I mean... It's sometimes so hilariously wrong or self-contradicting, which reminds me of how children are always super convinced everything they picked up somewhere has to be true, or maybe did even pick up the right thing but just interpreted it wrong. I tested chatgpt a while ago by giving it some real world bugs I encountered in code in the past. It was really hit or miss, sometimes it confidently reasoned in hilariously nonsensical directions, other times it almost nailed it on first try. What's even more interesting, and what I don't see mentioned that often when people show impressive results, is that often times if you just regenerate an answer with the exact same input, you get a completely dumb and wrong reply the next time, or vice versa. So consistency is another problem.