4 ms·
Yes agreed, there is indeed a rough consensus on what intelligence is and reasonable ways to approximately measure it. These standard tests have been applied to
by oersted 2y ago
Yes agreed, there is indeed a rough consensus on what intelligence is and reasonable ways to approximately measure it. These standard tests have been applied to LLMs from the beginning, they have not proven to be the most helpful to guide research, but there's value to applying benchmarks that have been battle-tested with humans.
It's just that OP was questioning this group's criteria for selecting the questions that determine intelligence. Then we get into endless discussions of semantics.
At the end of the day, you are just testing which questions your AI performs well on, and you can describe how you chose those questions. Claiming it measures "general intelligence" is just unhelpful and frustrating.
- godelski 2y agoThey were applied in the beginning because we really weren't that good at solving the tasks. So like any good researchers, we break it down. But this is like trying to test an elephant but you can't get access to an elephant so you instead train a dog. But putting a dog in an elephant costume doesn't make it an elephant. Sure, dog training will likely mean you can learn to train an elephant faster had you not first trained a dog. Some things transfer, but others don't I also want to stress that there is a rough consensus. But the ML field (which I'm a part of) often ignores this. I'm not sure why. We should be leveraging the work of others, not trying to start from scratch (unless there's good reason, in which case we must be explicit. But I'm just seeing simple claims of "intelligence is ill-defined" and treating that as if that means no definition instead of fuzzy definition. Which gets extra weird when people talk about moving goal posts. That's how progress works? Especially when exploring into the unknown?)