3 ms·
I think it's really hard to pin down what reasoning is and measure it precisely. How on earth would you do this?
by qnleigh 2y ago
I think it's really hard to pin down what reasoning is and measure it precisely. How on earth would you do this?
- daveguy 2y agoThe best example I have seen for true AGI benchmarking is ARC-AGI: https://arcprize.org/ https://arcprize.org/ As the ARC-AGI is more trained to we put more of our own expertise and knowledge into the algorithms. When a computer can get human level results on a new ARC-AGI like benchmark (transfer from other intelligence tasks), then we are very close.
- qnleigh 2y agoThe creators of the dataset have said themselves that it does not imply AGI > it is important to note that ARC-AGI is not an acid test for AGI – as we've repeated dozens of times this year. It's a research tool designed to focus attention on the most challenging unsolved problems in AI https://arcprize.org/blog/oai-o3-pub-breakthrough https://arcprize.org/blog/oai-o3-pub-breakthrough
- daveguy 2y agoThat was the whole point of the entire second paragraph.