2 ms·
Have a look at https://arcprize.org/ https://arcprize.org/ They have hundreds of challenges that humans can solve in under a minute which LLMs can not. Seems t
by Gigachad 1y ago
Have a look at https://arcprize.org/ https://arcprize.org/
They have hundreds of challenges that humans can solve in under a minute which LLMs can not. Seems the general trend is figuring out the rules or patterns of the challenge when there are few examples and no instructions.
- petesergeant 1y agoAh, it also needs to be challenging for humans. It's a prize to win something. I just didn't want people to throw the question into Claude Code.
- mewpmewp2 1y agoPerhaps coding exercises that require 2d or 3d thinking, or similar. This is where I have seen LLMs struggle a lot. There are probably other areas too.