2 ms·
> I gave 7 frontier LLMs a simple task: pilot a drone through a 3D voxel world and find 3 creatures. > Only one could do it. If I understood the chart correct
by zahlman 10mo ago
> I gave 7 frontier LLMs a simple task: pilot a drone through a 3D voxel world and find 3 creatures.
> Only one could do it.
If I understood the chart correctly, even the successful one only found 1/6 of the creatures across multiple runs.
- uoaei 10mo agoNo science detected. Without comparison to some null hypothesis (a random policy), this article is hogwash.
- zahlman 10mo agoGiven that all the other agents failed to find any creatures, it's hard to imagine that a random policy would except by extreme coincidence.
- TOMDM 10mo agoIt is possible to be consistently wrong in a way that randomness is not. For some problems, randomness outperforms incompetent reasoning