Planning is a weakness for LLMs sure but i don't think this benchmark is any good. Mostly because it fails to do what a good benchmark should do i.e Reasonably

Planning is a weakness for LLMs sure but i don't think this benchmark is any good. Mostly because it fails to do what a good benchmark should do i.e Reasonably inform you of the models capabilities. You would come away from this thinking LLMs couldn't so much as stack blocks or perform tasks of that kind. But they can. https://innermonologue.github.io/ https://innermonologue.github.io/ also the benchmark is too spatially focused to be a general planning benchmark.