This is absolutely relevant for testing LLMs, and benchmarks do contain questions like this. These aren't "gotchas", a good answer requires a level of cognition

This is absolutely relevant for testing LLMs, and benchmarks do contain questions like this. These aren't "gotchas", a good answer requires a level of cognition and reasoning you wouldn't expect, say, a child to be able to navigate well. Even if you don't expect to ask the question, the reason why these questions are asked for research is to validate some internal, unrevealed reasoning of more-normal questions. Considering the spectacular well-documented failure, Gemini's reliability for producing other answers is certainly in jeopardy.