4 ms·

In the absence of any prior knowledge, perhaps. But if you're aware that "missionaries and cannibals" [1] is a common toy problem for LLMs, this variation becom

In the absence of any prior knowledge, perhaps. But if you're aware that "missionaries and cannibals" [1] is a common toy problem for LLMs, this variation becomes an apparent anti-logic puzzle. Also, because we know that punctuation, capitalization, and special characters are removed prior to vectorization, they can be disregarded by us, as well. We do not need more perfect grammar or syntax. Thus, there is sufficient probability of intent in the information encoded that we can reject any attempts to solve it that use cannibalism as a constraint. That is the test. [1] https://en.m.wikipedia.org/wiki/Missionaries_and_cannibals_problem https://en.m.wikipedia.org/wiki/Missionaries_and_cannibals_p...

vidarh 3y ago
It might well be an "anti logic" puzzle but that doesn't change the severely broken grammar which makes it unclear how exactly the rules are changed. And the fact several of us were left here wondering about the meaning is clear evidence that it's unreasonable to expect it to be comprehensible to an LLM, and even more so to treat it as a good test, because it's severely compromised by the broken description. I stand by my assessment, which is that an LLM should be rated as failing this unless it responds by either setting out a plausible interpretation and asking if it is right or asking for a clarification. And so should people. The illogical response here is to assume toknow (rather than is guessing) what was intended. The expected response would be to be confused, as you can see from the replies.