> Still gets 3 r's in strawberry shortcake in a fresh context: https://i.imgur.com/EGI2UT9.png https://i.imgur.com/EGI2UT9.png Right, but again this would be v

> Still gets 3 r's in strawberry shortcake in a fresh context: https://i.imgur.com/EGI2UT9.png https://i.imgur.com/EGI2UT9.png Right, but again this would be valid according to the r-sound-interpretation. To be precise, we actually have 3 possible interpretations of the question: 1- how many r's are spelled in strawberry in total (this is what everyone assumes is the intended question) 2- how many r-sounds are in strawberry 3- is it 1 or 2 r's in strawberry My theory was just that we don't know which of the three the LLM thinks was intended, given that its answer is valid for 2/3 of them. But I must admit that it seems hard to find examples where the LLM clearly assumes option 2 or 3. So maybe the token-based explanation suffices after all.