2 ms·

> "how many r's in strawberry" or "s's in espresso". And what percentage of your red retina receptors are firing? The "number of letters" critique was broken

> "how many r's in strawberry" or "s's in espresso". And what percentage of your red retina receptors are firing? The "number of letters" critique was broken before it was introduced the first time. The models were specifically designed with preprocessing to not be able to perceive their input as strings of letters. Blind people are not dumb. (True as a pun and in context.) Sub-access sensory questions, or do-you-know-a-fact questions (which is what spelling becomes when you can't see the letters, and are not specifically trained to match all token encoded words to their letters) are not intelligence questions.

butterisgood 22d ago
The critique isn't broken. If the model cannot count letters in a word what happens when it needs to do something akin to counting the letters in a word? I believe it could easily write a tool to count the letters in a word for frequency, but ... dismissing this as if it doesn't matter seems a bit premature without deeper understanding of bad answers you can get from these tools
- Nevermark 18d ago
  I am not dismissing anything. But if a model is built to be insensitive to something by design, it isn't a good indicator of its overall capabilities. In fact, it is uniquely poor at testing its capabilities. I suggest you really try and count the red retina cells that are firing in your field of vision right now. Or, instead of looking at text, count the e's while someone is talking to you. NOTE: you know how to spell. But try it... Then consider why you can't. On the other hand, give a text file to a model, and it can count the e's easily. The same information is now in a stable form it can operate on with its higher level functioning. Who do models and humans have preprocessing layers that strip so much information away? To greatly reduce the cost of operating on information for most purposes - while making other types of operation impossible. When it is presented that way.

butterisgood 22d ago

The critique isn't broken. If the model cannot count letters in a word what happens when it needs to do something akin to counting the letters in a word? I believe it could easily write a tool to count the letters in a word for frequency, but ... dismissing this as if it doesn't matter seems a bit premature without deeper understanding of bad answers you can get from these tools

Nevermark 18d ago
I am not dismissing anything. But if a model is built to be insensitive to something by design, it isn't a good indicator of its overall capabilities. In fact, it is uniquely poor at testing its capabilities. I suggest you really try and count the red retina cells that are firing in your field of vision right now. Or, instead of looking at text, count the e's while someone is talking to you. NOTE: you know how to spell. But try it... Then consider why you can't. On the other hand, give a text file to a model, and it can count the e's easily. The same information is now in a stable form it can operate on with its higher level functioning. Who do models and humans have preprocessing layers that strip so much information away? To greatly reduce the cost of operating on information for most purposes - while making other types of operation impossible. When it is presented that way.