13 ms·

Hm, could explain why it can't play wordle: > Give me a five letter word, beginning with e, and having third letter be t. Not including I, l, u or h. >> etchy

by e12e 4y ago

Hm, could explain why it can't play wordle: > Give me a five letter word, beginning with e, and having third letter be t. Not including I, l, u or h. >> etchy > Does etchy contain h? >> No, etchy does not contain h. > which letters do etchy contain? >> Etchy contains the letters e, t, c, h, and y. > what were the criteria? >> The criteria specified that the word must have five letters, begin with e, and have the third letter be t. The word was not allowed to include the letters i, l, u, or h. > Does the word etchy meet the criteria? >> Yes, the word etchy meets the criteria because it has five letters, begins with e, has the third letter be t, and does not include the letters i, l, u, or h. Ed: note that insists that the third letter of etchy is t, not c. From what I've seen both of these errors are typical.

phire 4y ago
It's interesting that it can actually extract the letters from the word despite the tokenization approach. I have noticed that it struggles with ordinality and cardinality, which might be part of what's going wrong here. But the other issue with this type of interaction is that once it makes a mistake (claiming etchy meets the criteria), it has a tendency to double down on it and go crazy inventing the reasoning for why. That's because it's mistake gets baked into the input window and becomes a very strong influence over the output. In this case, you probably would have gotten completely different answers if you informed it etchy was an incorrect answer before asking why it got the answer.
mdrzn 4y ago
Tried your prompt, here's the response now: "Enter is a five letter word beginning with e and having t as the third letter." Try again. "Ether is a five-letter word that begins with e, has t as its third letter, and does not include the letters i, l, u, or h. It is a colorless, highly volatile, and flammable liquid that is used as a solvent, especially for gases." Try again. "Emit is a five letter word that begins with e and has the third letter t. It means to give off or produce (something, especially a gas or sound)." So it was correct only the first time.

by e12e 4y ago

phire 4y ago

It's interesting that it can actually extract the letters from the word despite the tokenization approach. I have noticed that it struggles with ordinality and cardinality, which might be part of what's going wrong here. But the other issue with this type of interaction is that once it makes a mistake (claiming etchy meets the criteria), it has a tendency to double down on it and go crazy inventing the reasoning for why. That's because it's mistake gets baked into the input window and becomes a very strong influence over the output. In this case, you probably would have gotten completely different answers if you informed it etchy was an incorrect answer before asking why it got the answer.

mdrzn 4y ago

Tried your prompt, here's the response now: "Enter is a five letter word beginning with e and having t as the third letter." Try again. "Ether is a five-letter word that begins with e, has t as its third letter, and does not include the letters i, l, u, or h. It is a colorless, highly volatile, and flammable liquid that is used as a solvent, especially for gases." Try again. "Emit is a five letter word that begins with e and has the third letter t. It means to give off or produce (something, especially a gas or sound)." So it was correct only the first time.