3 ms·

My default problem for testing these systems has been this word search puzzle where you get a list of words and a grid of letters, and you need to find each wor

My default problem for testing these systems has been this word search puzzle where you get a list of words and a grid of letters, and you need to find each word in the grid. [0] I just tried it with QvQ and it failed; none of the answers were correct. Interestingly enough, I recently tried the new Gemini release in AI Studio and it also failed at the first pass. With a bit of coaxing I was ultimately able to get it to find one word successfully, and then a second word once it understood the correct strategy. After that attempt I asked it for a program to search the grid for each word, and although the initial attempt failed, it only took 4 iterations of bug fixes to get a fully working program. The two main bugs were: the grid had uppercase letters while the words were lowercase, and one of the words (soup mix) has a space which needs to be stripped when searching the grid. Asking QvQ to generate a program to solve this puzzle... The first try gave me a program that would only check if the word was in the grid or not, but not the actual position of the word. This was partially my fault for using a bad prompt. I updated the prompt to include printing the position of each word, but it just got caught up thinking about the problem. It somehow made a mistake in converting the grid and became convinced that some of the words aren't actually present in the grid. After thinking about the problem for a very long time it ended up giving me an invalid program, and I don't feel particularly motivated to try and debug where it went wrong. What I find most interesting is that asking for a direct solution tends to fail, but asking for a program which solves the problem gets us much closer to a correct answer. Ideally the LLM should be able to just figure out that writing a program is the optimal solution, and then it can run that and extract the answer. [0] https://imgur.com/F8dm8Zo https://imgur.com/F8dm8Zo

zamadatix 2y ago
This seems analogous to the beat to death "strawberry" test where you're essentially testing if the model is trained on tokens or not. When you switch to having it write a Python program (in both scenarios) instead of trying to process it directly you stop trying to work directly with individual letter inputs+outputs so it does better.
- thorum 2y ago
  Reasoning models are supposed to be able to work around that kind of limitation (o1 was code named strawberry after all) so it’s not a bad test IMO.
- xigoi 2y ago
  However, it shows that the AI is not self-aware enough to realize by itself that it needs to write a program to get the correct answer.
  - zamadatix 2y ago
    If Calc 2 students don't respond to integration questions with Mathematica syntax is the conclusion they aren't self aware or they are aware how they are tested? I.e. what you're measuring is what the training environment considered a valid response, not what "self awareness" the AI has to produce the answer you individually expect it to. The model doesn't include a Python interpreter to resolve the answer with so answers which output a Python script then just sat back early were severely penalized vs actual attempts. Explicit requests for a Python script instead of an answer excluded as a Python script is a valid answer to that question. The following is a complete assumption... but I wonder if "separately trained model + added tooling explicitly called in the prompt" results in stronger reasoning strength compared to "model trained with the assumption it can use external tooling as part of its answers all the time". I.e. if you're not pushing a model to encode more of the mechanics behind things does it "reason" as well about what to do with them vs adding the tooling on top of the model at the end.

zamadatix 2y ago

This seems analogous to the beat to death "strawberry" test where you're essentially testing if the model is trained on tokens or not. When you switch to having it write a Python program (in both scenarios) instead of trying to process it directly you stop trying to work directly with individual letter inputs+outputs so it does better.

thorum 2y ago
Reasoning models are supposed to be able to work around that kind of limitation (o1 was code named strawberry after all) so it’s not a bad test IMO.
xigoi 2y ago
However, it shows that the AI is not self-aware enough to realize by itself that it needs to write a program to get the correct answer.
- zamadatix 2y ago
  If Calc 2 students don't respond to integration questions with Mathematica syntax is the conclusion they aren't self aware or they are aware how they are tested? I.e. what you're measuring is what the training environment considered a valid response, not what "self awareness" the AI has to produce the answer you individually expect it to. The model doesn't include a Python interpreter to resolve the answer with so answers which output a Python script then just sat back early were severely penalized vs actual attempts. Explicit requests for a Python script instead of an answer excluded as a Python script is a valid answer to that question. The following is a complete assumption... but I wonder if "separately trained model + added tooling explicitly called in the prompt" results in stronger reasoning strength compared to "model trained with the assumption it can use external tooling as part of its answers all the time". I.e. if you're not pushing a model to encode more of the mechanics behind things does it "reason" as well about what to do with them vs adding the tooling on top of the model at the end.