2 ms·

This seems analogous to the beat to death "strawberry" test where you're essentially testing if the model is trained on tokens or not. When you switch to having

This seems analogous to the beat to death "strawberry" test where you're essentially testing if the model is trained on tokens or not. When you switch to having it write a Python program (in both scenarios) instead of trying to process it directly you stop trying to work directly with individual letter inputs+outputs so it does better.

thorum 2y ago
Reasoning models are supposed to be able to work around that kind of limitation (o1 was code named strawberry after all) so it’s not a bad test IMO.
xigoi 2y ago
However, it shows that the AI is not self-aware enough to realize by itself that it needs to write a program to get the correct answer.
- zamadatix 2y ago
  If Calc 2 students don't respond to integration questions with Mathematica syntax is the conclusion they aren't self aware or they are aware how they are tested? I.e. what you're measuring is what the training environment considered a valid response, not what "self awareness" the AI has to produce the answer you individually expect it to. The model doesn't include a Python interpreter to resolve the answer with so answers which output a Python script then just sat back early were severely penalized vs actual attempts. Explicit requests for a Python script instead of an answer excluded as a Python script is a valid answer to that question. The following is a complete assumption... but I wonder if "separately trained model + added tooling explicitly called in the prompt" results in stronger reasoning strength compared to "model trained with the assumption it can use external tooling as part of its answers all the time". I.e. if you're not pushing a model to encode more of the mechanics behind things does it "reason" as well about what to do with them vs adding the tooling on top of the model at the end.

2 ms·

This seems analogous to the beat to death "strawberry" test where you're essentially testing if the model is trained on tokens or not. When you switch to having

by zamadatix 2y ago

thorum 2y ago
Reasoning models are supposed to be able to work around that kind of limitation (o1 was code named strawberry after all) so it’s not a bad test IMO.
xigoi 2y ago
However, it shows that the AI is not self-aware enough to realize by itself that it needs to write a program to get the correct answer.
- zamadatix 2y ago
  If Calc 2 students don't respond to integration questions with Mathematica syntax is the conclusion they aren't self aware or they are aware how they are tested? I.e. what you're measuring is what the training environment considered a valid response, not what "self awareness" the AI has to produce the answer you individually expect it to. The model doesn't include a Python interpreter to resolve the answer with so answers which output a Python script then just sat back early were severely penalized vs actual attempts. Explicit requests for a Python script instead of an answer excluded as a Python script is a valid answer to that question. The following is a complete assumption... but I wonder if "separately trained model + added tooling explicitly called in the prompt" results in stronger reasoning strength compared to "model trained with the assumption it can use external tooling as part of its answers all the time". I.e. if you're not pushing a model to encode more of the mechanics behind things does it "reason" as well about what to do with them vs adding the tooling on top of the model at the end.