Note that in the paper, all the numbers used were under 10. > AIW Variation 1, N=3,M=6,C=7 > AIW Variation 2, N=4,M=2,C=3 > AIW Variation 3, N=1,M=4,C=5 > A

Note that in the paper, all the numbers used were under 10. > AIW Variation 1, N=3,M=6,C=7 > AIW Variation 2, N=4,M=2,C=3 > AIW Variation 3, N=1,M=4,C=5 > AIW Variation 4, N=4,M=1,C=2. Also note that the resricted prompt is only one of the prompt variations tested by the paper. It also explores common techinques to get LLMs to perform better, including "thinking out loud". Even with these methods the models still fail to produce a correct answer. > Model prompt types. It is well known that so-called prompt engineering can heavily influence the model behavior and model response quality [26, 27, 28]. To account for the response variations due to various prompt forms, we created 3 distinct prompt types asking for the solution to the AIW problem: STANDARD, THINKING, and RESTRICTED. The STANDARD prompt type asks to solve the posed problem and output the final answer in the format as described above. This does not put any specific requirements on model behavior. The THINKING prompt type extends STANDARD with the request to think carefully and double check the solution for any mistakes. This should encourage model to invest more computation into obtaining the solution. In contrast to this, the RESTRICTED prompt urges the model to output only the final answer without any further text. This is supposed to restrict compute invested in producing output. We observe substantially shorter outputs across tested models compared to STANDARD and THINKING for this prompt type (Suppl. Fig. 13).