Even Mistral 7B ("Tiny") gets this right if you ask it to do the reasoning first. Kaley has 4 brothers. Each of her brothers has 2 sisters. How many sisters do

Even Mistral 7B ("Tiny") gets this right if you ask it to do the reasoning first. Kaley has 4 brothers. Each of her brothers has 2 sisters. How many sisters does Kaley have? Give the reasoning first and then the answer. Otherwise it starts by giving the wrong answer and then puts some reasons after. The 8x7B version ("Small") gives the reasoning first by default, but still gets the answer wrong unless I include the instructions to do that anyway. I thought it was funny, but Orca 2 13B consistently gave the wrong answer despite being trained to talk things out verbosely.