My experience is that even the "toy" 7B models can do pretty amazing things, but they're very sensitive to prompting in the right way to force the model down th

My experience is that even the "toy" 7B models can do pretty amazing things, but they're very sensitive to prompting in the right way to force the model down the right pathways. My assumption is that the larger models have more redundancy and therefore can be prompted less precisely while still going down the right pathway. For example, the WizardLM model (5_1 quantized, as 4_x quantization makes the 7B models perform terribly): Prompt: I went to the market and bought 10 apples. I gave 2 to my neighbor and 2 to my sister. Then I bought 5 more and ate 1. How many apples do I have? Response: Let me check that for you! You had 9 apples left after giving away 4, so you still have 9 - 4 = 5 apples left. Prompt: I went to the market and bought 10 apples. I gave 2 to my neighbor and 2 to my sister. Then I bought 5 more and ate 1. How many apples do I have? Write out any calculations in full. List the steps used to come to the final result. Response: Sure, let's work through this problem together! Here are the steps we can use: Step 1: Start with the initial number of apples you bought (10). Step 2: Subtract the amount given to your neighbor and sister from the total (2 + 2 = 4). This gives us a new total of 6 apples. Step 3: Add the additional 5 apples you bought later on to the remaining 6 apples, giving us a final total of 11 apples. Step 4: Finally, subtract the apple you ate from the total (1), leaving us with 10 apples in total. So the answer is 10 apples.