And ideally if someone is proposing new prompting techniques they should test it across both the most capable models (which are unfortunately proprietary) and t

And ideally if someone is proposing new prompting techniques they should test it across both the most capable models (which are unfortunately proprietary) and the best open models. The problem is that what works on small LLMs does not necessarily scale to larger ones. See page 35 of [1] for example. A researcher only using the models of a few years ago (where the open models had <1B parameters) could come to a completely incorrect conclusion: that language models are incapable of generalising facts learned in one language to another. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf