I mean, from the structure of things that makes sense. If you think of it as a statistically-derived processor/algorithm, unless there's some great task of sho

I mean, from the structure of things that makes sense. If you think of it as a statistically-derived processor/algorithm, unless there's some great task of short-term memory/memorization it's forced to undergo it's going to be biased towards more direct responses. Another thing to consider, regarding the fine-tuning, is that "instruction" may be biased towards sentence ordering, so that the actual order with which you present your prompt can be important. For example Prompt: <detailed situational specification>. <task to be performed> might perform considerably worse than <task to be performed>. <detailed situational specification>. I suspect it's just a short-mid-term resource constraint though since: - demonstrating extremely sharp memory performance conversationally isn't that common - total bout / token width isn't particularly wide in training - presumably fewer "transistors" than humans -- basically even 200B parameters isn't all that many Another concern that I'm not sure how to consider is that 4-letter tokenization could be an obvious reason ChatGPT is such a poor speller.