Let me rephrase: > The fact that GPT3 works at all for English indicates that English is too low level and forces a lot of redundancy (low entropy). I don't t

Let me rephrase: > The fact that GPT3 works at all for English indicates that English is too low level and forces a lot of redundancy (low entropy). I don't think the goal is to compress information/language and maximize "surprise".