Model sizes are adjusted to reach a given quality. Many papers justified empirically the use of more neurons allocated to language than to visual understanding,

Model sizes are adjusted to reach a given quality. Many papers justified empirically the use of more neurons allocated to language than to visual understanding, like Imagen[0] (figure 4.a). as it yields better results. Debatably, it is also true in the human brain. To me, it could highlight that there is more entropy in language than in images. After all, games have a similar property: some games, like noughts and crosses, or checkers, can have the same model architecture reach optimal play with smaller sizes than are necessary for chess and go[1]. It is certainly true that language has a lot of rules (grammatical, vocabular, syntactic) which are necessary to master it, but irrelevant to having a mental model of a scene; it just sequences a state of things into a stream of symbols. Maybe an additional entropy to learn in language which is absent in images includes motion and emotion. [0]: https://arxiv.org/pdf/2205.11487.pdf https://arxiv.org/pdf/2205.11487.pdf [1]: https://arxiv.org/pdf/2104.03113.pdf https://arxiv.org/pdf/2104.03113.pdf