4 ms·
if the intuition is based on images taking more space to encode than text, then it's a false intuition because the size of models is not correlated to the encod
by muyuu 4y ago
if the intuition is based on images taking more space to encode than text, then it's a false intuition because the size of models is not correlated to the encoding size of the individual data points - it's correlated instead to the amount of relevant choices the model can make and the complexity of the dynamics in the model
I don't think general assertions like "language is more complicated" are congruent or meaningful, it really depends on what the model is trying to achieve; it's the complexity of that which will require a larger or smaller model