3 ms·

It doesn't really suggest anything. Neither model will even close to saturation and all else equal, bigger models perform better in every way, including general

• naasking 3y ago
  But why do bigger models perform better? Arguably because there's a larger state space that can be used to remember more contexts, which helps with both generalization and case-specific processing.
  • famouswaffles 3y ago
    >But why do bigger models perform better? No one really knows the answer to this question. >Arguably because there's a larger state space that can be used to remember more contexts, which helps with both generalization and case-specific processing. What I'm trying to say is that both models in either scenario are very over-parameterized and under-trained. You say the answer is extra space ? The smaller model has not used anywhere near the space it has. They both have extra space. It's like arguing a bigger drum is better because of extra space when all the water you plan to store will not take even half of your smaller drum