4 ms·
nice project, but the model that was being studied is really just a toy-model (both in size and training data). as such, this model can indeed be approximated b
by matusp 3y ago
nice project, but the model that was being studied is really just a toy-model (both in size and training data). as such, this model can indeed be approximated by simpler models (I would suspect even n-gram LMs), but it might not be representative of how the larger LMs work.
- Closi 3y agoThis is probably true - i.e. you could make an even smaller model and then likely come up with an even-simpler explanation for how it worked.