3 ms·
A single weight can’t encode an individual word, but the ratio looks close to overfitting too me too.
by tgv 5y ago
A single weight can’t encode an individual word, but the ratio looks close to overfitting too me too.
- petters 5y agoIf 16-bit floating-point numbers are used, it can presumably encode all tokens. In theory. It would not be very easy to work with.
- inciampati 5y agoI've often wondered if a lighter reinforcement learning based model on top of a full text index might do as well or better than these putatively overfit language models. Curious if anyone knows of ongoing or recent work on this approach.