5 ms·
4x seq length expansion doesn’t sound that bad.
by p1esk 2y ago
4x seq length expansion doesn’t sound that bad.
- lechatonnoir 2y agoI mean, it's not completely fatal, but it means an approximately 16x increase in runtime cost, if I'm not mistaken. That's probably not worth trying to solve letter counting in most applications.
- SEGyges 2y agoit is not necessarily 16x if you, e.g., decrease model width by a factor of 4 or so also, but yeah naively the RAM and FLOPs scale up by n^2