3 ms·
I'm not sure that's it. Presumably they want to keep the cache in GPU memory?
by danielmarkbruce 2y ago
I'm not sure that's it. Presumably they want to keep the cache in GPU memory?
- jncfhnb 2y agoThat’s largely it imo. If you get get the embedding representations you could just recreate the model logic and then it’s no longer closed source.
- danielmarkbruce 2y agono, you couldn't. If they just handed you the embedding layer it wouldn't help that much. Plus, I'm reasonably certain they are caching the attention scores anyway.