3 ms·
Are you sure that kind of computation can be a general rule? Did you mean that the maximum rate it could be obtained is "bandwidth/size"?
by mdp2021 1y ago
Are you sure that kind of computation can be a general rule?
Did you mean that the maximum rate it could be obtained is "bandwidth/size"?
- deleted 1y ago[deleted]
- hnuser123456 1y agoYes, for most LLMs the transformer means the entire model and context is read from VRAM for every token.