4 ms·
Yes, for most LLMs the transformer means the entire model and context is read from VRAM for every token.
by hnuser123456 1y ago
Yes, for most LLMs the transformer means the entire model and context is read from VRAM for every token.
loading thread…