2 ms·
Why would increasing memory bandwidth reduce performance? You said "You can get a decent approximation for LLM performance in tokens/second by dividing the mode
by bawana 3y ago
Why would increasing memory bandwidth reduce performance? You said "You can get a decent approximation for LLM performance in tokens/second by dividing the model size in GB by the system's memory bandwidth"
- deleted 3y ago[deleted]
- AnthonyMouse 3y agoYeah the sentence is backwards, you divide the system's memory bandwidth by the size of the model.