3 ms·
The problem with that is LLM speed is mostly bottlenecked by memory bandwidth. Slower RAM means worse performance.
by ronsor 3y ago
The problem with that is LLM speed is mostly bottlenecked by memory bandwidth. Slower RAM means worse performance.