2 ms·
Speed should be proportional to the number of active parameters, so all 7B Q4 models will have similar performance.
by wmf 2y ago
Speed should be proportional to the number of active parameters, so all 7B Q4 models will have similar performance.