2 ms·
Larger models need more hardware resources to run And, depending on effort settings, they do more 'thinking', i.e., use more rounds of inference to generate lo
by generallyjosh 7mo ago
Larger models need more hardware resources to run
And, depending on effort settings, they do more 'thinking', i.e., use more rounds of inference to generate longer internal chains of thought
Both very good reasons to prefer a smaller model, if the small model is good enough for the task