3 ms·
I believe you when you say you're not changing the model file loaded onto the H100s or whatever, but there's something going on, beyond just being slower, when
by fragmede 9mo ago
I believe you when you say you're not changing the model file loaded onto the H100s or whatever, but there's something going on, beyond just being slower, when the GPUs are heavily loaded.
- clbrmbr 9mo agoI do wonder about reasoning effort.
- hauntsaninja 9mo agoReasoning effort is denominated in tokens, not time, so no difference beyond slowness at heavy load (I work at OpenAI)