3 ms·
what's the cost difference between groq/cerebras vs using something else for inferencing open source models? I'm guessing the speed comes at a cost?
by FezzikTheGiant 2y ago
what's the cost difference between groq/cerebras vs using something else for inferencing open source models? I'm guessing the speed comes at a cost?
- ilaksh 2y agoI don't know off the top of my head, only played with it a little not seriously.
- FezzikTheGiant 2y agofair enough
- el_isma 2y ago0.6/1$ per M tokens in groq/cerebras vs 0.3$ per M tokens in deepinfra (for llama 3.3 70b) But note the free tiers for groq and cerebras are very generous.