3 ms·
If request latency is your top concern and you don’t need it to be as smart as GPT-4, why are you using OpenAI and not a model hosted by Groq or similar?
by JimDabell 2y ago
If request latency is your top concern and you don’t need it to be as smart as GPT-4, why are you using OpenAI and not a model hosted by Groq or similar?
- mritchie712 2y agoWe need function calling, which wasn't great in Groq when we last tried it, but they seem to have made progress on it. We'll be testing it again soon.
- voiper1 2y agogroq just released: llama3-groq-70b-8192-tool-use-preview llama3-groq-8b-8192-tool-use-preview Sonnet3.5 also has tool usage, and is slightly cheaper than 4o ($3 input vs $5). If you _only_ need tool usage, then 4o-mini or others will be cheaper... mistal's Mixtral 8x22B has function calling for $2/$6, which is a bit cheaper than 4o.
- d13 2y agoGroq’s rate limits are so bad they’re only good enough for one person using it very rarely with an under 4k context size. It’s just a marketing stunt at the moment.
- fragmede 2y agohow much are you paying them?