3 ms·
What is Groq (rate limited) missing that you aren't?
by handfuloflight 2y ago
What is Groq (rate limited) missing that you aren't?
- DiederikVink 2y agoThat's a great question, but its hard to get enough insight into how Groq is serving models to properly know what's missing. If I had to hazard a guess, it would be that their system architecture (# of chips and chip architecture itself) might not be designed for a high concurrency situation.