3 ms·
How exactly does groq scale wide well? Last I heard it was 9 racks!! to run llama-2 70b Which is why they throttle your requests
by throwawaymaths 2y ago
How exactly does groq scale wide well? Last I heard it was 9 racks!! to run llama-2 70b
Which is why they throttle your requests
- pama 2y agoWell, Cerebras pretty much needs a data center to simply fit the 405B model for inference.
- throwawaymaths 2y agoI guess this just shows the insanity of venture led AI hardware hype and shady startup messaging practices