4 ms·
I foresee this being the game changer that will see more companies selfhosting their own AI applications. If you can serve an entire office with just one or two
by LorenDB 3y ago
I foresee this being the game changer that will see more companies selfhosting their own AI applications. If you can serve an entire office with just one or two LPUs, you're kinda stupid to not do it.
- zachbee 3y agoYou need an entire rack or two worth of Groq systems to get this performance, because the only memory the system has is on-chip SRAM (220MB). If you're trying to run on a small number of systems, you need local DRAM or HBM, which Groq doesn't have.