4 ms·
You need terabytes of memory to run a frontier class model
by Cantinflas 2mo ago
You need terabytes of memory to run a frontier class model
- calgoo 2mo agoI wonder, if you can run at 8k or 15k t/s, you could in theory run 10 or 20 agents (or more) at the same time and generate hundreds of versions, then just analyze them. Think thinking mode x1000 at least... Would be interesting to see how good it would be
- est 2mo agoIf you look closely, 80% of those terabytes of memory is stationary model weights.