3 ms·
User tokens onto the chip and output tokens out are tiny.
by cma 2y ago
User tokens onto the chip and output tokens out are tiny.
- twothreeone 2y agoNot if you're serving tens of thousands of users at the same time.
- cma 2y agoStill tiny at 100,000.