4 ms·
Expanding free access is mostly an inference cost Serving cheaply at that scale means routing, batching, and cache hits, not a better model :D
by OsamaJaber 2mo ago
Expanding free access is mostly an inference cost
Serving cheaply at that scale means routing, batching, and cache hits, not a better model :D