3 ms·
Yeah, and this is basically what I was asking. 100 tokens/s on the user's end, on a host that is batching requests, is very impressive.
by brucethemoose2 3y ago
Yeah, and this is basically what I was asking.
100 tokens/s on the user's end, on a host that is batching requests, is very impressive.