3 ms·
How does 12 tokens/sec equate to satisfactorily severing thousands of end-customere? I did enjoy the headline, and the register has formerly been an incredibly
by metadat 2y ago
How does 12 tokens/sec equate to satisfactorily severing thousands of end-customere?
I did enjoy the headline, and the register has formerly been an incredibly good news outlet. Prior to the founder passing away last week.
- atherton33 2y agoFrom the article, it's not 12 total, but 12 per user for 100 concurrent generations.
- metadat 2y agoThank you atherton friend.
- washadjeffmad 2y agoCapacity doesn't equate to quality, but I could easily see an 8B finetune with exl2 at low context working for short, simple customer interactions, akin to oversubscribing a 1Gbit uplink for 100 customers at 50Mbps.