4 ms·
Hooray for containerized models, but a game changing feature for such containers could be K-V/prefix caching. Previous discussion: https://news.ycombinator.com
by waldrews 2y ago
Hooray for containerized models, but a game changing feature for such containers could be K-V/prefix caching.
Previous discussion:
https://news.ycombinator.com/item?id=40034972#40036309 https://news.ycombinator.com/item?id=40034972#40036309