4 ms·
What's wrong with just using Torchserve[1]? We've been using it to serve embedding models in production. [1] https://pytorch.org/serve/ https://pytorch.org/ser
by rvrs 3y ago
What's wrong with just using Torchserve[1]? We've been using it to serve embedding models in production.
[1] https://pytorch.org/serve/ https://pytorch.org/serve/
- eigenvalue 3y agoI wanted something that works natively with llama-cpp and langchain that is also small and easy to hack on. And I also wanted everything to be seamlessly cached in SQlite. And also something that had built in semantic search with FAISS. And string similarity using multiple different similarity measures beyond just cosine similarity. And token-level embedding support. There is much more flexibility when you do it yourself to add whatever functionality you want quickly.