4 ms·
I've covered this before in articles such as this: https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/bu
by dmezzetti 2y ago
I've covered this before in articles such as this: https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/building-an-efficient-sparse-keyw...
You can make anything performant if you know the right buttons to push. While Rust makes it easy in some ways, Rust is also a difficult language to develop with for many developers. There is a tradeoff.
I'd also say LangChain's primary goal isn't performance it's convenience and functionality coverage.
- timonv 2y agoCool, that's a fun read! I recently added sparse vector support to fastembed-rs, with Splade, not bm-25. Still, would be nice to compare the two.