4 ms·
Ask HN: Semantic Vector Searching in WASM?
I'm working on a simple markdown-driven CMS, in Python, that outputs to static HTML files that can be served directly from GitHub (as an example).
I'm interested in implementing local-only search of this content, w/o using a backend, and preferably w/o cobbling together some NLP algorithms in JavaScript.
I've also been training ML models for other use cases and experimenting around w/ vector databases.
I started looking into creating embeddings using Python, from my markdown content, potentially using word2vec or finalfusion to help me get those embeddings into a Rust environment that I can compile w/ Rust into WASM, and use cosine similarity for my search.
Now, I neither have any Rust nor WASM experience, but that doesn't really deter me much. I'm just curious if the overhead would be significant, or if I should skip the semantic embeddings and do a different type of embedding such as TD-IDF.
- cmcollier 3y agoAs a general rule (for now), you'll get the best search result for your dev time, with straight ahead BM25 via a js lib. In terms of overhead, with lower doc counts there's not much overhead with embeddings and knn/ann. Imagine 384 floats per doc or whatever embedding size. At scale it becomes more problematic and less comparable. With all that said, messing around with vector ops and WASM sounds more fun :)
- jzombie 3y agoThanks for the feedback!
- TachyonicBytes 3y agoWould this[1] library help you? It's a Rust vector similarity search engine, written to be compiled to Wasm. I discovered it through articles like these[2]. [1] https://github.com/tantaraio/voy [2] https://dawchihliou.github.io/articles/share-rust-types-with-typescript-for-webassembly-in-30-seconds
- jzombie 3y agoI will check them out! Thanks!
- dhruv_anand 3y agoNot sure if this helps, but you can create embeddings using transformers.js (feature extraction pipeline) in the browser.
- jzombie 3y agoPotentially! Part of my reasoning for doing this is to familiarize myself a bit with Rust/WASM, but if I get stuck, this could be an alternative.