2 ms·
Implementations like these are incredibly useful for understanding how something like HNSW works, and being able to make derivative work. I put together a tiny
by jasonjmcghee 2y ago
Implementations like these are incredibly useful for understanding how something like HNSW works, and being able to make derivative work.
I put together a tiny little implementation a while ago, the key thing being, it writes the index as a few parquet files, so you can host the index on a CDN and read from it via http range requests (e.g. via duckdb wasm).
Definitely isn't beating any benchmarks, but free (or wildly cheap) to host, as you serve it directly from a CDN and processing is done locally.
https://github.com/jasonjmcghee/portable-hnsw https://github.com/jasonjmcghee/portable-hnsw
- kamranjon 2y agoThis is incredibly cool, I’m surprised more people haven’t contributed to this - it seems like it’s only a few optimizations away from being performant enough for a pretty broad set of use cases.
- jasonjmcghee 2y agoAppreciate the kind words! Obviously feel free to hack on it. When I first built it, I spent some time trying to tackle the issue of needing to update the entire file (and create an invalidation) if you want to update the database, which might be fine, but closes a lot of doors. I kind of hit a wall on finding a convincing approach to solving it, given the constraints of the setup.