2 ms·
I've been experimenting with something similar to this on top of SQLite. My experiments so far have involved storing the embeddings as binary strings represent
by simonw 4y ago
I've been experimenting with something similar to this on top of SQLite.
My experiments so far have involved storing the embeddings as binary strings representing the floating point arrays in a SQLite blob column: openai-to-sqlite is my tool for populating those: https://datasette.io/tools/openai-to-sqlite https://datasette.io/tools/openai-to-sqlite
I then query them using an in-memory FAISS vector search index using my datasette-faiss plugin: https://datasette.io/plugins/datasette-faiss https://datasette.io/plugins/datasette-faiss
- deleted 4y ago[deleted]
- xrd 4y agoAwesome, I came here to ask about SQLite and you already answered it! I love your writing, Simon. Have you blogged about this yet?
- simonw 4y agoYeah I wrote these up in https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer... a few weeks ago.