3 ms·

Retrieval Augmented Generation Based on SQLite

88 points by emzo 1y ago

deleted 1y ago
[deleted]
wredcoll 1y ago
This looks cool, and I'm interested in these keywords, but I read the entire readme and I'm still unsure what problem it's actually solving. Anyone want to help out?
- webstrand 1y ago
  This is for LLMs. In general RAG takes a user prompt and uses it to find potentially relevant documents in the database. It then enriches the original prompt with those documents so that the LLM has context that wasn't in its training dataset.
- Octplane 1y ago
  RAG -> Vector search -> means that your documents are not indexed as full text but as Vectorized objects which mean that then you can search using concepts instead of exacts strings you would use with a regular "Fulltext search". This makes the search less precise and more powerful at the same time (ie it could look clever to some extent).
- ncruces 1y ago
  This might provide context: https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/index.html https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/in...
almosthere 1y ago
Sqlite has an embedding search? Or is that being provided by this tool?
- Octplane 1y ago
  It's provided via https://github.com/asg017/sqlite-vec https://github.com/asg017/sqlite-vec
- ethan_smith 1y ago
  SQLite itself doesn't have native embedding search, but extensions like sqlite-vss and sqlite-vectorize add vector similarity search capabilities to SQLite.
rcarmo 1y ago
This and SQLite-vec (or whatever extension is trendy these days) can do a lot in a very limited amount of compute.
anoojb 1y ago
Would love to see a system that blends cheap lexical (Fulltext Search) or semantic/vector search using SQLite and chooses the best approach given the input.
- bob1029 1y ago
  If you want the best possible solution vertical for most business, I'd be looking at using Lucene for FTS duty. Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else: https://lucene.apache.org/core/8_0_0/highlighter/org/apache/lucene/search/highlight/Highlighter.html https://lucene.apache.org/core/8_0_0/highlighter/org/apache/... It is also very easy to customize this engine and align the document tokenization & indexing concerns with your specific retrieval scenarios.
deleted 1y ago
[deleted]

88 points by emzo 1y ago

deleted 1y ago

[deleted]

This looks cool, and I'm interested in these keywords, but I read the entire readme and I'm still unsure what problem it's actually solving. Anyone want to help out?

webstrand 1y ago
This is for LLMs. In general RAG takes a user prompt and uses it to find potentially relevant documents in the database. It then enriches the original prompt with those documents so that the LLM has context that wasn't in its training dataset.
Octplane 1y ago
RAG -> Vector search -> means that your documents are not indexed as full text but as Vectorized objects which mean that then you can search using concepts instead of exacts strings you would use with a regular "Fulltext search". This makes the search less precise and more powerful at the same time (ie it could look clever to some extent).
ncruces 1y ago
This might provide context: https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/index.html https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/in...

almosthere 1y ago

Sqlite has an embedding search? Or is that being provided by this tool?

Octplane 1y ago
It's provided via https://github.com/asg017/sqlite-vec https://github.com/asg017/sqlite-vec
ethan_smith 1y ago
SQLite itself doesn't have native embedding search, but extensions like sqlite-vss and sqlite-vectorize add vector similarity search capabilities to SQLite.

rcarmo 1y ago

This and SQLite-vec (or whatever extension is trendy these days) can do a lot in a very limited amount of compute.

anoojb 1y ago

Would love to see a system that blends cheap lexical (Fulltext Search) or semantic/vector search using SQLite and chooses the best approach given the input.

bob1029 1y ago
If you want the best possible solution vertical for most business, I'd be looking at using Lucene for FTS duty. Having the FTS engine provide a google-style snippet of the most relevant document chunk is the holy grail for RAG applications. Lucene does this kind of thing better than anyone else: https://lucene.apache.org/core/8_0_0/highlighter/org/apache/lucene/search/highlight/Highlighter.html https://lucene.apache.org/core/8_0_0/highlighter/org/apache/... It is also very easy to customize this engine and align the document tokenization & indexing concerns with your specific retrieval scenarios.

deleted 1y ago

[deleted]