4 ms·
You’re basically reinventing embeddings. Using an LLM for this is overkill since capturing the meaning of a sentence can be done with a model 1000x smaller. I
by serjester 2y ago
You’re basically reinventing embeddings. Using an LLM for this is overkill since capturing the meaning of a sentence can be done with a model 1000x smaller.
I have a chunking library that does something similar to this, and there’s actually quite a few different libraries that have implemented some variant of “semantic chunking”.
[1] https://github.com/Filimoa/open-parse https://github.com/Filimoa/open-parse
- nostrebored 2y agoExcellent, now show me a single one working in production with various docs. I’ve talked through this problem with dozens of startups working on RAG, and everyone has the same problem. LLMs are arguably not reinventing embeddings if you’re using them to infer the structure of a broad document. Understanding the “external structure” of the document, the internal structure of each segment, and the semantic structure of each chunk is important.
- serjester 2y agoSee my other comment on this thread - larger and larger context windows make perfect chunking irrelevant. We’ve ingested millions of docs and have a product in production, using LLM’s for chunking would have 10Xed our ingestion costs for marginal performance improvement.
- J_Shelby_J 2y agoGood to be validated. I guess I'll publish a blog post on the topic when I finalize what I've been working on.