3 ms·
Excellent, now show me a single one working in production with various docs. I’ve talked through this problem with dozens of startups working on RAG, and every
by nostrebored 2y ago
Excellent, now show me a single one working in production with various docs.
I’ve talked through this problem with dozens of startups working on RAG, and everyone has the same problem.
LLMs are arguably not reinventing embeddings if you’re using them to infer the structure of a broad document. Understanding the “external structure” of the document, the internal structure of each segment, and the semantic structure of each chunk is important.
- serjester 2y agoSee my other comment on this thread - larger and larger context windows make perfect chunking irrelevant. We’ve ingested millions of docs and have a product in production, using LLM’s for chunking would have 10Xed our ingestion costs for marginal performance improvement.
- J_Shelby_J 2y agoGood to be validated. I guess I'll publish a blog post on the topic when I finalize what I've been working on.