3 ms·
It'd be interesting to see examples involving the "G" in RAG: I'm wary that having a fine-grained breakdown of, say, 2 vs. 5 sentences, would affect things sign
by refulgentis 2y ago
It'd be interesting to see examples involving the "G" in RAG: I'm wary that having a fine-grained breakdown of, say, 2 vs. 5 sentences, would affect things significantly.
If I'm understanding correctly, you end up with a "turtles all the way down" problem: you need to chunk using some size = K, in order to get the inputs you need to decide bespoke chunk sizes X, Y, and Z.
- bbor 2y agoYeah but this becomes very, very relevant for larger corpuses. It’s not just about pulling the right sized snippet in the end (really that could be done by an intermediate LLM call…), it’s also about supporting effective vector similarity searches to find the right snippet in the first place. If you’re embedding all five pages into a single vector, and only two of them are relevant to your query, that makes the search much less effective. AFAIU.
- refulgentis 2y agoExactly: you have to do all the small embeddings to get the larger embeddings. Also, in practice, embedding are much smaller than that, the popular MiniLM v2 is 256 words. Leaving me unsure how this helps