3 ms·
I'm impressed by your chunking and retrieval strategies. I think this aspect is often overly simplistic. One aspect I don't quite understand is why you filter
by pstorm 3y ago
I'm impressed by your chunking and retrieval strategies. I think this aspect is often overly simplistic.
One aspect I don't quite understand is why you filter by the sliding window chunks vs just using the medium chunks? If I understand it correctly, you find the large chunks that contain the matched small chunks from the first retrieval. Then in the third retrieval, you are getting the medium chunks that comprise the large chunks? What extra value does that provide?
- joeyxiong 3y agoThank you for your comment. The sliding window approach allows me to dynamically identify relevant "large chunks," which can be thought of as sections in a document. Often, your questions may pertain to multiple such sections. Using only medium chunks for retrieval could result in sparse or fragmented information. The third retrieval focuses on "medium chunks" within these identified large chunks. This ensures that only the most relevant information is passed to the Language Model, enhancing both time efficiency and focus. For example, if you're asking for a paper summary, I can zero in on medium chunks within the Abstract, Introduction, and Conclusion sections, eliminating noise from other irrelevant sections. Additionally, this strategy helps manage token limitations, like GPT-3.5's 4000-token cap, by selectively retrieving information
- pstorm 3y agoAh I see! So, the large/sliding window chunks act as a pre-filter for the medium chunks. That makes a lot of sense. I appreciate the response