3 ms·
From my perspective, it’s not clear why you would want to use bulk summarisation of all context versus summarisation over “relevant” vectors, since it is both s
by darkteflon 3y ago
From my perspective, it’s not clear why you would want to use bulk summarisation of all context versus summarisation over “relevant” vectors, since it is both substantially more expensive and less effective, since you are effectively polluting the context window with “irrelevant” context. And the problem is compounded as you scale up - even as you scale up trivially.
Admittedly I’m hand-waving a bit around “relevant” and “irrelevant” - clearly your vector search setup has to be fit for purpose. That’s a talent all on its own, so I wonder if we will see competing approaches at the vectorstore level or if it’s relatively settled. Anyway, I’m out of my depth at that point so I’ll leave it there.
- danielmarkbruce 3y agoI think it's unsettled and we'll see some clever things which combine approaches. On the surface it seems like preprocessing a corpus in clever ways will be useful.
- kordlessagain 3y agoIf we read a document, that's preprocessing it. It's useful for being able to discuss later, or bring that understanding to bear on a different, yet related problem space. I agree that a combined approach is likely useful.