3 ms·
Sounds like a ‘mipmap’ sort of approach could be useful. Add the whole document, then the chapters, then paragraphs. How does the system know how much context
by taneq 3y ago
Sounds like a ‘mipmap’ sort of approach could be useful. Add the whole document, then the chapters, then paragraphs.
How does the system know how much context is required, though? If I ask “Steve wears turtlenecks, what company does Steve work at?” then a paragraph about a Steve who likes turtlenecks and a paragraph about a Steve who works at MS would both be highly relevant but when combined they would result in the wrong answer.
- didgeoridoo 3y agoThe boring answer is to bash domain expertise and evals together until your data pipeline seems to be spitting out embeddings that give you acceptable results for whatever you’re trying to do. I do wonder if there is some additional cleverness that’s possible with fast, long-context LLMs, maybe a summarizing/preprocessing step that can slice up the document into a sensible hierarchy of topics that can be used as the embeddings source. Never seen this in practice though.