3 ms·
> time and generate possible questions from the LLM based on the context of the current semantically split chunk. Possible but very compute intensive. Imagine
by ekianjo 2y ago
> time and generate possible questions from the LLM based on the context of the current semantically split chunk.
Possible but very compute intensive. Imagine if you have hundreds of thousands of chunks...
- EngineeringStuf 2y agoThe number of chunks would be the same regardless of either approach. The generation of questions can be done out-of-band by a cheaper model. Their current implementation approach seems to require some computation per request. It would be a balance to see which strategy provides the most value. The speed of responses overall would be faster.