2 ms·
Why would you want chunks that big for vector search? Wouldn't there be too much information in each chunk, making it harder to match a query to a concept withi
by rahimnathwani 2y ago
Why would you want chunks that big for vector search? Wouldn't there be too much information in each chunk, making it harder to match a query to a concept within the chunk?
- nostrebored 2y agoThe problem is that often semantic meaning depends on state multiple paragraphs or sections away. This is a coarse way to tackle that