3 ms·
> They don’t care about content; only token count. That’s fine, but what would be best is to chunk by topic. I’ve had a lot of success with chunking documents
by Swizec 2y ago
> They don’t care about content; only token count. That’s fine, but what would be best is to chunk by topic.
I’ve had a lot of success with chunking documents by subtitle. Works especially well for web published documents because each section tends to be fairly short.