I had some success with simple aliasing at the beginning and end of chunks. In my next project, I'll try an idea that I saw somewhere: 1. do naive chunking lik

I had some success with simple aliasing at the beginning and end of chunks. In my next project, I'll try an idea that I saw somewhere: 1. do naive chunking like before 2. calculate the embeddings of each chunk 3. clusterize the chunks by their embeddings to see which chunks actually bring new information to the corpus 4. summarize similar chunks into smaller chunks Sounds like a smart way of using embeddings to reduce the amount of context misses. I'm not sure it works well, though :)