14 ms·
Hey, no problem on the language. Always happy for any thoughts :) We will open source next week. A couple of corrections: I never meant to say our method was s
by nutanc 2y ago
Hey, no problem on the language. Always happy for any thoughts :) We will open source next week.
A couple of corrections: I never meant to say our method was superior. We have not done any benchmarks etc enough to write a paper. We have just been using this approach in our RAG pipeline and we have been happy with it. Sorry, will add a disclaimer in the post.
Agreed. For semantic chunking, only read and compare works best. Not sure how to scale that to a bench mark. Guess for the PG essay, only PG can answer if the topics align with his thoughts :)[https://x.com/nutanc/status/1838813258972549549 https://x.com/nutanc/status/1838813258972549549]
We will release the algorithm. I just put this out on Hackernews thinking this will also not be noticed as my all other posts :)
But looks like some interest is there. Will post a follow up update soon. Sorry for the lack of details on the original post.