I read that but there is no good technique to rule out close duplicates. I know because I had tried to build one for my product. At best it relies on BLEU, embe

I read that but there is no good technique to rule out close duplicates. I know because I had tried to build one for my product. At best it relies on BLEU, embedding distance and other proxies which are far from ideal.