3 ms·
The content is pretty freeform-- one could be in html, could be in pdf, etc. To give a more concrete example, imagine having a million resumes (some in doc, so
by passioncurious 10y ago
The content is pretty freeform-- one could be in html, could be in pdf, etc.
To give a more concrete example, imagine having a million resumes (some in doc, some in pdf, some in html) and you want to dedupe them.
- yehosef 10y agoYou'll have to normalize them to text to make it work - the format is not important for the deduplication (meaning the same text in a word doc or pdf is really the same thing to you, even though at the byte level the files may be very different.)