4 ms·
As a HN data processing note, to remove the garbled characters, you need to convert the smart quotes from HN (among other things like long dashes) into normal A
by minimaxir 11y ago
As a HN data processing note, to remove the garbled characters, you need to convert the smart quotes from HN (among other things like long dashes) into normal ASCII characters.
EDIT: Looks like the garbled characters were fixed.
- orf 11y agoYeah thanks for pointing that out, I took the nuclear approach and just ran all everything through unicodedata.normalize("NFC", ...) which seems to have done the trick.