4 ms·
Kinda interesting ... inaccurate (put a contraction in the text box, it broke it into "don" and "t") ... but mildly interesting Punctuation is always a difficu
by warrenm 3y ago
Kinda interesting ... inaccurate (put a contraction in the text box, it broke it into "don" and "t") ... but mildly interesting
Punctuation is always a difficult corner case - dealt with various rules for managing them when I did assignments like this in college years ago
A quick pair of shell lines that replicate the site:
`tr [:punct:] " " < tmp.txt | tr [:space:] "\n" | sort -u`
`tr [:punct:] " " < tmp.txt | tr [:space:] "\n" | sort -u | wc`
This converts punctuation to spaces, then spaces to newlines, then does a unique sort (and a word count (with bytes, newlines, and individual words listed))