4 ms·
I think we can do better than 11 characters! I happen to have a corpus which includes pretty much every word ever written in a book, including many misspelled,
by neilk 3y ago
I think we can do better than 11 characters!
I happen to have a corpus which includes pretty much every word ever written in a book, including many misspelled, mistranscribed, or otherwise non-dictionary words.
After eliminating nonsense, non-English, or other mistakes, I think the real winner, coming it at 12 characters, is:
teetertotter
That's a relatively common word. Even though it's usually seen hyphenated, the unhyphenated form is recognized by all the online dictionaries I found.
----
And some other candidates, just for fun, in the 13 or 12 character range:
proproprietor
priorityqueue
reporterette
preprototype
"proproprietor" seems more like a misspelling. Should have a hyphen, or be two words.
"priorityqueue" is of course familiar to hackers here, but is more of a jargon term, and is only concatenated due to appearing in source code. Invariably it's two words when actually written out.
"reporterette" is antique, but appeared in a NYTimes headline as late as 2018 - the author reflected on her career, including sexist epithets. https://www.nytimes.com/2018/12/02/opinion/george-hw-bush-maureen-dowd.html https://www.nytimes.com/2018/12/02/opinion/george-hw-bush-ma...
"preprototype" is used exactly as is, in lots of scientific papers, up to the current day. That's a pretty good one too, and could be a tie for "teetertotter", but it's verging on jargon.
- deleted 3y ago[deleted]
- soultrees 3y agoHow did you scrape that data? How do you store and retrieve it? Is it just a standard db or a vector db? Sorry for the questions, but it seems like an interesting, yet probably common data set and as someone who is venturing down this path, I’d like to learn more about building my own dataset similar to this from scratch.
- neilk 3y ago> standard db or vector db lol, it's a 42MB text file from Google Books Ngrams. The format looks like this: $ head words-all.txt a 14219615690 a! 196012 a" 84 a' 47713 a'0 3036 a'1 4070 a'10 99 a'11 56 I queried it with perl and sort. $ time perl -wlane 'if ($F[0] =~ /^[qwertyuiop]+$/) { print length($F[0]), "\t", $F[0] }' words-all.txt | sort -rn > qwertywords real 0m1.915s user 0m1.896s sys 0m0.025s I can't remember exactly which file I downloaded, but according to my notes I got it from here back in 2012 or so. https://storage.googleapis.com/books/ngrams/books/datasetsv2.html https://storage.googleapis.com/books/ngrams/books/datasetsv2... There seems to be a newer corpus published in 2020: https://storage.googleapis.com/books/ngrams/books/datasetsv3.html https://storage.googleapis.com/books/ngrams/books/datasetsv3...