5 ms·
Running some code over War and Peace from Project Gutenberg gives these numbers for sentence length. I cleaned the text of non-alpha | {sentence terminators}.
by pnathan 16y ago
Running some code over War and Peace from Project Gutenberg gives these numbers for sentence length. I cleaned the text of non-alpha | {sentence terminators}.
Then I discarded any sentences with less than 4 characters under the assumption that they were spurious (ie, artifacts from the boilerplate PG text).
Text with artifacts assumption:
STDEV: 95.0115812751
MEAN: 74.4095505188
N: 26681
Text without artifacts assumption:
STDEV: 82.3010550235
MEAN: 76.425092356
N: 30805
Looking at the high point of the first standard deviation of the artifacts-removed text, we see that it is at 169 characters.
Same point for the text with artifacts, we have 158 characters.
So assuming that the translated War and Peace represents a reasonable sample of English, 160 characters is fairly close to the length of a sentence.
edit: formatting.
- true_religion 16y ago> So assuming that the translated War and Peace represents a reasonable sample of English, 160 characters is fairly close to the length of a sentence. I would say the mean is close to the length of a sentence, and the first high mark via standard deviation is a long sentence. So a standard SMS should be able to fit 2 or so normally worded sentences or 3 concisely worded sentences.
- pnathan 16y agoIMO post-Hemingway English has shorter sentences on average than it used to; I wouldn't mind targeting, say, Twilight with my code and getting the stats on it, but I am loath to steal Twilight just for personal amusement.