6 ms·
Hacker News' Reading Level
- steveklabnik 16y agoIt would be equally interesting to only see the reading level of the articles, rather than the comments.
- iwwr 16y agoAn elevated reading level is not necessarily a sign of more thoughtful or insightful comments. It could just be contrived banality disguised as conceptual depth.
- olalonde 16y ago> An elevated reading level is not necessarily a sign of more thoughtful or insightful comments. It could just be contrived banality disguised as conceptual depth. Put simply, high reading level doesn't mean good content. (trying to prove your point :D)
- j_baker 16y ago"contrived banality disguised as conceptual depth" Is it possible this phrase is an example of itself? :-)
- mbm 16y agoIrony: (n) - incongruity between what might be expected and what actually occurs
- mahmud 16y agoI am sure it was indented as such.
- astrofinch 16y agoWords that describe themselves: short, convoluted, awkward, multilingual.
- matwood 16y agoThis is a great comment. I've always thought 'reading level' as a bit bogus and looking at the wrong thing. When determining the quality of content I look at it from two ways. Does content convey the idea the writer intended and does the reader comprehend the idea the writer intended. If I'm reading something I would prefer it be written as simply as possible. From a writers standpoint though this can be much harder than just explaining an idea in complex terms. To paraphrase Pascal, "I apologize for the length of this letter, but I did not have time to make it shorter." I like to think length in this case meant complexity.
- lionhearted 16y agoI like words, so I know a lot of words. But I try to take the wordiness way down. Why lose people who don't know the same words? I mean, even if you don't care about losing people who don't have time to learn a more precise-but-obscure synonym, there's always people who learned English as a third language. Why alienate them just to look a little clever? Even this comment has too many big words. If I had more time, I'd edit it down more. "alienate" would become "turn off" and "precise-but-obscure synonym" would become... I dunno, something simpler. Big obscure words that lock people out is really missing the forest from the trees.
- jiganti 16y agoYou're only considering the negative aspect of "big words". Sure, there are many instances in which peope unnecessarily use lesser-known synonyms, but it is helpful when people utilize a vocabulary in order to be concise. I think that while the current mantra of "don't use big words so everyone can understand" is called for because of the amount of people abusing vocabulary so others will consider them intelligent, we should remain cognizant of the benefits succinct communication provides.
- billswift 16y agoAnd it is even more important for precision, when a less common word is closer to what you want to say than a more popular one. This is one point Robert Heinlein made in Expanded Universe when he was defending Doc Smith's "space operas" and the vocabulary that was used in them. (Note that since "the map is not the territory", nothing, not your thoughts or even a physical description, can be exactly captured in words, but that doesn't mean it is not sometimes important or at least useful to try to be as precise as possible.)
- achille 16y agoCan someone briefly explain how google determines reading level? I'm assuming it's using something like the Flesch–Kincaid test: http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_test http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabil... A brief search on wikipedia reveals a few readability tests, but they all seem to be based on sentence/syllable ratios, not content complexity. http://en.wikipedia.org/wiki/Category:Readability_tests http://en.wikipedia.org/wiki/Category:Readability_tests And in general they all rank multi-syllable (longer) words higher. Which would mean a conversation between two Java API writers would be ranked higher than a ruby conversation :) Java vs Ruby vs Lisp: http://i.imgur.com/tq3pA.png http://i.imgur.com/tq3pA.png
- aeurielesn 16y ago> I'm assuming it's using something like the Flesch–Kincaid test Yes, It seems to be since it is only available for content in english. Although, as far as I know they have not said anything regarding the algorithm used yet. You can see (what it seems to be) the official announcement here: http://www.google.com/support/forum/p/Web%20Search/thread?tid=60e0869f5f9fad86&hl=en http://www.google.com/support/forum/p/Web%20Search/thread?ti...
- derefr 16y agoI figured that, since Google must read every word of every page to spider it, they must thereby have, as a byproduct, the world's most accurate database of word usage frequencies. "Reading level" would then just be a measure of the average frequency of all the words on a page (thus making words learned in a first year ESL class simple, and technical jargon advanced—quite the same as the measure of difficulty used by language proficiency exams.) The fact that the average of Simple English Wikipedia articles seems to be more intermediate than basic, though (29/52/17), would argue against that—unless the calculations are being biased by all the very infrequent proper nouns.
- nl 16y agoI'd be lying if I said I don't doubt you are not incorrect ;) What you are proposing is a statistically generated version of the Gunning Fog Index (http://en.wikipedia.org/wiki/Gunning_fog_index http://en.wikipedia.org/wiki/Gunning_fog_index) or the Flesch–Kincaid test (http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_test http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabil...). If I were Google, I'd try that, but I'd also try something like working out percentage deviation from a Markov chain generated from their crawl. A method like that would show that my first sentence is pretty unreadable, while an algorithm based on word complexity would see it as pretty simple.
- deleted 16y ago[deleted]
- cmelbye 16y agoI like how the result for "noobcomments" is marked as a Basic Reading Level.
- deleted 16y ago[deleted]
- jpwagner 16y agoyeah, but "best" is 100% intermediate
- sp332 16y agoAs simple as possible, but not simpler :-)
- dheerosaur 16y agoThe first thing I noticed was "noobstories" being marked Advanced. It is probably because noobs are trying their best to get karma by submitting some advanced stories.
- Mithrandir 16y agohttps://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Areddit.com https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... and https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Aen.wikipedia.org%2Fwiki%2F https://encrypted.google.com/search?hl=en&tbs=rl%3A1&...
- quizbiz 16y ago^ is comparing Reddit with Wikipedia. Results are not shocking.
- dantle 16y agohttps://encrypted.google.com/search?hl=en&safe=off&tbs=rl%3A1&q=site%3Asimple.wikipedia.org%2Fwiki%2F& https://encrypted.google.com/search?hl=en&safe=off&t...; So much for Simple English wikipedia!
- G_Wen 16y agoInteresting, perhaps in their effort to avoid short scientific terms they have substituted in longer more common words. For example describing a tumor as "an abnormal new mass of tissue that serves no purpose". http://wordnetweb.princeton.edu/perl/webwn?s=tumor http://wordnetweb.princeton.edu/perl/webwn?s=tumor The seemingly paradoxical result in an increase in understandability but a decrease in usability. Think of it as reading a passage where almost every noun has been substituted by a dictionary definition.
- rchowe 16y agoI suspect this is because the algorithm measures word use and not grammar. HN tends to have people who follow the "simple is better" mantra and who write in a tone similar to either an executive summary or an online dialog, thus an accessible reading level.
- redthrowaway 16y agoWell, it's official: 4chan is smarter than us. http://www.google.com/search?q=site:news.ycombinator.com&hl=en&tbs=rl:1#sclient=psy&hl=en&tbs=rl:1&q=site:4chan.org&aq=f&aqi=&aql=&oq=&gs_rfai=&pbx=1&fp=1bde53b2ade8e603 http://www.google.com/search?q=site:news.ycombinator.com&...
- olalonde 16y agoMaybe Google's algorithm categorizes made up words as intermediate? :D
- citricsquid 16y agoWell 4chan.org is simply the homepage, if you link to the actual boards (specifically /b/): https://encrypted.google.com/search?hl=en&tbs=rl:1&q=site:boards.4chan.org/b/ https://encrypted.google.com/search?hl=en&tbs=rl:1&q...
- redthrowaway 16y agoYou, sir, have made my day.
- citricsquid 16y agoThere's hope for us yet!
- olalonde 16y agoYou guys are not really helping.
- redthrowaway 16y agoLOL XD
- deleted 16y ago[deleted]
- cruise02 16y agoI know it's comparing apples to oranges, but I contribute a lot on Stack Overflow so I took a look at the results. https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Astackoverflow.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... I was expecting a higher proportion in the "advanced" category. For comparison: Math Overflow: https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Amathoverflow.net&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... OnStartups: https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Aanswers.onstartups.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... English Language & Usage: https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Aenglish.stackexchange.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... CS Theory: https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Acstheory.stackexchange.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... Seasoned Advice (Cooking): https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Acooking.stackexchange.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... Physics: https://encrypted.google.com/search?hl=en&tbs=rl%3A1&q=site%3Aphysics.stackexchange.com&btnG=Search&aq=f&aqi=&aql=&oq=&gs_rfai= https://encrypted.google.com/search?hl=en&tbs=rl%3A1&... It's no surprise really, but it seems like the more technical jargon used on a site, the higher the reading level Google assigns.
- duck 16y agoAccording to Google a HN thread about vomiting is "advanced". http://news.ycombinator.com/item?id=315490 http://news.ycombinator.com/item?id=315490
- lpolovets 16y agoIt would be cool to be able to compare two sites on one page, as with Google Trends. A few examples that I thought were neat: - msnbc (44/55/1) vs bbc.co.uk (15/82/2) - facebook (40/37/22) vs linkedin (2/90/6) (I wondering why facebook has so much "advanced" content according to Google) - wordpress (35/47/16) vs xanga (76/23/1) - boston college (5/41/53) vs harvard (2/6/91)
- Alex3917 16y agoAt the risk of stating the obvious, you can get the results for the reading level of your own HN comments by adding your user name. It's kind of interesting going through the leader board and looking up different people's scores.
- aaronblohowiak 16y agogo to ltu for advanced reading, here for startups.
- deleted 16y ago[deleted]
- carucez 16y agoI got to thinking, how would America's universities compare? It's an interesting thought: The best universities should have the most advanced reading level. I've published my findings, a complete ranking, and source code: http://log.largevoid.com/2010/12/ranking-colleges-by-reading-level/ http://log.largevoid.com/2010/12/ranking-colleges-by-reading...
- harshpotatoes 16y agoI think it's worth noting that most of the articles submitted to HN have no comments on them, meaning if those pages without comments were indexed by google and included in their measure of our reading level, there would be many pages consisting of only the words: "flag, 1 point by xxxx, no comments, Hacker News, new, threads..." etc etc. So maybe that has some influence on google's calculation.
- fluidcruft 16y agoRedditors will flip their buckets when they see: http://www.google.com/search?q=site:reddit.com&hl=en&num=10&lr=&ft=i&cr=&safe=images&tbs=rl:1 http://www.google.com/search?q=site:reddit.com&hl=en&... http://www.google.com/search?q=site:digg.com&hl=en&num=10&lr=&ft=i&cr=&safe=images&tbs=rl:1 http://www.google.com/search?q=site:digg.com&hl=en&n...