4 ms·
I'd be lying if I said I don't doubt you are not incorrect ;) What you are proposing is a statistically generated version of the Gunning Fog Index (http://en.w
by nl 16y ago
I'd be lying if I said I don't doubt you are not incorrect ;)
What you are proposing is a statistically generated version of the Gunning Fog Index (http://en.wikipedia.org/wiki/Gunning_fog_index http://en.wikipedia.org/wiki/Gunning_fog_index) or the Flesch–Kincaid test (http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_test http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabil...).
If I were Google, I'd try that, but I'd also try something like working out percentage deviation from a Markov chain generated from their crawl. A method like that would show that my first sentence is pretty unreadable, while an algorithm based on word complexity would see it as pretty simple.
- derefr 16y ago> If I were Google, I'd try that, but I'd also try something like working out percentage deviation from a Markov chain generated from their crawl. Indeed, that was my second thought, but I wonder if the gains are really all that large over a raw statistical analysis of the word bag, and whether they're worth the extra analysis space/time. It really depends on what Google is planning on doing with this metadata, internally; if an order-of-ten precision is fine (to pick out decisive categorizations), the raw analysis may be all that's needed.
- nl 16y agoWell we could always try it out. Here's 24GB (compressed) of ngram data from Google: http://googleresearch.blogspot.com/2006/08/all-our-n-gram-are-belong-to-you.html http://googleresearch.blogspot.com/2006/08/all-our-n-gram-ar...
- biotech 16y ago> I'd be lying if I said I don't doubt you are not incorrect Holy Shnikes, that's a tough one to parse! I wasn't sure what you were saying here, so I'm gonna break it down, working from the end of the sentence: 1. I'd be lying if I said I don't doubt you are not incorrect 2. I'd be lying if I said I don't doubt you are CORRECT 3. I'd be lying if I said I don't think you are INCORRECT 4. I'd be lying if I said I think you are CORRECT 5. I think you are INCORRECT The idea is that each of the previous statements are saying basically the same thing; I'm just cancelling negatives each time. Anyway, am I correct to assume that you think that the GP is incorrect?
- billswift 16y agoYour transition 2 -> 3 is not justifiable, "doubt" is not the opposite of "think". A better parsing leaves "doubt" alone and would end with: "I doubt you are correct."
- gimpf 16y agoMy dear, you just lost the "don't".
- oh_ryan 16y agoI'd be lying if I said I don't doubt you are not incorrect I'd be lying if I said I don't doubt you are correct I'd be lying if I said I think you are correct
- nl 16y agoI had to triple check to make sure I got it right, and I wrote it. Yes, I think the GP is incorrect.