3 ms·
I feel like the author of this criticism is screaming into the void. I have no doubt that the study is probably flawed, and likely has not proven beyond doubt a
by undecisive 7y ago
I feel like the author of this criticism is screaming into the void. I have no doubt that the study is probably flawed, and likely has not proven beyond doubt any of the hypotheses it has put forward. While FK may not be a great test of linguistic complexity, it is a test.
To truly annul the study, the author needs a smoking gun - for example:
- Take a couple of the source texts that score wildly far apart
- Rework the punctuation such that an objective reader still agrees that the transcription is a valid representation of the original speech
- Show that the readability scores in the re-punctuated texts are inverted - or at the very least, score within a small percentage of each other.
It's a bit like trying to measure the happiness of a city by counting the percentage of smiling people as they walk down the high street. There are many social reasons a person might smile from the depths of misery, different people might classify a smile differently, and different cultures might view walking down the street with a grin as either frowned upon or mandated. Weather, time of day, month of year, public holidays - etc - all will affect the results.
But so long as you work hard to eliminate these biases, you still couldn't use such a measure to test whether Beijing is happier than Belfast, but you might use it as an indicator of whether left handed people are happier than right handed people. You can't tell whether men are happier than women (social pressures will influence facial indicators along gender lines from an early age in many cultures) but you might be able to begin a conversation on whether conservatives are happier than liberals. It won't be conclusive proof, but as a starting point for future research, it still holds some validity.
My point is that nothing in this criticism conclusively proves that this particular measure is unsuited for measuring against the stated outcomes. Even a measure that is the current darling of the linguistics community will have its drawbacks, and the author needs to prove that the weaknesses in the measure present a bias in and of themselves, not just that they are generally deficient.
To my mind, the better criticism is intent. A person can - and should - craft a speech based on the audience. A speech given at a university or research facility should have a different complexity score than one given at a rally or disaster zone. The interesting question to my mind is whether more effort is put into this by conservatives or liberals, and then comparing that to those in power vs their contenders.
But then, that paper wouldn't have such a catchy title, would it?
- deleted 7y ago[deleted]
- ykts 7y agoThe smoking gun you're looking for is in the Language Log posts linked from the blog post. In particular, https://languagelog.ldc.upenn.edu/nll/?p=21847 https://languagelog.ldc.upenn.edu/nll/?p=21847 shows how trivial changes to punctuation can inflate a paragraph's score from 4.4 to 12.5.