4 ms·

“We find that newspapers from larger schools, located in wealthier, educated, and urban ZIP codes are more likely to be classified as high quality. We then demo

by jdkee 5y ago

“We find that newspapers from larger schools, located in wealthier, educated, and urban ZIP codes are more likely to be classified as high quality. We then demonstrate that the filter's measurement of quality is unaligned with other sensible metrics, such as factuality or literary acclaim. We argue that privileging any corpus as high quality entails a language ideology, and more care is needed to construct training corpora for language models, with better transparency and justification for the inclusion or exclusion of various texts.” So better grammar and use of language?

TimTheTinker 5y ago
Right, invoking "ideology" seems a bit unnecessary and politically motivated. "More highly educated people produce better content." In other news, poor neighborhoods have more violent crime than rich neighborhoods... > We then demonstrate that the filter's measurement of quality is unaligned with other sensible metrics, such as factuality or literary acclaim. So the filter can't check whether something is scientifically accurate or artistically appealing. That makes sense. Of course it can't.
- SamoyedFurFluff 5y ago
  I’m kind of confused by the literary acclaim bit. If wealthy people produced better content why isn’t the best written content of all, pinnacles of literary acclaim (often acclaimed by the same wealthy population) within this cohort? If it’s because wealthy populations write better then why isn’t the pinnacle of writing within this cohort? Intuitively this seems non obvious to me. What am I missing?
  - giraffe_lady 5y ago
    It's just measuring adherence to a particular standard of written english. To use it to determine whether one work is "better" or not, you need to make a value judgement about which written standard or dialect you want to consider the target and compare to. Choosing that target is the ideology mentioned in an earlier comment: holding one dialect as superior or more fit to a purpose is ideological in a pure technical sense. You could also choose a different standard to get different results, for example imagine telling it that this is the goal for "best" https://www.bbc.com/pidgin https://www.bbc.com/pidgin Anyway since it mentioned using wikipedia as a source for "good" english, it makes sense to me to intuit it that way. Prose on wikipedia is detailed and held to a very specific standard, which is legible to the filter. Prose on wikipedia is also not usually going to win literary awards though, since that depends on more than careful adherence to a standard.
lurgburg 5y ago
"Better" is just presupposing the conclusion: that there is something inherently superior about certain styles. That thinking is precisely the "language ideology" the authors are concerned about! Better for who? For what purpose?
claudiawerner 5y ago
Dialect variation, English second-language speakers, etc. may be classified as better or worse (though this is controversial and I'm not entirely convinced for the dialect variation case), but that doesn't mean that they don't have something important to say. Some of the most productive conversations I've had are with people with far less mastery of English than native speakers, and even people whose grammar and spelling are poor.

by jdkee 5y ago

TimTheTinker 5y ago

Right, invoking "ideology" seems a bit unnecessary and politically motivated. "More highly educated people produce better content." In other news, poor neighborhoods have more violent crime than rich neighborhoods... > We then demonstrate that the filter's measurement of quality is unaligned with other sensible metrics, such as factuality or literary acclaim. So the filter can't check whether something is scientifically accurate or artistically appealing. That makes sense. Of course it can't.

SamoyedFurFluff 5y ago
I’m kind of confused by the literary acclaim bit. If wealthy people produced better content why isn’t the best written content of all, pinnacles of literary acclaim (often acclaimed by the same wealthy population) within this cohort? If it’s because wealthy populations write better then why isn’t the pinnacle of writing within this cohort? Intuitively this seems non obvious to me. What am I missing?
- giraffe_lady 5y ago
  It's just measuring adherence to a particular standard of written english. To use it to determine whether one work is "better" or not, you need to make a value judgement about which written standard or dialect you want to consider the target and compare to. Choosing that target is the ideology mentioned in an earlier comment: holding one dialect as superior or more fit to a purpose is ideological in a pure technical sense. You could also choose a different standard to get different results, for example imagine telling it that this is the goal for "best" https://www.bbc.com/pidgin https://www.bbc.com/pidgin Anyway since it mentioned using wikipedia as a source for "good" english, it makes sense to me to intuit it that way. Prose on wikipedia is detailed and held to a very specific standard, which is legible to the filter. Prose on wikipedia is also not usually going to win literary awards though, since that depends on more than careful adherence to a standard.

lurgburg 5y ago

"Better" is just presupposing the conclusion: that there is something inherently superior about certain styles. That thinking is precisely the "language ideology" the authors are concerned about! Better for who? For what purpose?

claudiawerner 5y ago

Dialect variation, English second-language speakers, etc. may be classified as better or worse (though this is controversial and I'm not entirely convinced for the dialect variation case), but that doesn't mean that they don't have something important to say. Some of the most productive conversations I've had are with people with far less mastery of English than native speakers, and even people whose grammar and spelling are poor.