3 ms·

Interesting indeed. Isn’t a machine learning model kind of an overkill for this kind of data analysis? Wouldn’t linear regressions suffice in this instance? Don

Interesting indeed. Isn’t a machine learning model kind of an overkill for this kind of data analysis? Wouldn’t linear regressions suffice in this instance? Don’t you risk introducing unknown biases or even overfitting by using such a powerful technique when a simpler one is available? This kind of reminds me of when some journalist on twitter used machine learning to find hate-speech against British politician. Of course the journalist found exactly what they were looking for. There was more hate speech against conservatives... Of course, the journalist was only an amateur in statistics, and used a flawed model, that could have yielded anything they wanted disclaimer: I didn’t read the article because paywall, and waiting for an archive to be posted.

nonethewiser 4y ago
Do you have a link to the research that found conservatives were targeted with hate speech more frequently?
- runarberg 4y ago
  It took me a while, but I found it News source: https://www.bbc.com/news/uk-63330885 https://www.bbc.com/news/uk-63330885 Twitter thread: https://twitter.com/petesherlock79/status/1590253174845153280?s=20&t=LauWoQPTCqGGNaX7S9xEWA https://twitter.com/petesherlock79/status/159025317484515328... And a response thread explaining the flaws in the methodology: As news source: https://novaramedia.com/2022/11/09/the-bbcs-investigation-into-online-abuse-is-total-bullshit/ https://novaramedia.com/2022/11/09/the-bbcs-investigation-in... And on twitter: https://twitter.com/AyoCaesar/status/1590403667974582272 https://twitter.com/AyoCaesar/status/1590403667974582272 Basically the model was developed at google to find “toxic speech”, but the researchers had a very biased definition on what constituted as “toxic”. Racist speech didn’t count, while angry shouts did.
  - nonethewiser 4y ago
    I would expect there is more racist speech against white people than anyone else. Regardless, race doesn’t put someone in a particular political group so I don’t see how it debunks anything. Did the critics measure racist speech?
    - runarberg 4y ago
      Short answer: Yes, they tested the model with racist speech, and it failed. > I entered a selection of racist tweets I’d received in the past year into Perspective’s API, of varying lengths and sentence complexity. Just to make it easy for the machine, I deliberately chose one which included a common racial slur against South Asians. > None of these were registered as potentially toxic at all by the AI – but, “You’re a fucking G”, a compliment, popped up with a 90.29% likelihood of being toxic. Your hypotheses that white MPs experience more racist speech than non-white MPs, is testable. I have a pretty strong feeling that you are wrong, but I haven’t seen any data to support either (only anecdotes; and this deeply flawed analysis). What you can do to test your hypotheses is find all British MPs on twitter, scrape all tweets directed at them, and measure the proportion of racist speech against them. However, I advice you to do a more traditional cluster analysis, rather than a training model, the latter is very likely to yield a flawed model just like the journalist’s “8 month labor of love”.
bee_rider 4y ago
You can’t write an article about just scrolling through twitter, but if you pile up enough linear algebra it becomes an Officially Interesting Thing to report on.

nonethewiser 4y ago

Do you have a link to the research that found conservatives were targeted with hate speech more frequently?

runarberg 4y ago
It took me a while, but I found it News source: https://www.bbc.com/news/uk-63330885 https://www.bbc.com/news/uk-63330885 Twitter thread: https://twitter.com/petesherlock79/status/1590253174845153280?s=20&t=LauWoQPTCqGGNaX7S9xEWA https://twitter.com/petesherlock79/status/159025317484515328... And a response thread explaining the flaws in the methodology: As news source: https://novaramedia.com/2022/11/09/the-bbcs-investigation-into-online-abuse-is-total-bullshit/ https://novaramedia.com/2022/11/09/the-bbcs-investigation-in... And on twitter: https://twitter.com/AyoCaesar/status/1590403667974582272 https://twitter.com/AyoCaesar/status/1590403667974582272 Basically the model was developed at google to find “toxic speech”, but the researchers had a very biased definition on what constituted as “toxic”. Racist speech didn’t count, while angry shouts did.
- nonethewiser 4y ago
  I would expect there is more racist speech against white people than anyone else. Regardless, race doesn’t put someone in a particular political group so I don’t see how it debunks anything. Did the critics measure racist speech?
  - runarberg 4y ago
    Short answer: Yes, they tested the model with racist speech, and it failed. > I entered a selection of racist tweets I’d received in the past year into Perspective’s API, of varying lengths and sentence complexity. Just to make it easy for the machine, I deliberately chose one which included a common racial slur against South Asians. > None of these were registered as potentially toxic at all by the AI – but, “You’re a fucking G”, a compliment, popped up with a 90.29% likelihood of being toxic. Your hypotheses that white MPs experience more racist speech than non-white MPs, is testable. I have a pretty strong feeling that you are wrong, but I haven’t seen any data to support either (only anecdotes; and this deeply flawed analysis). What you can do to test your hypotheses is find all British MPs on twitter, scrape all tweets directed at them, and measure the proportion of racist speech against them. However, I advice you to do a more traditional cluster analysis, rather than a training model, the latter is very likely to yield a flawed model just like the journalist’s “8 month labor of love”.

bee_rider 4y ago

You can’t write an article about just scrolling through twitter, but if you pile up enough linear algebra it becomes an Officially Interesting Thing to report on.