4 ms·

Mozilla conducted this research using RegretsReporter, an open-source browser extension that converted thousands of YouTube users into YouTube watchdogs. Resea

by tpmx 5y ago

Mozilla conducted this research using RegretsReporter, an open-source browser extension that converted thousands of YouTube users into YouTube watchdogs. Research volunteers encountered a range of regrettable videos, reporting everything from COVID fear-mongering to political misinformation Like videos discussing the lab leak theory?

uniqueuid 5y ago
The lab leak may have been in there (you can probably check in the appendix to the report). But to be honest, it's pretty irrelevant. Content analysis is usually performed using the current state of knowledge. It is to be expected that known facts and what is perceived as truth changes after the fact. This does not invalidate the analysis itself. Here's the explanation from the report, which I find pretty reasonable: > Over the course of three months, the working group developed a conceptual framework for classifying some of the videos, based on YouTube’s Community Guidelines. The working group decided to use YouTube’s Community Guidelines to guide the qualitative analysis because it provides a useful taxonomy of problematic video content and also represents a commitment from YouTube as to what sort of content should be included on their platform.
cromwellian 5y ago
Isn't there a problem with the sampling here? What if people who likely install RegretsReporter are also people who are more likely to view tangentially related problematic content in the first place, and then are shown more of it. Also, the dividing line on what's consider problematic is difficult, as we saw recently with Right Wing Watch. And the proposed regulation seems even more problematic. If the AI model were public, then it would be far easier for people to create adversarial content to game it. This is a problem with any models built on public data, including OpenAI's GPT stuff, or GitHub's Copilot. Detailed knowledge of how it works allows more efficient adversarial attacks, and the more these services become important public infrastructure, the more value such attacks will be. Imagine a co-pilot attack that inserts hard-to-discover Heartbleed-esque bugs into code for example. It seems way way too early to be discussing regulation of algorithms of a field so young and changing so rapidly. 5 years from now, the network architectures may be unrecognizably different in both form and function. It might be better to have some industry-wide group like the W3C or IETF which sets these standards, and have tech published reports and audits for compliance, but done in a way to prevent attacks.
- uniqueuid 5y ago
  > Isn't there a problem with the sampling here? What if people who likely install RegretsReporter are also people who are more likely to view tangentially related problematic content in the first place, and then are shown more of it. Also, the dividing line on what's consider problematic is difficult, as we saw recently with Right Wing Watch. As I commented above, sampling is a problem, but the sample was only used to gather candidate videos which were then manually analyzed following Youtube's own content guidelines. So the takeaway is: According to the study's authors, Youtube recommends videos despite them violating their own guidelines.
  - cromwellian 5y ago
    Does Right Wing Watch violate the guidelines? It’s difficult enough to make AI that could enforce these guidelines, but even with a staff of tens of thousands of human evaluators I doubt you could avoid even 10% false positives given how much people argue over classification already. This kind of filtering, some say censoring, is super problematic both to pull off And to please all stakeholders, which is why a one size fits all regulation is likely to fail and create a new class of problems.
throwawayboise 5y ago
I keep telling YouTube that I'm "Not interested" in COVID-19 videos, but every other day they shove it in my face anyway.
kube-system 5y ago
> Like videos discussing the lab leak theory? Which lab leak theory? There are widely varying claims on this topic, ranging from narrow and nuanced scientific discussions to all-out nonsensical conjecture. Labelling all of these claims with a broad brush and equating them is simply a straw man.
- kiawe_fire 5y ago
  And yet, the “study” itself appears to imply that a video is extremist just because it has a title asserting that there is a left-leaning bias in some of the media. Seems false equivalencies, lack of nuances, and straw man arguments only matter some of the time.
  - kube-system 5y ago
    I wasn't defending the study. I doubt crowdsourced measurements of "regret" are good at measuring the accuracy of a video's content.
    - kiawe_fire 5y ago
      Fair enough, and your point is perfectly valid taken on its own.
justshowpost 5y ago
When Donald Trump talks about lab leak it is clearly racist conspiracy and harmful video, otherwise its okay https://comicallyincorrect.com/wp-content/uploads/2021/05/04-wuhan-con-la-1080-1050x750.jpg https://comicallyincorrect.com/wp-content/uploads/2021/05/04...

by tpmx 5y ago

uniqueuid 5y ago

The lab leak may have been in there (you can probably check in the appendix to the report). But to be honest, it's pretty irrelevant. Content analysis is usually performed using the current state of knowledge. It is to be expected that known facts and what is perceived as truth changes after the fact. This does not invalidate the analysis itself. Here's the explanation from the report, which I find pretty reasonable: > Over the course of three months, the working group developed a conceptual framework for classifying some of the videos, based on YouTube’s Community Guidelines. The working group decided to use YouTube’s Community Guidelines to guide the qualitative analysis because it provides a useful taxonomy of problematic video content and also represents a commitment from YouTube as to what sort of content should be included on their platform.

cromwellian 5y ago

Isn't there a problem with the sampling here? What if people who likely install RegretsReporter are also people who are more likely to view tangentially related problematic content in the first place, and then are shown more of it. Also, the dividing line on what's consider problematic is difficult, as we saw recently with Right Wing Watch. And the proposed regulation seems even more problematic. If the AI model were public, then it would be far easier for people to create adversarial content to game it. This is a problem with any models built on public data, including OpenAI's GPT stuff, or GitHub's Copilot. Detailed knowledge of how it works allows more efficient adversarial attacks, and the more these services become important public infrastructure, the more value such attacks will be. Imagine a co-pilot attack that inserts hard-to-discover Heartbleed-esque bugs into code for example. It seems way way too early to be discussing regulation of algorithms of a field so young and changing so rapidly. 5 years from now, the network architectures may be unrecognizably different in both form and function. It might be better to have some industry-wide group like the W3C or IETF which sets these standards, and have tech published reports and audits for compliance, but done in a way to prevent attacks.

uniqueuid 5y ago
> Isn't there a problem with the sampling here? What if people who likely install RegretsReporter are also people who are more likely to view tangentially related problematic content in the first place, and then are shown more of it. Also, the dividing line on what's consider problematic is difficult, as we saw recently with Right Wing Watch. As I commented above, sampling is a problem, but the sample was only used to gather candidate videos which were then manually analyzed following Youtube's own content guidelines. So the takeaway is: According to the study's authors, Youtube recommends videos despite them violating their own guidelines.
- cromwellian 5y ago
  Does Right Wing Watch violate the guidelines? It’s difficult enough to make AI that could enforce these guidelines, but even with a staff of tens of thousands of human evaluators I doubt you could avoid even 10% false positives given how much people argue over classification already. This kind of filtering, some say censoring, is super problematic both to pull off And to please all stakeholders, which is why a one size fits all regulation is likely to fail and create a new class of problems.

throwawayboise 5y ago

I keep telling YouTube that I'm "Not interested" in COVID-19 videos, but every other day they shove it in my face anyway.

kube-system 5y ago

> Like videos discussing the lab leak theory? Which lab leak theory? There are widely varying claims on this topic, ranging from narrow and nuanced scientific discussions to all-out nonsensical conjecture. Labelling all of these claims with a broad brush and equating them is simply a straw man.

kiawe_fire 5y ago
And yet, the “study” itself appears to imply that a video is extremist just because it has a title asserting that there is a left-leaning bias in some of the media. Seems false equivalencies, lack of nuances, and straw man arguments only matter some of the time.
- kube-system 5y ago
  I wasn't defending the study. I doubt crowdsourced measurements of "regret" are good at measuring the accuracy of a video's content.
  - kiawe_fire 5y ago
    Fair enough, and your point is perfectly valid taken on its own.

justshowpost 5y ago

When Donald Trump talks about lab leak it is clearly racist conspiracy and harmful video, otherwise its okay https://comicallyincorrect.com/wp-content/uploads/2021/05/04-wuhan-con-la-1080-1050x750.jpg https://comicallyincorrect.com/wp-content/uploads/2021/05/04...