6 ms·

"Cornell University is studying human susceptibility to digital disinformation generated by language models." "The Middlebury Institute of International Studie

"Cornell University is studying human susceptibility to digital disinformation generated by language models." "The Middlebury Institute of International Studies Center on Terrorism, Extremism, and Counterterrorism (CTEC) is exploring how GPT-2 could be misused by terrorists and extremists online." "The University of Oregon is developing a series of “bias probes” to analyze bias within GPT-2." But apparently no university studies the social and economic impact of using terabytes of public data to train algorithms that for all practical reasons end up being inaccessible to an average person. If things go on the way they're going right now, in 20 years millions of people will be "mechanical turked". Most of information processing tools will be mediated exclusively through companies like Google and Amazon. They will be less like normal tools (e.g. word processors) and more like systems you have to be a part of. Can you imagine the levels of inequality involved? The hyper-centralization of power? This is the foremost challenge presented by AI, not some hypothetical nonsense involving terrorists using a text generator. And it's not like there aren't any solutions. Douglas Engelbart, for example, pointed out a great way of introducing technology into society without screwing most of the society over: http://dougengelbart.org/content/view/138 http://dougengelbart.org/content/view/138 We kind of followed his vision for a while, with good results, but AI seems to be going in an entirely different direction.

comradesmith 7y ago
http://talktotransformer.com http://talktotransformer.com Seems accessible to me
- postalrat 7y ago
  That isn't running the full model.
  - enkiv2 7y ago
    It's running the latest model to be released, which is 774M. Since the initial announcement, larger models have been released every few months, so we're on track to have the full model released by 2020. (This is what OP is literally about -- the roadmap for releasing larger subsets of the model.)
- inimino 7y ago
  "Accessible" means more than "there's a web service available somewhere that's running some version of it".
newhaus1994 7y ago
We (the Middlebury Institute's CTEC) are an extremism and terrorism research lab, and so we're tracking the ways that tech is used by terrorists and extremists. For a lot of nonstate orgs with sophisticated propaganda arms, an ideologically cohesive text generation capability would be a huge advantage in scaling up info ops. We are looking to measure whether or not GPT-2 or other neural text generators are useful for this, or if that risk is, as you say, nonsense.
- 40four 7y ago
  I think their point isn't that terrorists leveraging this tech not a problem. It is certainly a problem. But the greater problem being a few large entities being the only ones who have access to or control over it. I think it's pretty clear that terrorists or any other bad actor will find great value & utility in this tech. The article from OpenAI says 'Humans can be convinced by synthetic text.' & research at Cornell found people find it almost as convincing as New York Times articles. I would be interested in learning about the methods you guys are using to determine of it? I wonder how that could be measured? So let's assume the answer is 'YES! This technology is dangerous". The Middlebury program, Cornell, and more and more universities and research groups find the same thing. Then what will the recommendations be? Certainly not to release it into the wild. I think they will be to keep it locked up. To keep it in the hands of a few large and powerful companies, with the resources to 'manage' such a thing. This seems to be what the original comment is trying to illustrate, and I think it's an interesting point to consider the implications of long term. The tech exists now. There is no going back. So is it worse to let it out of the box, or to let a but a few have control over it?
  - newhaus1994 7y ago
    In spite of all that we're studying wrt abuse potential, I (and my team) generally support open-sourcing tech, and I hope that we can contribute not to "oh this is dangerous, don't release" but rather to "oh this is dangerous, it's already released, what are we going to do now?"
    - 7y ago
skybrian 7y ago
It seems like this is missing the point of public data? When you make an edit to Wikipedia, anyone in the world can read it. You don't benefit when they read an article, but it doesn't cost you anything either. "Anyone" includes researchers. That's part of the deal. Yes, they benefit, but you aren't harmed. That's zero-sum thinking.
- insulanus 7y ago
  I think you are correct that nothing is taken away from an author when someone reads their Wikipedia article. Perhaps what the poster above you is saying is that there is a continuum of information, some more personal and sensitive, like your current location, and some less personal, like the Wikipedia article on Elephants. Taking data about specific humans, (or humans in general), and turning it into code that has predictive power seems like a different type of power than the knowledge given by an encyclopedia.
  - skybrian 7y ago
    It doesn't cost you anything when someone uses your Reddit posts to train a model either. The supposed harm is very tenuous.
    - gambler 7y ago
      Look up some semi-recent talks on social media by Jaron Lanier.
jcims 7y ago
Whoa, it’s remarkable how well that paper holds up for being nearly 60 years old. I wonder if it’s just a timeless problem.
avivo 7y ago
I'm not sure if this captures this full set of tensions at play. I'm a huge fan of Doug Engelbart, and he was my inspiration for a long time...but it turns out augmentation of power without checks and balances can end up very messy (I'm also a fan of James Madison, though Engelbart is closer to a first love!). One could argue that I actually started work on misinformation though Engelbart. I was working on various projects specifically to achieve a vision like his, and it is still my guiding light. But it turns out that some technologies (initially focusing on Facebook and YouTube's engagement dynamics...), without appropriate checks and balances, are sort of anti-augmentation of intellect. They give an asymmetric advantage to those who are trying to weaken our intellects. So I ended up dropping those projects to attempt to address urgent misinformation issues in May of 2016. Going back to GPT-2 and research release, I and my co-author recently went deep into understanding the types of risks and tradeoffs in a recent paper. You can see the summary here: https://medium.com/@aviv/reducing-malicious-use-of-synthetic-media-research-9def6ab81aaf https://medium.com/@aviv/reducing-malicious-use-of-synthetic... or go directly to arXiv https://arxiv.org/pdf/1907.11274.pdf https://arxiv.org/pdf/1907.11274.pdf . The goal of our paper is specifically to go past the angry invective of the "here is the most important problem" (in your case, "AI inequality") and actually dive into the weeds of threat models and tradeoffs.
- gambler 7y ago
  You're just proving my point. If we go past the fashionable rhetoric about AI and apply the same reasoning to existing technologies, we could conclude that Photoshop was too dangerous to release to the public without "appropriate checks and balances". It can be used to create harmful memes and doctor photos! So what tools should an average person be allowed to use? Also, with so many people concerned about disinformation, where is research on tools that would empower individual users to process information in better ways and make sounded judgements?

comradesmith 7y ago

http://talktotransformer.com http://talktotransformer.com Seems accessible to me

postalrat 7y ago
That isn't running the full model.
- enkiv2 7y ago
  It's running the latest model to be released, which is 774M. Since the initial announcement, larger models have been released every few months, so we're on track to have the full model released by 2020. (This is what OP is literally about -- the roadmap for releasing larger subsets of the model.)
inimino 7y ago
"Accessible" means more than "there's a web service available somewhere that's running some version of it".

newhaus1994 7y ago

We (the Middlebury Institute's CTEC) are an extremism and terrorism research lab, and so we're tracking the ways that tech is used by terrorists and extremists. For a lot of nonstate orgs with sophisticated propaganda arms, an ideologically cohesive text generation capability would be a huge advantage in scaling up info ops. We are looking to measure whether or not GPT-2 or other neural text generators are useful for this, or if that risk is, as you say, nonsense.

40four 7y ago
I think their point isn't that terrorists leveraging this tech not a problem. It is certainly a problem. But the greater problem being a few large entities being the only ones who have access to or control over it. I think it's pretty clear that terrorists or any other bad actor will find great value & utility in this tech. The article from OpenAI says 'Humans can be convinced by synthetic text.' & research at Cornell found people find it almost as convincing as New York Times articles. I would be interested in learning about the methods you guys are using to determine of it? I wonder how that could be measured? So let's assume the answer is 'YES! This technology is dangerous". The Middlebury program, Cornell, and more and more universities and research groups find the same thing. Then what will the recommendations be? Certainly not to release it into the wild. I think they will be to keep it locked up. To keep it in the hands of a few large and powerful companies, with the resources to 'manage' such a thing. This seems to be what the original comment is trying to illustrate, and I think it's an interesting point to consider the implications of long term. The tech exists now. There is no going back. So is it worse to let it out of the box, or to let a but a few have control over it?
- newhaus1994 7y ago
  In spite of all that we're studying wrt abuse potential, I (and my team) generally support open-sourcing tech, and I hope that we can contribute not to "oh this is dangerous, don't release" but rather to "oh this is dangerous, it's already released, what are we going to do now?"
  - 7y ago

skybrian 7y ago

It seems like this is missing the point of public data? When you make an edit to Wikipedia, anyone in the world can read it. You don't benefit when they read an article, but it doesn't cost you anything either. "Anyone" includes researchers. That's part of the deal. Yes, they benefit, but you aren't harmed. That's zero-sum thinking.

insulanus 7y ago
I think you are correct that nothing is taken away from an author when someone reads their Wikipedia article. Perhaps what the poster above you is saying is that there is a continuum of information, some more personal and sensitive, like your current location, and some less personal, like the Wikipedia article on Elephants. Taking data about specific humans, (or humans in general), and turning it into code that has predictive power seems like a different type of power than the knowledge given by an encyclopedia.
- skybrian 7y ago
  It doesn't cost you anything when someone uses your Reddit posts to train a model either. The supposed harm is very tenuous.
  - gambler 7y ago
    Look up some semi-recent talks on social media by Jaron Lanier.

jcims 7y ago

Whoa, it’s remarkable how well that paper holds up for being nearly 60 years old. I wonder if it’s just a timeless problem.

avivo 7y ago

I'm not sure if this captures this full set of tensions at play. I'm a huge fan of Doug Engelbart, and he was my inspiration for a long time...but it turns out augmentation of power without checks and balances can end up very messy (I'm also a fan of James Madison, though Engelbart is closer to a first love!). One could argue that I actually started work on misinformation though Engelbart. I was working on various projects specifically to achieve a vision like his, and it is still my guiding light. But it turns out that some technologies (initially focusing on Facebook and YouTube's engagement dynamics...), without appropriate checks and balances, are sort of anti-augmentation of intellect. They give an asymmetric advantage to those who are trying to weaken our intellects. So I ended up dropping those projects to attempt to address urgent misinformation issues in May of 2016. Going back to GPT-2 and research release, I and my co-author recently went deep into understanding the types of risks and tradeoffs in a recent paper. You can see the summary here: https://medium.com/@aviv/reducing-malicious-use-of-synthetic-media-research-9def6ab81aaf https://medium.com/@aviv/reducing-malicious-use-of-synthetic... or go directly to arXiv https://arxiv.org/pdf/1907.11274.pdf https://arxiv.org/pdf/1907.11274.pdf . The goal of our paper is specifically to go past the angry invective of the "here is the most important problem" (in your case, "AI inequality") and actually dive into the weeds of threat models and tradeoffs.

gambler 7y ago
You're just proving my point. If we go past the fashionable rhetoric about AI and apply the same reasoning to existing technologies, we could conclude that Photoshop was too dangerous to release to the public without "appropriate checks and balances". It can be used to create harmful memes and doctor photos! So what tools should an average person be allowed to use? Also, with so many people concerned about disinformation, where is research on tools that would empower individual users to process information in better ways and make sounded judgements?