4 ms·

This was only a matter of time. For the DEFCON AI Village in August I talked about the implications of this sort of tech, and how that impacts how we release "

by avivo 8y ago

This was only a matter of time. For the DEFCON AI Village in August I talked about the implications of this sort of tech, and how that impacts how we release "exploit" code / think about "cognitive vulnerabilities": https://medium.com/@aviv/what-does-a-world-with-automated-social-engineering-look-like-79cd09b5a7b1 https://medium.com/@aviv/what-does-a-world-with-automated-so.... If you are doing work in this space, either in ML research or related security, you need to be thinking about implications (also see e.g. https://maliciousaireport.com https://maliciousaireport.com).

levesque 8y ago
I mean, the ideas are there. The scope of the project is probably too big to reproduce for now, but eventually it will be accessible to your average spammer / scammer. We will get there. We won't be able to get these tools locked, make them exclusive for a certain type of responsible AI specialists. Someone will spill the beans, the models. People with bad intentions will reproduce these results. To me, the real deal is how we will manage these outbursts when they happen. I assume discriminative models will solve the problem for a while, but as with Generative adversarial networks, you will be able to train models that are harder to discriminate against. I posit we're in for a big societal change (maybe more a content crisis) sometime in the next 10 years. Pretty sure we won't be able to keep it from falling in bad hands.
- Nydhal 8y ago
  On the flip side, this will give an edge to specific groups that can communicate in highly vernacular/non-written/non-standard languages.
- ctoth 8y ago
  “Spam-filters, actually. Once they became selfmodifying, spam-filters and spam-bots got into a war to see which could act more human, and since their failures invoked a human judgement about whether their material were convincingly human, it was like a trillion Turing-tests from which they could learn. From there came the first machineintelligence algorithms, and then my kind.” I Row-Boat, Cory Doctorow, 2005: https://craphound.com/overclocked/Cory_Doctorow_-_Overclocked_-_I_Row-Boat-A4.pdf https://craphound.com/overclocked/Cory_Doctorow_-_Overclocke...
  - mr_toad 8y ago
    https://xkcd.com/810/ https://xkcd.com/810/
GistNoesis 8y ago
"you need to be thinking about implications" Thinking about it, won't put food on your table. These are not necessarily similar to "exploit" codes, but more like a box of innocuous tricks often needed for good purposes. Gather enough of those throw enough compute, then you go from something that a human can perceive as obviously false, to something human-like. There are plenty of people out there, with the competence to replicate those results, yet only a few big companies reap most the rewards, monopolize user data and interactions, and open-source for free results which could have been monetized by a company to provide useful services to users. Seriously, how can one expect that most of those Phd who didn't get recruited don't use AI for nefarious uses?

by avivo 8y ago

levesque 8y ago

I mean, the ideas are there. The scope of the project is probably too big to reproduce for now, but eventually it will be accessible to your average spammer / scammer. We will get there. We won't be able to get these tools locked, make them exclusive for a certain type of responsible AI specialists. Someone will spill the beans, the models. People with bad intentions will reproduce these results. To me, the real deal is how we will manage these outbursts when they happen. I assume discriminative models will solve the problem for a while, but as with Generative adversarial networks, you will be able to train models that are harder to discriminate against. I posit we're in for a big societal change (maybe more a content crisis) sometime in the next 10 years. Pretty sure we won't be able to keep it from falling in bad hands.

Nydhal 8y ago
On the flip side, this will give an edge to specific groups that can communicate in highly vernacular/non-written/non-standard languages.
ctoth 8y ago
“Spam-filters, actually. Once they became selfmodifying, spam-filters and spam-bots got into a war to see which could act more human, and since their failures invoked a human judgement about whether their material were convincingly human, it was like a trillion Turing-tests from which they could learn. From there came the first machineintelligence algorithms, and then my kind.” I Row-Boat, Cory Doctorow, 2005: https://craphound.com/overclocked/Cory_Doctorow_-_Overclocked_-_I_Row-Boat-A4.pdf https://craphound.com/overclocked/Cory_Doctorow_-_Overclocke...
- mr_toad 8y ago
  https://xkcd.com/810/ https://xkcd.com/810/

GistNoesis 8y ago

"you need to be thinking about implications" Thinking about it, won't put food on your table. These are not necessarily similar to "exploit" codes, but more like a box of innocuous tricks often needed for good purposes. Gather enough of those throw enough compute, then you go from something that a human can perceive as obviously false, to something human-like. There are plenty of people out there, with the competence to replicate those results, yet only a few big companies reap most the rewards, monopolize user data and interactions, and open-source for free results which could have been monetized by a company to provide useful services to users. Seriously, how can one expect that most of those Phd who didn't get recruited don't use AI for nefarious uses?