5 ms·

And there wouldn't even be a way to prove it was happening. Given the opacity of deep learning models, it's possible that no one, not even the people at Faceboo

And there wouldn't even be a way to prove it was happening. Given the opacity of deep learning models, it's possible that no one, not even the people at Facebook who developed the algorithm, would realize that it had trained itself to induce suicides.

rcthompson 9y ago
Thinking about it, this actually reminds me of an eerily similar anecdote from a completely different context. Researchers tried to use an evolutionary algorithm to evolve a robot that could walk (in a computer physics simulation). Since none of the initial randomly generated robots could make consistent forward progress, they started out by optimizing for maximum distance traveled before coming to a complete stop. As the evolution proceeded, the robots kept making it farther and farther, until they seemed to hit a limit. When the researches looked at what the robots had evolved into, they realized that the robots had evolved to simply fall forward and roll or slide as far as possible. The limit they hit was the farthest a robot within the designated size constraints could possibly fall. This was obviously an evolutionary dead end from the perspective of evolving a walking robot.
- em3rgent0rdr 9y ago
  Achieved a local optimum, not global optimum. One technique to avoid getting stuck in such local optimums is to have some children which have extreme mutations.
  - rcthompson 9y ago
    You're making it sound like a simple parameter optimization issue, but it's hard to imagine a mutation that breaks out of the "fall forward as far as possible" paradigm without discarding the parent phenotype entirely, which would defeat the purpose of using an evolutionary algorithm in the first place. Anyway, the point was that a complex optimization problem resulted in an unexpected "suicidal" phenotype, i.e. falling over instead of walking.
machinehermit 9y ago
I think more so this discussion just shows how humans naturally like sensational nonsense over reality. Reality is always so boring compared to fiction.

rcthompson 9y ago

Thinking about it, this actually reminds me of an eerily similar anecdote from a completely different context. Researchers tried to use an evolutionary algorithm to evolve a robot that could walk (in a computer physics simulation). Since none of the initial randomly generated robots could make consistent forward progress, they started out by optimizing for maximum distance traveled before coming to a complete stop. As the evolution proceeded, the robots kept making it farther and farther, until they seemed to hit a limit. When the researches looked at what the robots had evolved into, they realized that the robots had evolved to simply fall forward and roll or slide as far as possible. The limit they hit was the farthest a robot within the designated size constraints could possibly fall. This was obviously an evolutionary dead end from the perspective of evolving a walking robot.

em3rgent0rdr 9y ago
Achieved a local optimum, not global optimum. One technique to avoid getting stuck in such local optimums is to have some children which have extreme mutations.
- rcthompson 9y ago
  You're making it sound like a simple parameter optimization issue, but it's hard to imagine a mutation that breaks out of the "fall forward as far as possible" paradigm without discarding the parent phenotype entirely, which would defeat the purpose of using an evolutionary algorithm in the first place. Anyway, the point was that a complex optimization problem resulted in an unexpected "suicidal" phenotype, i.e. falling over instead of walking.

machinehermit 9y ago

I think more so this discussion just shows how humans naturally like sensational nonsense over reality. Reality is always so boring compared to fiction.