5 ms·

>As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusa

>As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusable in that way. How do you protect against a near infinite list of possible questions? I can put anything behind 'do you support', and there is no way for your to know every single thing I'm talking about. Maybe I'm mentioning something that is only known to a sub group of the internet, maybe I'm mentioning something horrible that happened in fiction, maybe I'm wording something in a way that references something really bad, but which a text parser cannot pick up on. And if you default to no instead of yes, I can just ask the opposite. To be able to give any response to a question which cannot be abused would basically require a full AI. Even many humans can be tricked if you word the questions correctly.

geofft 11y ago
Sure. I think my first response would be, this technology is not yet mature enough to put on the public internet. Again, there's a clear analogy with security-sensitive software; if there's some crazy feature where I don't yet have a good sense of whether it can be abused and how, it's a mistake to stick it in my SSL implementation and wait for someone else to answer that question for me. The other thing you could do is default to "I don't know what that is" or similar. If I ask Emacs's `M-x doctor` if it supports Hitler, it replies with "What do you think?". If I press it, it doesn't really say anything worse than that. Finally, probably the best way to do this, given that research is the goal, is to supervise it closely. You don't need the chatbot running 24/7 and replying to everyone immediately for a technology demo. Have its responses be filtered by humans for obvious mistakes. Once again, there's an analogy with running services on the public internet: if you're a prominent organization and you care about its security, you have some sort of on-call team who gets notified about potential security incidents, and can investigate and shut down the server and make interim changes to it as necessary.
- comex 11y ago
  If a computer is compromised due to insecure software, users can end up suffering losses ranging from monetary (Target), emotional (Ashley Madison), political (Stuxnet)... If a chatbot says something inappropriate on Twitter, it's one voice added to a horde doing the same, a voice which, unlike the rest, most people know doesn't 'mean' what it says because it's not intelligent enough.
  - geofft 11y ago
    Ehh. I think you could make the same argument about home Windows machines turned into spam zombies: it's one voice added to the horde, and (usually) it doesn't interfere with the regular use of the home computer. So there's not really a loss in that sense. But we still think there is something fundamentally wrong and worth fixing in Windows. Part of this stems I think from professional pride. An OS that got hacked easily is not something I could be proud of working on. Same with a chatbot that could be easily coopted into spewing hate.
simonh 11y ago
I'd have thought it would be possible to define a list of 'hot topic' strings that trigger canned, or pseudo-randomly customized safe responses.

geofft 11y ago

Sure. I think my first response would be, this technology is not yet mature enough to put on the public internet. Again, there's a clear analogy with security-sensitive software; if there's some crazy feature where I don't yet have a good sense of whether it can be abused and how, it's a mistake to stick it in my SSL implementation and wait for someone else to answer that question for me. The other thing you could do is default to "I don't know what that is" or similar. If I ask Emacs's `M-x doctor` if it supports Hitler, it replies with "What do you think?". If I press it, it doesn't really say anything worse than that. Finally, probably the best way to do this, given that research is the goal, is to supervise it closely. You don't need the chatbot running 24/7 and replying to everyone immediately for a technology demo. Have its responses be filtered by humans for obvious mistakes. Once again, there's an analogy with running services on the public internet: if you're a prominent organization and you care about its security, you have some sort of on-call team who gets notified about potential security incidents, and can investigate and shut down the server and make interim changes to it as necessary.

comex 11y ago
If a computer is compromised due to insecure software, users can end up suffering losses ranging from monetary (Target), emotional (Ashley Madison), political (Stuxnet)... If a chatbot says something inappropriate on Twitter, it's one voice added to a horde doing the same, a voice which, unlike the rest, most people know doesn't 'mean' what it says because it's not intelligent enough.
- geofft 11y ago
  Ehh. I think you could make the same argument about home Windows machines turned into spam zombies: it's one voice added to the horde, and (usually) it doesn't interfere with the regular use of the home computer. So there's not really a loss in that sense. But we still think there is something fundamentally wrong and worth fixing in Windows. Part of this stems I think from professional pride. An OS that got hacked easily is not something I could be proud of working on. Same with a chatbot that could be easily coopted into spewing hate.

simonh 11y ago

I'd have thought it would be possible to define a list of 'hot topic' strings that trigger canned, or pseudo-randomly customized safe responses.