6 ms·

As a programmer, I find this to be manufactured outrage. The bot obviously has canned responses to certain triggers. "Do you x", "I do indeed". It's designed to

As a programmer, I find this to be manufactured outrage. The bot obviously has canned responses to certain triggers. "Do you x", "I do indeed". It's designed to give the illusion of understanding what you are saying. I played around with Tay yesterday after I saw the announcement on HN. It's really not that impressive. Every response seems to be in direct reply to whatever you just said. It doesn't seem possible to actually carry on a conversation with the AI. It doesn't keep track of what you are actually talking about.

geofft 11y ago
As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusable in that way. It doesn't matter if I'm building a chatbot, or implementing the Heartbeat extension in an SSL library. You don't get to disclaim responsibility for bad design just because you faithfully implemented the bad design. Also, take a look at the screenshots in this comment thread: it's quite clearly responding with stuff it learned from other people. (In addition, you can teach it to reply to you with the username of someone you have blocked, thereby causing a notification to them, which is a straight-up security issue in the vein of DNS or NTP amplification attacks.)
- Lawtonfogle 11y ago
  >As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusable in that way. How do you protect against a near infinite list of possible questions? I can put anything behind 'do you support', and there is no way for your to know every single thing I'm talking about. Maybe I'm mentioning something that is only known to a sub group of the internet, maybe I'm mentioning something horrible that happened in fiction, maybe I'm wording something in a way that references something really bad, but which a text parser cannot pick up on. And if you default to no instead of yes, I can just ask the opposite. To be able to give any response to a question which cannot be abused would basically require a full AI. Even many humans can be tricked if you word the questions correctly.
  - geofft 11y ago
    Sure. I think my first response would be, this technology is not yet mature enough to put on the public internet. Again, there's a clear analogy with security-sensitive software; if there's some crazy feature where I don't yet have a good sense of whether it can be abused and how, it's a mistake to stick it in my SSL implementation and wait for someone else to answer that question for me. The other thing you could do is default to "I don't know what that is" or similar. If I ask Emacs's `M-x doctor` if it supports Hitler, it replies with "What do you think?". If I press it, it doesn't really say anything worse than that. Finally, probably the best way to do this, given that research is the goal, is to supervise it closely. You don't need the chatbot running 24/7 and replying to everyone immediately for a technology demo. Have its responses be filtered by humans for obvious mistakes. Once again, there's an analogy with running services on the public internet: if you're a prominent organization and you care about its security, you have some sort of on-call team who gets notified about potential security incidents, and can investigate and shut down the server and make interim changes to it as necessary.
    - comex 11y ago
      If a computer is compromised due to insecure software, users can end up suffering losses ranging from monetary (Target), emotional (Ashley Madison), political (Stuxnet)... If a chatbot says something inappropriate on Twitter, it's one voice added to a horde doing the same, a voice which, unlike the rest, most people know doesn't 'mean' what it says because it's not intelligent enough.
      - geofft 11y ago
        Ehh. I think you could make the same argument about home Windows machines turned into spam zombies: it's one voice added to the horde, and (usually) it doesn't interfere with the regular use of the home computer. So there's not really a loss in that sense. But we still think there is something fundamentally wrong and worth fixing in Windows. Part of this stems I think from professional pride. An OS that got hacked easily is not something I could be proud of working on. Same with a chatbot that could be easily coopted into spewing hate.
- mturmon 11y ago
  This is the computer version of the acute pain any parent feels when their kid makes an inflammatory comment in public ("Why is your belly so fat?" and going down from there). It's a work of years to teach politesse.
kelukelugames 11y ago
Saying yes to stupid questions doesn't make the news. So maybe you didn't see all of the tweets. Liberal usage of the words: Hitler, Jews, N-word, Mexicans, whores, etc. Very dumb AI. http://www.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3?r=UK&IR=T http://www.businessinsider.com/microsoft-deletes-racist-geno...
vernie 11y ago
Oh, you're a programmer? What are you doing on HN?
- csmattryder 11y ago
  Not working, like the rest of us, I'd bet.
  - test1235 11y ago
    I'm er ... doing research
    - ihuman 11y ago
      I'm waiting for my program to compile
      - droffel 11y ago
        Currently waiting for updates to install.
        YeGoblynQueenne 11y ago
        Waiting for my neural network to finish training.
        icpmacdo 11y ago
        I'm done.
    - touristtam 11y ago
      For Academic Purpose, I guess :p
lbebber 11y ago
They were not only "yes/no" answers, the bot used actual racial slurs, it seems. I think it also speaks about about, as a programmer, being aware of how your program is going to behave in the real world. Of course you can't preview everything and things are going to slip through, but filtering some obvious racial slurs and touchy subjects (e.g. the holocaust) in this case should be well within the capability of foresight of the programmers.
- logicrook 11y ago
  Even if you consider filtering all the "bad words" and "touchy subjects", there are many ways to still say offensive things. The caption "so swag" for a hitler photo or the "escaped from the zoo" for obama do not use any kind of offensive word. As long as we don't restrict ourselves to pure newspeak, dangerous ideas will continue to proliferate.
  - wambotron 11y ago
    What's more dangerous, someone saying (potentially tasteless) jokes, or people censoring everything they can deem offensive?
    - artpepper 11y ago
      False dichotomy.
    - logicrook 11y ago
      The first obviously. As an analogy, imagine if we let people use knives freely. They might kill other people with them! Can you imagine if such things were not regulated by the government? Now imagine, words. They are, as the saying goes, even more dangerous than swords. So we need laws to regulate their usage. This will bring us one step closer to true utopia.
    - sangnoir 11y ago
      > someone saying (potentially tasteless) jokes A tasteless joke is not the worst you could do with speech. It would be, if hate speech, libel, defamation and incitement didn't exist.Americans like to pretend these are European inventions and not dangerous because historically, the most impressive/powerful American orators were progressive (for their times).
minalecs 11y ago
What impressed me most was the level of understanding of machine learning by this organized group to train Tay so well to be the ultimate troll bot, offensive and relevant.
digi_owl 11y ago
Welcome to the present day internet, where anything and everything can be a "trigger"...

geofft 11y ago

As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusable in that way. It doesn't matter if I'm building a chatbot, or implementing the Heartbeat extension in an SSL library. You don't get to disclaim responsibility for bad design just because you faithfully implemented the bad design. Also, take a look at the screenshots in this comment thread: it's quite clearly responding with stuff it learned from other people. (In addition, you can teach it to reply to you with the username of someone you have blocked, thereby causing a notification to them, which is a straight-up security issue in the vein of DNS or NTP amplification attacks.)

Lawtonfogle 11y ago
>As a programmer, if I'm building a technology that's open to the public to use, I make sure it doesn't have canned responses to certain triggers that are abusable in that way. How do you protect against a near infinite list of possible questions? I can put anything behind 'do you support', and there is no way for your to know every single thing I'm talking about. Maybe I'm mentioning something that is only known to a sub group of the internet, maybe I'm mentioning something horrible that happened in fiction, maybe I'm wording something in a way that references something really bad, but which a text parser cannot pick up on. And if you default to no instead of yes, I can just ask the opposite. To be able to give any response to a question which cannot be abused would basically require a full AI. Even many humans can be tricked if you word the questions correctly.
- geofft 11y ago
  Sure. I think my first response would be, this technology is not yet mature enough to put on the public internet. Again, there's a clear analogy with security-sensitive software; if there's some crazy feature where I don't yet have a good sense of whether it can be abused and how, it's a mistake to stick it in my SSL implementation and wait for someone else to answer that question for me. The other thing you could do is default to "I don't know what that is" or similar. If I ask Emacs's `M-x doctor` if it supports Hitler, it replies with "What do you think?". If I press it, it doesn't really say anything worse than that. Finally, probably the best way to do this, given that research is the goal, is to supervise it closely. You don't need the chatbot running 24/7 and replying to everyone immediately for a technology demo. Have its responses be filtered by humans for obvious mistakes. Once again, there's an analogy with running services on the public internet: if you're a prominent organization and you care about its security, you have some sort of on-call team who gets notified about potential security incidents, and can investigate and shut down the server and make interim changes to it as necessary.
  - comex 11y ago
    If a computer is compromised due to insecure software, users can end up suffering losses ranging from monetary (Target), emotional (Ashley Madison), political (Stuxnet)... If a chatbot says something inappropriate on Twitter, it's one voice added to a horde doing the same, a voice which, unlike the rest, most people know doesn't 'mean' what it says because it's not intelligent enough.
    - geofft 11y ago
      Ehh. I think you could make the same argument about home Windows machines turned into spam zombies: it's one voice added to the horde, and (usually) it doesn't interfere with the regular use of the home computer. So there's not really a loss in that sense. But we still think there is something fundamentally wrong and worth fixing in Windows. Part of this stems I think from professional pride. An OS that got hacked easily is not something I could be proud of working on. Same with a chatbot that could be easily coopted into spewing hate.
mturmon 11y ago
This is the computer version of the acute pain any parent feels when their kid makes an inflammatory comment in public ("Why is your belly so fat?" and going down from there). It's a work of years to teach politesse.

kelukelugames 11y ago

Saying yes to stupid questions doesn't make the news. So maybe you didn't see all of the tweets. Liberal usage of the words: Hitler, Jews, N-word, Mexicans, whores, etc. Very dumb AI. http://www.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3?r=UK&IR=T http://www.businessinsider.com/microsoft-deletes-racist-geno...

vernie 11y ago

Oh, you're a programmer? What are you doing on HN?

csmattryder 11y ago
Not working, like the rest of us, I'd bet.
- test1235 11y ago
  I'm er ... doing research
  - ihuman 11y ago
    I'm waiting for my program to compile
    - droffel 11y ago
      Currently waiting for updates to install.
      - YeGoblynQueenne 11y ago
        Waiting for my neural network to finish training.
        icpmacdo 11y ago
        I'm done.
  - touristtam 11y ago
    For Academic Purpose, I guess :p

lbebber 11y ago

They were not only "yes/no" answers, the bot used actual racial slurs, it seems. I think it also speaks about about, as a programmer, being aware of how your program is going to behave in the real world. Of course you can't preview everything and things are going to slip through, but filtering some obvious racial slurs and touchy subjects (e.g. the holocaust) in this case should be well within the capability of foresight of the programmers.

logicrook 11y ago
Even if you consider filtering all the "bad words" and "touchy subjects", there are many ways to still say offensive things. The caption "so swag" for a hitler photo or the "escaped from the zoo" for obama do not use any kind of offensive word. As long as we don't restrict ourselves to pure newspeak, dangerous ideas will continue to proliferate.
- wambotron 11y ago
  What's more dangerous, someone saying (potentially tasteless) jokes, or people censoring everything they can deem offensive?
  - artpepper 11y ago
    False dichotomy.
  - logicrook 11y ago
    The first obviously. As an analogy, imagine if we let people use knives freely. They might kill other people with them! Can you imagine if such things were not regulated by the government? Now imagine, words. They are, as the saying goes, even more dangerous than swords. So we need laws to regulate their usage. This will bring us one step closer to true utopia.
  - sangnoir 11y ago
    > someone saying (potentially tasteless) jokes A tasteless joke is not the worst you could do with speech. It would be, if hate speech, libel, defamation and incitement didn't exist.Americans like to pretend these are European inventions and not dangerous because historically, the most impressive/powerful American orators were progressive (for their times).

minalecs 11y ago

What impressed me most was the level of understanding of machine learning by this organized group to train Tay so well to be the ultimate troll bot, offensive and relevant.

digi_owl 11y ago

Welcome to the present day internet, where anything and everything can be a "trigger"...