6 ms·

>Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external r

by v7x 7y ago

>Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external research demo. Read: We don't want the internet to turn our program into Tay 2.0 because we are super duper serious. I get why these companies do this. But I can't help but feel like attempting to eliminate "bias" and tackle "safety" are counter productive to the goal of developing softwate desinged to mimic a species that is by and large biased and unsafe. Purposefully hindering a program because you're afraid it might produce results is not a healthy method of development, especially in an area like AI where we are still working to understand the basics of the field. Let the programs "learn" what's objective before trying to constrain it with the subjective.

mc32 7y ago
I guess I get the bias issue (though difficult to remain neutral on some subjects.) But safety? It’s a bot. No matter what it says to me there is no way I’m going to feel unsafe. I mean ok, don’t doxx people (I hope that’s kinda default behavior though).
- deleted 7y ago
  [deleted]
- DuskStar 7y ago
  "Our bot keeps asking for us to connect it to the internet and suggests that it'll be able to learn faster that way. We're really excited about this development, but a good fraction of our developers have started running around screaming about the apocalypse, so we're postponing public release until the chatbot tells us how to calm them down"
- IanCal 7y ago
  Maybe not you, but explicitly encouraging others to take harmful steps is a potential outcome. For example, there's a clear safety issue with a conversational bot that tells you to kill yourself if you talk to it about feeling suicidal or depressed. What if you talk to it about immigration and by picking commonly upvoted statements on certain online communities it starts to encourage violence? There are clear ways to me that a chatbot can have safety issues.
  - mc32 7y ago
    That’s a decent argument but then should we go to Hollywood and sanitize their violence in their movies (even those wanting to teach us about the evil of violence) for fear they might incite the wrong people who see such as calls to violence? What I mean is that that possibility exists in other media yet we don’t feel ambiguous about it.
    - jonas21 7y ago
      First of all, Hollywood does occasionally sanitize films and TV shows. For example, the suicide scene was removed from "13 Reasons Why" after concerns from mental health experts that it might inspire copycats [1]. But even ignoring that, I think most people have a good understanding of what a movie is and that fictional characters are not real. Movies have been around for a long time, most people have been watching them their entire lives. I don't think at this point that most people have nearly as good an understanding of chatbots. If you can converse with one and it mostly delivers responses like a person would, some people will start treating it like a person and ascribe values to its statements, and that's where it potentially gets dangerous. [1] https://www.nytimes.com/2019/07/16/arts/television/netflix-deleted-13-reasons-why-suicide-scene.html https://www.nytimes.com/2019/07/16/arts/television/netflix-d...
      - mc32 7y ago
        So you’re saying people can tell fiction from reality in film (even if film re-enacts actual events), but can’t use that same construct to evaluate a “bot”? Maybe... but it seems like a bit of a stretch.
        IanCal 7y ago
        People feel sad when their bomb disposal robots die. https://www.theatlantic.com/technology/archive/2013/09/funerals-for-fallen-robots/279861/ https://www.theatlantic.com/technology/archive/2013/09/funer... We're just not great at keeping what we know and how we feel in line.
    - 7y ago
jcims 7y ago
Maybe there needs to be a trail rating for chat bot safety. General public gets the bunny slope conversation, Tracy Morgan gets the double black diamonds. You work your way up by not attempting to cancel the project for something untoward that was said.
baron_harkonnen 7y ago
I suspect that the real reason is because non-cherry picked answers don't work as well and that if you interact with it you'll be no more impressed that decades of chatbots that aren't all that much more impressive than ELIZA.
- Polylactic_acid 7y ago
  I think part of the reason is that chat bots tend to be racist without any intentional effort to make them so. I saw a post last year where someone built the most default language AI setup using the most popular training data and then got it to rank words based on their positivity. Without any intentional effort the AI ranked English names as positive and African/Asian/etc names as neutral or negative words.
RcouF1uZ4gsC 7y ago
> we are not currently releasing an external research demo. Without an external research demo this is all just hype. It is pretty easy for your bot team to come up with and release conversation where your bot does really well. Having it work in the real world outside the research lab is the big issue.
- trevyn 7y ago
  Disclosing their architecture and current thinking is not "just hype".
KarlKemp 7y ago
These measures don’t seem to be too onerous, certainly less than what a government might come up with after the first Desaster following your approach. But it seems that’s not your most salient concern here, considering the scarequotes around „bias“, and the reversion of objective nd subjective you introduce.
redis_mlc 7y ago
I'm glad you mentioned Tay. When I need a laugh, I just read about Tay running amok: https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch... https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
01100011 7y ago
Maybe the last thing the human race needs is a machine that behaves like a human with the speed and precision of a computer?
mschuster91 7y ago
> Let the programs "learn" what's objective before trying to constrain it with the subjective. What is "objective" and what is not is not always something that is obvious and implementable in code. The hot topic of course is politics and religion - feed a bot the speeches of Donald Trump and another bot the speeches of Bernie Sanders, both will not be seen as "objective". Train it with both and the bot will still not be seen as "objective".

by v7x 7y ago

mc32 7y ago

I guess I get the bias issue (though difficult to remain neutral on some subjects.) But safety? It’s a bot. No matter what it says to me there is no way I’m going to feel unsafe. I mean ok, don’t doxx people (I hope that’s kinda default behavior though).

deleted 7y ago
[deleted]
DuskStar 7y ago
"Our bot keeps asking for us to connect it to the internet and suggests that it'll be able to learn faster that way. We're really excited about this development, but a good fraction of our developers have started running around screaming about the apocalypse, so we're postponing public release until the chatbot tells us how to calm them down"
IanCal 7y ago
Maybe not you, but explicitly encouraging others to take harmful steps is a potential outcome. For example, there's a clear safety issue with a conversational bot that tells you to kill yourself if you talk to it about feeling suicidal or depressed. What if you talk to it about immigration and by picking commonly upvoted statements on certain online communities it starts to encourage violence? There are clear ways to me that a chatbot can have safety issues.
- mc32 7y ago
  That’s a decent argument but then should we go to Hollywood and sanitize their violence in their movies (even those wanting to teach us about the evil of violence) for fear they might incite the wrong people who see such as calls to violence? What I mean is that that possibility exists in other media yet we don’t feel ambiguous about it.
  - jonas21 7y ago
    First of all, Hollywood does occasionally sanitize films and TV shows. For example, the suicide scene was removed from "13 Reasons Why" after concerns from mental health experts that it might inspire copycats [1]. But even ignoring that, I think most people have a good understanding of what a movie is and that fictional characters are not real. Movies have been around for a long time, most people have been watching them their entire lives. I don't think at this point that most people have nearly as good an understanding of chatbots. If you can converse with one and it mostly delivers responses like a person would, some people will start treating it like a person and ascribe values to its statements, and that's where it potentially gets dangerous. [1] https://www.nytimes.com/2019/07/16/arts/television/netflix-deleted-13-reasons-why-suicide-scene.html https://www.nytimes.com/2019/07/16/arts/television/netflix-d...
    - mc32 7y ago
      So you’re saying people can tell fiction from reality in film (even if film re-enacts actual events), but can’t use that same construct to evaluate a “bot”? Maybe... but it seems like a bit of a stretch.
      - IanCal 7y ago
        People feel sad when their bomb disposal robots die. https://www.theatlantic.com/technology/archive/2013/09/funerals-for-fallen-robots/279861/ https://www.theatlantic.com/technology/archive/2013/09/funer... We're just not great at keeping what we know and how we feel in line.
  - 7y ago

jcims 7y ago

Maybe there needs to be a trail rating for chat bot safety. General public gets the bunny slope conversation, Tracy Morgan gets the double black diamonds. You work your way up by not attempting to cancel the project for something untoward that was said.

baron_harkonnen 7y ago

I suspect that the real reason is because non-cherry picked answers don't work as well and that if you interact with it you'll be no more impressed that decades of chatbots that aren't all that much more impressive than ELIZA.

Polylactic_acid 7y ago
I think part of the reason is that chat bots tend to be racist without any intentional effort to make them so. I saw a post last year where someone built the most default language AI setup using the most popular training data and then got it to rank words based on their positivity. Without any intentional effort the AI ranked English names as positive and African/Asian/etc names as neutral or negative words.

RcouF1uZ4gsC 7y ago

> we are not currently releasing an external research demo. Without an external research demo this is all just hype. It is pretty easy for your bot team to come up with and release conversation where your bot does really well. Having it work in the real world outside the research lab is the big issue.

trevyn 7y ago
Disclosing their architecture and current thinking is not "just hype".

KarlKemp 7y ago

These measures don’t seem to be too onerous, certainly less than what a government might come up with after the first Desaster following your approach. But it seems that’s not your most salient concern here, considering the scarequotes around „bias“, and the reversion of objective nd subjective you introduce.

redis_mlc 7y ago

I'm glad you mentioned Tay. When I need a laugh, I just read about Tay running amok: https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch... https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)

01100011 7y ago

Maybe the last thing the human race needs is a machine that behaves like a human with the speed and precision of a computer?

mschuster91 7y ago

> Let the programs "learn" what's objective before trying to constrain it with the subjective. What is "objective" and what is not is not always something that is obvious and implementable in code. The hot topic of course is politics and religion - feed a bot the speeches of Donald Trump and another bot the speeches of Bernie Sanders, both will not be seen as "objective". Train it with both and the bot will still not be seen as "objective".