3 ms·

Disclaimer - I work at Microsoft They put a lot of work into the research to make this safe already - they had the model red team itself, in a reinforcement le

Disclaimer - I work at Microsoft They put a lot of work into the research to make this safe already - they had the model red team itself, in a reinforcement learning environment probably. It won't be perfect, but at this point, they need a huge user base to adversarial test it any further. Otherwise no progress at all would be made. edit - I should say I have no internal knowledge about this.

gordon_freeman 4y ago
Do you think MSFT has rolled this out too soon rather than getting it tested with huge test user base like their own employees similarly to what Google is doing?
- buildbot 4y ago
  No, I think to really get these battle tested it needs to be in everyone's hands. Microsoft is a limited selection of people, and the world is vast. Plus, internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.
  - thesuperbigfrog 4y ago
    >> internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. >> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated. Hopefully better than previous attempts to put AI on the Internet: https://youtu.be/ajGX7odA87k?t=20m12s https://youtu.be/ajGX7odA87k?t=20m12s
jimrandomh 4y ago
And that research is valuable, but it's definitely nowhere near the point where safety/alignment progress can't be made without a user base. A few recent favorites of mine: https://arxiv.org/pdf/2212.03827.pdf https://arxiv.org/pdf/2212.03827.pdf https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solid...
- the8472 4y ago
  Note that that kind of research was only possible on the (smaller) open models. When the new models are proprietary ones then outsiders can't do that kind of analysis. If they had regenerated the tokenization dictionary to match the training data instead of reusing an older set then this wouldn't transfer from GPT2 to ChatGPT.
PeterStuer 4y ago
Safety is a subjective quality. What are the downside of "put"[ing] a lot of work into the research to make this safe already "?

gordon_freeman 4y ago

Do you think MSFT has rolled this out too soon rather than getting it tested with huge test user base like their own employees similarly to what Google is doing?

buildbot 4y ago
No, I think to really get these battle tested it needs to be in everyone's hands. Microsoft is a limited selection of people, and the world is vast. Plus, internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.
- thesuperbigfrog 4y ago
  >> internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. >> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated. Hopefully better than previous attempts to put AI on the Internet: https://youtu.be/ajGX7odA87k?t=20m12s https://youtu.be/ajGX7odA87k?t=20m12s

jimrandomh 4y ago

And that research is valuable, but it's definitely nowhere near the point where safety/alignment progress can't be made without a user base. A few recent favorites of mine: https://arxiv.org/pdf/2212.03827.pdf https://arxiv.org/pdf/2212.03827.pdf https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solid...

the8472 4y ago
Note that that kind of research was only possible on the (smaller) open models. When the new models are proprietary ones then outsiders can't do that kind of analysis. If they had regenerated the tokenization dictionary to match the training data instead of reusing an older set then this wouldn't transfer from GPT2 to ChatGPT.

PeterStuer 4y ago

Safety is a subjective quality. What are the downside of "put"[ing] a lot of work into the research to make this safe already "?