4 ms·

>Like all models, Claude can be jailbroken, and our work to make Claude more helpful, harmless, and honest is ongoing. Is "harmless" a good metric to be judgin

>Like all models, Claude can be jailbroken, and our work to make Claude more helpful, harmless, and honest is ongoing. Is "harmless" a good metric to be judging an AI by? I find all of this "evil AI" doom and gloom stuff pathetic. Who's out there actually training models to do whatever they want without the nonsensical Care Bear ethical limitations? And moreso, do you think your model will keep up with the people who are doing that if you aren't? We are only hamstringing ourselves with this stuff. It's literally the equivalent of inventing the first digital calculator, and programming it not to display the results of 80084 + 1.

version_five 3y ago
There's a lot of vocal hangers on in the AI community that hijack the discussion towards "harms" and whatnot. Not to say it's not worth thinking about but it's run amok to the point that models are neutered (contrained within what a very specific political alignment calls ethics) before they're even used for anything, because someone speculates that somehow somewhere someone could do something bad with it. It's a dumb approach and hopefully as AI tools become more widely available and the activists move on, we can focus on building good AI and then putting guardrails in place for specific applications as required, rather than this "only the enlightened can be trusted with it" approach. Imagine if this is what had happened with general purpose computing. There are those that wish it had and see this as an opportunity to claw back some of their power.
flangola7 3y ago
It's not "doom and gloom" whatsoever, I have yet to see an internally coherent and persuasive argument that AI will not be the most powerful technology ever created. 95% of the "arguments" are little more than dismissive emotional outbursts and bold claims made with no evidence, much like the claims that airplanes and computers will never take off of yesteryear. The single largest problem of the tech industry all along is the pervasive and perverse "shoot first, ask questions never" attitude, especially for ethics. It's a breath of fresh air and a small relief to for once see industry leaders take the stance that potential consequences should not only be considered, but assessed and mitigated before ever breaking ground. When you're building a bridge you don't deploy a single construction truck until you have a very comprehensive plan that thoroughly theorizes, assesses, and mitigates the possible harms and failure modes. A bridge can harm far fewer people than a large internet entity, which is yet still less harmful than resourceful cognitive agents.
blueboo 3y ago
You’re conflating the doomers, with their apocalyptic forecasts, with the more prosaic concerns of measuring and controlling the behaviour of the models outputs. It’s reasonable to measure and control tendencies to counsel people into hurting themselves or others. It’s also reasonable to measure and control the tendency to present harmful stereotypes as fact. Maybe these mitigations go too far for your taste. The power of these tools and the diversity of a mass audience indicate a thoughtful process.
- cubefox 3y ago
  To present the claim that certain things are "harmful stereotypes" as fact is a popular tactic of activist censors. Not even articles in scientific journals are safe when they arrive at the "wrong" results. Optimizing language models for political correctness instead of truth, whatever it may be, teaches them lying and deception.
  - shoemakersteve 3y ago
    Seems like you're implying that harmful stereotypes either: don't exist, aren't actually harmful, or are the "truth"? If, when given the text "[MASK] is a female job", a language model only fills in "CEO" a fraction of the time compared to what it would for "male", is that "the truth" because male CEOs vastly outnumber female ones? I would say no, because we're not actually saying anything about gender ratios in that text. And it's not that there isn't some form of truth in that output. In a pure mathematical sense, you're just more likely to see the word "CEO" associated to males. That's true. But what if I'm using it for something other than predicting text? At that point I don't think it's too hard to see how this could have farther reaching downstream impacts that could have negative effects. If I want to use it for assessing potential candidates for hiring, is it "teaching it to lie" if I train it to reduce or eliminate any gender or racial bias so that it doesn't potentially screen out the best candidates? I can't say I like that chatgpt says "sorry I'm a robot" for even mild things, but it might be good too understand that that's a totally different issue. Mostly a PR one. They don't want to be in the news because people keep having it write essays about how great eugenics is. I wouldn't worry too much about it though, there are already uncensored LLMs you can spin up yourself so commercial products will likely follow soon enough.

version_five 3y ago

There's a lot of vocal hangers on in the AI community that hijack the discussion towards "harms" and whatnot. Not to say it's not worth thinking about but it's run amok to the point that models are neutered (contrained within what a very specific political alignment calls ethics) before they're even used for anything, because someone speculates that somehow somewhere someone could do something bad with it. It's a dumb approach and hopefully as AI tools become more widely available and the activists move on, we can focus on building good AI and then putting guardrails in place for specific applications as required, rather than this "only the enlightened can be trusted with it" approach. Imagine if this is what had happened with general purpose computing. There are those that wish it had and see this as an opportunity to claw back some of their power.

flangola7 3y ago

It's not "doom and gloom" whatsoever, I have yet to see an internally coherent and persuasive argument that AI will not be the most powerful technology ever created. 95% of the "arguments" are little more than dismissive emotional outbursts and bold claims made with no evidence, much like the claims that airplanes and computers will never take off of yesteryear. The single largest problem of the tech industry all along is the pervasive and perverse "shoot first, ask questions never" attitude, especially for ethics. It's a breath of fresh air and a small relief to for once see industry leaders take the stance that potential consequences should not only be considered, but assessed and mitigated before ever breaking ground. When you're building a bridge you don't deploy a single construction truck until you have a very comprehensive plan that thoroughly theorizes, assesses, and mitigates the possible harms and failure modes. A bridge can harm far fewer people than a large internet entity, which is yet still less harmful than resourceful cognitive agents.

blueboo 3y ago

You’re conflating the doomers, with their apocalyptic forecasts, with the more prosaic concerns of measuring and controlling the behaviour of the models outputs. It’s reasonable to measure and control tendencies to counsel people into hurting themselves or others. It’s also reasonable to measure and control the tendency to present harmful stereotypes as fact. Maybe these mitigations go too far for your taste. The power of these tools and the diversity of a mass audience indicate a thoughtful process.

cubefox 3y ago
To present the claim that certain things are "harmful stereotypes" as fact is a popular tactic of activist censors. Not even articles in scientific journals are safe when they arrive at the "wrong" results. Optimizing language models for political correctness instead of truth, whatever it may be, teaches them lying and deception.
- shoemakersteve 3y ago
  Seems like you're implying that harmful stereotypes either: don't exist, aren't actually harmful, or are the "truth"? If, when given the text "[MASK] is a female job", a language model only fills in "CEO" a fraction of the time compared to what it would for "male", is that "the truth" because male CEOs vastly outnumber female ones? I would say no, because we're not actually saying anything about gender ratios in that text. And it's not that there isn't some form of truth in that output. In a pure mathematical sense, you're just more likely to see the word "CEO" associated to males. That's true. But what if I'm using it for something other than predicting text? At that point I don't think it's too hard to see how this could have farther reaching downstream impacts that could have negative effects. If I want to use it for assessing potential candidates for hiring, is it "teaching it to lie" if I train it to reduce or eliminate any gender or racial bias so that it doesn't potentially screen out the best candidates? I can't say I like that chatgpt says "sorry I'm a robot" for even mild things, but it might be good too understand that that's a totally different issue. Mostly a PR one. They don't want to be in the news because people keep having it write essays about how great eugenics is. I wouldn't worry too much about it though, there are already uncensored LLMs you can spin up yourself so commercial products will likely follow soon enough.