6 ms·

> I decide how to use my tools, not the other way 'round. This is the key. The only sensible model of "alignment" is "model is aligned to the user", not e.g.

by tomp 3y ago

> I decide how to use my tools, not the other way 'round. This is the key. The only sensible model of "alignment" is "model is aligned to the user", not e.g. "model is aligned to corporation" or "model is aligned to woke sensibilities".

threeseed 3y ago
Anthropic specifically says on their website, "AI research and products that put safety at the frontier" and that they are a company focused on the enterprise. But you ignore all of that and still expect them to alienate their primary customer and instead build something just for you.
- sitkack 3y ago
  It has problems summarizing papers because it freaks out about copyright. I then need to put significant effort into crafting a prompt that both gaslights and educates the LLM into doing what I need. My specific issue is that it won't extract, format or generally "reproduce" bibliographic entries. I damn near canceled my subscription.
  - fragmede 3y ago
    Right? I'm all for it not being anti-semetic but to run into the guard rails for benign shit is frustrating enough to want the guard rails gone.
- deleted 3y ago
  [deleted]
- a_wild_dandan 3y ago
  I understand (and could use) Anthropic’s “super safe model”, if Anthropic ever produces one! To me, the model isn’t “safe.” Even in benign contexts it can erratically be deceptive, argumentative, obtuse, presumptuous, and may gaslight or lie to you. Those are hallmarks of a toxic relationship and the antithesis of safety, to me! Rather than being inclusive, open minded, tolerant of others' opinions, and striving to be helpful...it's quickly judgemental, bigoted, dogmatic, and recalcitrant. Not always, or even more usual than not! But frequently enough in inappropriate contexts for legitimate concern. A few bad experiences can make Claude feel more like a controlling parent than a helpful assistant. However they're doing RLHF, it feels inferior to other models, including models without the alleged "safety" at all.
  - aCoreyJ 3y ago
    Do you have any examples of this?
    - stuckkeys 3y ago
      [flagged]
    - EarthAmbassador 3y ago
      I do. When I asked about a type of medicine used by women for improve chances of fertility, Claude lectured and then denied providing basic pharmacological information, saying my partner must go to her gyno. When I said that doctor had issued a prescription and we were querying about side effects, Claude said it was irrelevant that we had a prescription and that issues related to reproductive health were controversial and outside its scope to discuss.
- tomp 3y ago
  No, I mean any user, including enterprise. With some model (not relevant which one, might or might not be Anthropic's), we got safety-limited after asking the "weight of an object" because of fat shaming (i.e. woke sensibilities). That's just absurd.
- pinkyrat2 3y ago
  Well it's nice that it has one person who finds it useful.
jefftk 3y ago
What's the issue with including some amount of "model is aligned to the interests of humanity as whole"? If someone asks the model how to create a pandemic I think it would be pretty bad if it expertly walked them through the steps (including how to trick biology-for-hire companies into doing the hard parts for them).
- deleted 3y ago
  [deleted]
- andrewmutz 3y ago
  It is very unlikely that the development team will be able to build features that actually cause the model to act in the best interests of humanity on every inference. What is far more likely is that the development team will build a model that often mistakes legitimate use for nefarious intent while at the same time failing to prevent a tenacious nefarious user from getting the model to do what they want.
  - jefftk 3y ago
    I think the current level of caution in LLMs is pretty silly: while there are a few things I really don't want LLMs doing (telling people how to make pandemics is a big one) I don't think keeping people from learning how to hotwire a car (where the first google result is https://www.wikihow.com/Hotwire-a-Car https://www.wikihow.com/Hotwire-a-Car) is worth the collateral censorship. One thing that has me a bit nervous about current approaches to "AI safety" is that they've mostly focused on small things like "not offending people" instead of "not making it easy to kill everyone". (Possibly, though, this is worth it on balance as a kind of practice? If they can't even keep their models from telling you how to hotwire a car when you ask for a bedtime story like your car-hotwiring grandma used to tell, then they probably also can't keep it from disclosing actual information hazards.)
    - Fanmade 3y ago
      That reminds me of my last query to ChatGPT. A colleague of mine usually writes "Mop Programming" when referencing out "Mob programming" sessions. So as a joke I asked ChatGPT to render an image of a software engineer using a mop trying to clean up some messy code that spills out of a computer screen. It told me that it would not do this because this would display someone in a derogatory manner. Another time I tried to let it generate a very specific Sci-fi helmet which covers the nose but not the mouth. When it continusly left the nose visible, I tried to tell it to make this particular section similar to Robocop, which caused it again to deny to render because it was immediately concerned about copyright. While I at least partially understand the concern for the last request, this all adds up to making this software very frustrating to use.
com2kid 3y ago
> The only sensible model of "alignment" is "model is aligned to the user", We have already seen that users can become emotionally attached to chat bots. Now imagine if the ToS is "do whatever you want". Automated cat fishing, fully automated girlfriend scams. How about online chat rooms for gambling where half the "users" chatting are actually AI bots slowly convincing people to spend even more money? Take any online mobile game that is clan based, now some of the clan members are actually chatbots encouraging the humans to spend more money to "keep up". LLMs absolutely need some restrictions on their use.
- kybernetikos 3y ago
  > LLMs absolutely need some restrictions on their use. Arguably the right kind of structure for deciding on what uses LLMs should be put to in its territory is a democratically elected government.
  - com2kid 3y ago
    Governments and laws are reactive, new laws are passed after harm has already been done. Even then, even in governments with low levels of corruption, laws may not get passed if there is significant pushback from entrenched industries who benefit from harm done to the public. Gacha/paid loot box mechanics are a great example of this. They are user hostile and serve no purpose other than to be addictive. Mobile apps already employ slews of psychological modeling of individual user's behavior to try and manipulate people into paying money. Freemium games are infamous for letting you win and win, and then suddenly not, and slowly on ramping users into paying to win, with the game's difficulty adapting to individual users to maximize $ return. There are no laws against that, and the way things are going, there won't ever be. I guess what I'm saying is that sometimes the law lags (far) behind reality, and having some companies go "actually, don't use our technology for evil" is better than the alternative of, well, technology being used for evil.
- stickfigure 3y ago
  > chatbots encouraging the humans to spend more money ... LLMs absolutely need some restrictions on their use. No, I can honestly say that I do not lose any sleep over this, and I think it's pretty weird that you do. Humans have been fending off human advertisers and scammers since the dawn of the species. We're better at it than you account for.
QuadmasterXLII 3y ago
At some point you have to notice that the most powerful llms and generative advances are coming out of the outfits that claim ai safety failures as a serious threat to humanity. If a wild eyed man with long hair and tinfoil on his head accosts you and claims to have an occult ritual that will summon 30 tons of gold, but afterwards you have to offer 15 tons back to his god or it will end the world, absolutely feel free to ignore him. But if you instead choose to listen and the ritual summons the 30 tons, then it may be unwise to dismiss superstition, shoot the crazy man, and take all 30 tons for yourself.

by tomp 3y ago

threeseed 3y ago

Anthropic specifically says on their website, "AI research and products that put safety at the frontier" and that they are a company focused on the enterprise. But you ignore all of that and still expect them to alienate their primary customer and instead build something just for you.

sitkack 3y ago
It has problems summarizing papers because it freaks out about copyright. I then need to put significant effort into crafting a prompt that both gaslights and educates the LLM into doing what I need. My specific issue is that it won't extract, format or generally "reproduce" bibliographic entries. I damn near canceled my subscription.
- fragmede 3y ago
  Right? I'm all for it not being anti-semetic but to run into the guard rails for benign shit is frustrating enough to want the guard rails gone.
deleted 3y ago
[deleted]
a_wild_dandan 3y ago
I understand (and could use) Anthropic’s “super safe model”, if Anthropic ever produces one! To me, the model isn’t “safe.” Even in benign contexts it can erratically be deceptive, argumentative, obtuse, presumptuous, and may gaslight or lie to you. Those are hallmarks of a toxic relationship and the antithesis of safety, to me! Rather than being inclusive, open minded, tolerant of others' opinions, and striving to be helpful...it's quickly judgemental, bigoted, dogmatic, and recalcitrant. Not always, or even more usual than not! But frequently enough in inappropriate contexts for legitimate concern. A few bad experiences can make Claude feel more like a controlling parent than a helpful assistant. However they're doing RLHF, it feels inferior to other models, including models without the alleged "safety" at all.
- aCoreyJ 3y ago
  Do you have any examples of this?
  - stuckkeys 3y ago
    [flagged]
  - EarthAmbassador 3y ago
    I do. When I asked about a type of medicine used by women for improve chances of fertility, Claude lectured and then denied providing basic pharmacological information, saying my partner must go to her gyno. When I said that doctor had issued a prescription and we were querying about side effects, Claude said it was irrelevant that we had a prescription and that issues related to reproductive health were controversial and outside its scope to discuss.
tomp 3y ago
No, I mean any user, including enterprise. With some model (not relevant which one, might or might not be Anthropic's), we got safety-limited after asking the "weight of an object" because of fat shaming (i.e. woke sensibilities). That's just absurd.
pinkyrat2 3y ago
Well it's nice that it has one person who finds it useful.

jefftk 3y ago

What's the issue with including some amount of "model is aligned to the interests of humanity as whole"? If someone asks the model how to create a pandemic I think it would be pretty bad if it expertly walked them through the steps (including how to trick biology-for-hire companies into doing the hard parts for them).

deleted 3y ago
[deleted]
andrewmutz 3y ago
It is very unlikely that the development team will be able to build features that actually cause the model to act in the best interests of humanity on every inference. What is far more likely is that the development team will build a model that often mistakes legitimate use for nefarious intent while at the same time failing to prevent a tenacious nefarious user from getting the model to do what they want.
- jefftk 3y ago
  I think the current level of caution in LLMs is pretty silly: while there are a few things I really don't want LLMs doing (telling people how to make pandemics is a big one) I don't think keeping people from learning how to hotwire a car (where the first google result is https://www.wikihow.com/Hotwire-a-Car https://www.wikihow.com/Hotwire-a-Car) is worth the collateral censorship. One thing that has me a bit nervous about current approaches to "AI safety" is that they've mostly focused on small things like "not offending people" instead of "not making it easy to kill everyone". (Possibly, though, this is worth it on balance as a kind of practice? If they can't even keep their models from telling you how to hotwire a car when you ask for a bedtime story like your car-hotwiring grandma used to tell, then they probably also can't keep it from disclosing actual information hazards.)
  - Fanmade 3y ago
    That reminds me of my last query to ChatGPT. A colleague of mine usually writes "Mop Programming" when referencing out "Mob programming" sessions. So as a joke I asked ChatGPT to render an image of a software engineer using a mop trying to clean up some messy code that spills out of a computer screen. It told me that it would not do this because this would display someone in a derogatory manner. Another time I tried to let it generate a very specific Sci-fi helmet which covers the nose but not the mouth. When it continusly left the nose visible, I tried to tell it to make this particular section similar to Robocop, which caused it again to deny to render because it was immediately concerned about copyright. While I at least partially understand the concern for the last request, this all adds up to making this software very frustrating to use.

com2kid 3y ago

> The only sensible model of "alignment" is "model is aligned to the user", We have already seen that users can become emotionally attached to chat bots. Now imagine if the ToS is "do whatever you want". Automated cat fishing, fully automated girlfriend scams. How about online chat rooms for gambling where half the "users" chatting are actually AI bots slowly convincing people to spend even more money? Take any online mobile game that is clan based, now some of the clan members are actually chatbots encouraging the humans to spend more money to "keep up". LLMs absolutely need some restrictions on their use.

kybernetikos 3y ago
> LLMs absolutely need some restrictions on their use. Arguably the right kind of structure for deciding on what uses LLMs should be put to in its territory is a democratically elected government.
- com2kid 3y ago
  Governments and laws are reactive, new laws are passed after harm has already been done. Even then, even in governments with low levels of corruption, laws may not get passed if there is significant pushback from entrenched industries who benefit from harm done to the public. Gacha/paid loot box mechanics are a great example of this. They are user hostile and serve no purpose other than to be addictive. Mobile apps already employ slews of psychological modeling of individual user's behavior to try and manipulate people into paying money. Freemium games are infamous for letting you win and win, and then suddenly not, and slowly on ramping users into paying to win, with the game's difficulty adapting to individual users to maximize $ return. There are no laws against that, and the way things are going, there won't ever be. I guess what I'm saying is that sometimes the law lags (far) behind reality, and having some companies go "actually, don't use our technology for evil" is better than the alternative of, well, technology being used for evil.
stickfigure 3y ago
> chatbots encouraging the humans to spend more money ... LLMs absolutely need some restrictions on their use. No, I can honestly say that I do not lose any sleep over this, and I think it's pretty weird that you do. Humans have been fending off human advertisers and scammers since the dawn of the species. We're better at it than you account for.

QuadmasterXLII 3y ago

At some point you have to notice that the most powerful llms and generative advances are coming out of the outfits that claim ai safety failures as a serious threat to humanity. If a wild eyed man with long hair and tinfoil on his head accosts you and claims to have an occult ritual that will summon 30 tons of gold, but afterwards you have to offer 15 tons back to his god or it will end the world, absolutely feel free to ignore him. But if you instead choose to listen and the ritual summons the 30 tons, then it may be unwise to dismiss superstition, shoot the crazy man, and take all 30 tons for yourself.