5 ms·

From the slide deck on the livestream: "[Operator safety risks and mitigations] Harmful tasks: User is misaligned" Looking forward to seeing some more of the

From the slide deck on the livestream: "[Operator safety risks and mitigations] Harmful tasks: User is misaligned" Looking forward to seeing some more of the examples for when openai considers their users as "misaligned", whatever that actually even means anymore.

tedsanders 2y ago
I assume here it means complying with requests that could harm other people. It's pretty common for businesses to tell their employees not to assist customers doing bad things, so not surprised to see AIs trained to not to assist customers doing bad things. Examples: - "operator, please sign up for 100 fake Reddit accounts and have them regularly make posts praising product X." - "operator, please order the components need to make a high-yield bomb." - "operator, please go harass my ex on Instagram"
- madeofpalk 2y ago
  "operator, please perform this computationally expensive action on my competitors website 1000000 times"
- hammock 2y ago
  Isn't that reddit/home depot/instagram's problem? Not a job for the guy you hired to do a thing
  - bilbo0s 2y ago
    If it makes you feel any better, law enforcement makes sure reddit, Home Depot, and instagram are "aligned" as well. Don't worry though, it's all on the up and up. No backdoors or google-like search facilities our anything like that. It's not at all automated in that sort of unseemly fashion. They always go to court. Where they talk to a judge, that they totally don't go golfing with, and ask them for a warrant for the data they found on the instagram/home depot/reddit systems. Oh wait, no, I mean, a warrant to try to find data on the instagram/home depot/reddit systems. /s
  - jsheard 2y ago
    It's OpenAIs problem if sites start throttling/challenging/blocking their agent traffic in response to abuse.
- swatcoder 2y ago
  It's pretty troubling and illiberal to use the same word for a software tool being constrained by its manufacturer's moral framework and for a human user being constrained to that manufacturer's moral framework. While you can see how the word is formally valid and analogous in both cases, the connotation is that the user is being judged by the moral standards of a commercial vendor, which is about as Cyberpunk Dystopian as you can get.
  - easterncalculus 2y ago
    This is putting it in better words than I came up with myself.
  - dumah 2y ago
    Being restricted from doing crimes by a vendor of commercial software isn’t a cyberpunk dystopia. Buy or download something else. It’s a typical restriction of software terms of service to prohibit use outside of applicable laws and regulations.
    - lolinder 2y ago
      If "alignment" were just about crimes we wouldn't need a special word for it, we would just say "legal". Alignment is not just about crimes, it's about the AI behaving in a way that is very specifically tailored to the moral framework and practical needs of the creator. Alignment has always gone well beyond the minimum required by law. And I don't think anyone is saying that a piece of software refusing to behave in a way that the creator doesn't want is a cyberpunk dystopia, they're saying that calling the user themselves misaligned is horrifying.
      - kridsdale3 2y ago
        I agree. But I had a devils-advocate moment. Hacking in Counter Strike to have perfect aim and see your opponents through walls is legal. You aren't violating some anti-computer-misuse statute like DMCA. But Valve has every right to call the users of those scripts assholes who ruin the game for everyone and to ban them.
- jfengel 2y ago
  I appreciate that they all say please.
darioush 2y ago
As the storyline unfolds "AI" seems to be code for "machine learning based censorship". Soon we will have home appliances and vehicles telling you about how aligned you are, and whether you need to improve your alignment score before you can open your fridge. It is only a matter of time before this will apply to your financial transactions as well.
- mattstir 2y ago
  I can sympathize with vague notions of AI dystopia, but this might be stretching the concept a bit too far. This kind of service is extremely abusable ("Operator, go to Wikipedia and start mass-vandalizing articles" or "Go to this website and try these people's email addresses with random passwords until it locks their accounts") and building some alignment goals into it doesn't seem like a terribly draconian idea. Also, if you were under the impression that machine-learned (or otherwise) restrictions aren't already applied to purchases made with your cards, you're in for an unfortunate bit of news there as well.
  - darioush 2y ago
    You can also write a python script to achieve the same goals. Except it's not python's responsibility to interpret the intent of your script, just as it's not your phone's responsibility to interpret the contents of your conversation. So our tools are not our morality police. We have a legal system that can operate within the bounds of law and due process. I am well aware of the already applied levels of machine learning policing, I am just not very excited that society has decided that "this is the way now", and also doesn't seem to be bothered by the environmental costs of building and running all these GPUs (which does seem to be the case when they are used for censorship resistant transactions), or the ethical concerns about a non-profit becoming a for-profit etc.
    - infecto 2y ago
      The difference being you would be running that python script yourself. If you by chance hosted it somewhere there is high probability that the host would get a notice and shut you down. I honestly don't see much difference here. There will be multiple providers and perhaps great ways to run these types of tools locally, all have different risk measures.
fassssst 2y ago
As an analogy, Americans are allowed to buy guns but they’re not allowed to do whatever they want with them. An agent on the internet could be used for more harm than a gun.
moffkalast 2y ago
OAI has decided to stop aligning models and focus on aligning the users instead.
- TeMPOraL 2y ago
  "Society is fixed, biology is mutable", but taken to the extreme?
  - incognito124 2y ago
    First time hearing about it, nice read
deleted 2y ago
[deleted]
csours 2y ago
Myeaah, we need to fix that misalignment. --- Private, you better realign yourself in the next 60 seconds! --- So sorry, your alignment score seems to be too low for this promotion. --- Citizens, peacefully disperse and align yourselves.
throwaway123128 2y ago
The Nazis called it "Gleichschaltung". Same principle, different applications.
kandesbunzler 2y ago
If someone used this service to do really bad things then you morons would cry too so they cant really win either way with you people
kridsdale3 2y ago
> Looking forward to seeing some more of the examples for when openai considers their users as "misaligned" All humans with politics not aligned with "The median sentiment of the San Francisco Board of Supervisors"

tedsanders 2y ago

I assume here it means complying with requests that could harm other people. It's pretty common for businesses to tell their employees not to assist customers doing bad things, so not surprised to see AIs trained to not to assist customers doing bad things. Examples: - "operator, please sign up for 100 fake Reddit accounts and have them regularly make posts praising product X." - "operator, please order the components need to make a high-yield bomb." - "operator, please go harass my ex on Instagram"

madeofpalk 2y ago
"operator, please perform this computationally expensive action on my competitors website 1000000 times"
hammock 2y ago
Isn't that reddit/home depot/instagram's problem? Not a job for the guy you hired to do a thing
- bilbo0s 2y ago
  If it makes you feel any better, law enforcement makes sure reddit, Home Depot, and instagram are "aligned" as well. Don't worry though, it's all on the up and up. No backdoors or google-like search facilities our anything like that. It's not at all automated in that sort of unseemly fashion. They always go to court. Where they talk to a judge, that they totally don't go golfing with, and ask them for a warrant for the data they found on the instagram/home depot/reddit systems. Oh wait, no, I mean, a warrant to try to find data on the instagram/home depot/reddit systems. /s
- jsheard 2y ago
  It's OpenAIs problem if sites start throttling/challenging/blocking their agent traffic in response to abuse.
swatcoder 2y ago
It's pretty troubling and illiberal to use the same word for a software tool being constrained by its manufacturer's moral framework and for a human user being constrained to that manufacturer's moral framework. While you can see how the word is formally valid and analogous in both cases, the connotation is that the user is being judged by the moral standards of a commercial vendor, which is about as Cyberpunk Dystopian as you can get.
- easterncalculus 2y ago
  This is putting it in better words than I came up with myself.
- dumah 2y ago
  Being restricted from doing crimes by a vendor of commercial software isn’t a cyberpunk dystopia. Buy or download something else. It’s a typical restriction of software terms of service to prohibit use outside of applicable laws and regulations.
  - lolinder 2y ago
    If "alignment" were just about crimes we wouldn't need a special word for it, we would just say "legal". Alignment is not just about crimes, it's about the AI behaving in a way that is very specifically tailored to the moral framework and practical needs of the creator. Alignment has always gone well beyond the minimum required by law. And I don't think anyone is saying that a piece of software refusing to behave in a way that the creator doesn't want is a cyberpunk dystopia, they're saying that calling the user themselves misaligned is horrifying.
    - kridsdale3 2y ago
      I agree. But I had a devils-advocate moment. Hacking in Counter Strike to have perfect aim and see your opponents through walls is legal. You aren't violating some anti-computer-misuse statute like DMCA. But Valve has every right to call the users of those scripts assholes who ruin the game for everyone and to ban them.
jfengel 2y ago
I appreciate that they all say please.

darioush 2y ago

As the storyline unfolds "AI" seems to be code for "machine learning based censorship". Soon we will have home appliances and vehicles telling you about how aligned you are, and whether you need to improve your alignment score before you can open your fridge. It is only a matter of time before this will apply to your financial transactions as well.

mattstir 2y ago
I can sympathize with vague notions of AI dystopia, but this might be stretching the concept a bit too far. This kind of service is extremely abusable ("Operator, go to Wikipedia and start mass-vandalizing articles" or "Go to this website and try these people's email addresses with random passwords until it locks their accounts") and building some alignment goals into it doesn't seem like a terribly draconian idea. Also, if you were under the impression that machine-learned (or otherwise) restrictions aren't already applied to purchases made with your cards, you're in for an unfortunate bit of news there as well.
- darioush 2y ago
  You can also write a python script to achieve the same goals. Except it's not python's responsibility to interpret the intent of your script, just as it's not your phone's responsibility to interpret the contents of your conversation. So our tools are not our morality police. We have a legal system that can operate within the bounds of law and due process. I am well aware of the already applied levels of machine learning policing, I am just not very excited that society has decided that "this is the way now", and also doesn't seem to be bothered by the environmental costs of building and running all these GPUs (which does seem to be the case when they are used for censorship resistant transactions), or the ethical concerns about a non-profit becoming a for-profit etc.
  - infecto 2y ago
    The difference being you would be running that python script yourself. If you by chance hosted it somewhere there is high probability that the host would get a notice and shut you down. I honestly don't see much difference here. There will be multiple providers and perhaps great ways to run these types of tools locally, all have different risk measures.

fassssst 2y ago

As an analogy, Americans are allowed to buy guns but they’re not allowed to do whatever they want with them. An agent on the internet could be used for more harm than a gun.

moffkalast 2y ago

OAI has decided to stop aligning models and focus on aligning the users instead.

TeMPOraL 2y ago
"Society is fixed, biology is mutable", but taken to the extreme?
- incognito124 2y ago
  First time hearing about it, nice read

deleted 2y ago

[deleted]

csours 2y ago

Myeaah, we need to fix that misalignment. --- Private, you better realign yourself in the next 60 seconds! --- So sorry, your alignment score seems to be too low for this promotion. --- Citizens, peacefully disperse and align yourselves.

throwaway123128 2y ago

The Nazis called it "Gleichschaltung". Same principle, different applications.

kandesbunzler 2y ago

If someone used this service to do really bad things then you morons would cry too so they cant really win either way with you people

kridsdale3 2y ago

> Looking forward to seeing some more of the examples for when openai considers their users as "misaligned" All humans with politics not aligned with "The median sentiment of the San Francisco Board of Supervisors"