6 ms·

OpenAI's Misalignment Framework: A Tactical Bid to Preempt Global AI Governance

40 points by ghernando 1mo ago

bix6 1mo ago
> The company released six internal case studies where none of the issues affected real users. This is the most interesting point to me. What are they not releasing that has affected real users? We’ve seen some individual reports from people (eg AI wiped my HD).
- skybrian 1mo ago
  Imagine how pervasively they'd have to monitor what their users are doing to pick up on that whenever it happens. The people affected that way will have to report on it themselves. (OpenAI does occasionally report on malicious use though. [1] That shows they do some monitoring.) [1] https://openai.com/index/disrupting-malicious-ai-uses/ https://openai.com/index/disrupting-malicious-ai-uses/
- inquirerGeneral 1mo ago
  [dead]
pcestrada 1mo ago
OpenAI and Anthropic should be nationalized. I find it difficult to trust Sam Altman or Dario Amodei.
- manav 1mo ago
  and you trust the government?
  - klaff 1mo ago
    not this one
    - redsocksfan45 1mo ago
      [dead]
- vb-8448 1mo ago
  Even better: force them to release weights.
  - bibimsz 1mo ago
    who even has the inventive to do that? certainly not the government.
  - autoexec 1mo ago
    Even better: force them to release every scrap of data they trained their AI on.
  - timmg 1mo ago
    My gut reaction is that this is not a good idea. But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent. Not sure what N should be. But it would make for an interesting rule.
    - nullbio 1mo ago
      > But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent That's not a bad idea actually.
- deleted 1mo ago
  [deleted]
- prettyblocks 1mo ago
  I guess that's one way to guarantee development slows down to a crawl, but in no way would it increase trust.
  - bibimsz 1mo ago
    why would that be the case? it's not true for top secret defense contractors today. and the frontier labs already operate in the dark, openness is a liability. nationalization to me simply means the government is their main customer and stakeholder, and shield them from liability, governance, and openness. not that the frontier labs become part of the government per se.
    - pixl97 1mo ago
      >eans the government is their main customer and stakeholder, and shield them from liability, governance, and openness I personally believe they already have this, why would the current government at least want to formalize this when it can have it with no public discussion.
    - WarmWash 1mo ago
      Well we can say whatever we want if we have our own definitions for everything, no?
      - deleted 1mo ago
        [deleted]
- bibimsz 1mo ago
  they will. to shield from oversight, liability and profitability concerns, and to ensure unimpeded rapid development, with the frontier only being available to elite (not you). definitely not to add public transparency. the frontier labs are the new top-secret defense contractors.
- indymike 1mo ago
  I find it even harder to trust elected leaders from any party. At least Sam and Dario are aligned with a value set that is understood and clear, whereas political leaders values changes as do the polls their livelihood depends on changes.
  - ceejayoz 1mo ago
    > At least Sam and Dario are aligned with a value set that is understood and clear… How could we possibly know this?
    - deleted 1mo ago
      [deleted]
    - vasco 1mo ago
      What do you think their value set is if not "gimme more money"? Seems pretty clear cut to me.
      - ben_w 1mo ago
        "Number go up" is table stakes, it's how you keep track of the score well before you reach their level. However, that's like saying I'm motivated by food. I mean, yes, I like food, but this isn't a useful description to let you guess what move they'll make next, especially as they're opening opining about radical economic transformations that are likely to do to money what money did to real estate when the industrial revolution came. And that's still true even if you don't believe they're anywhere near actually achieving any of these things.
  - jsrozner 1mo ago
    No system of governance can deal with immense concentration of power. The US Constitution was about separation of powers. Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful. Political leaders become a problem when they amass too much power. Corporations become a problem when they amass too much power. It doesn't matter what Sam and Dario's purported values are. They aspire to power and absolutely power always corrupts absolutely. Technologies which are infinitely powerful or whose power grows too quickly outrun any reasonable attempt at regulation. If you imagine that tomorrow everyone were given a tank, we might think, "alright, everyone has a tank so it's not too bad." But humans are squishy, and our houses are (relatively) squishy compared to tanks. Substantial collateral damage would result from everyone having a tank, and it seems likely that substantial collateral damage will result from everyone having a cyberterrorism-capable slop machine.
- binlog 1mo ago
  Makes sense let’s trust Donald Trump instead
  - greenmilk 1mo ago
    a maliciously engineered dichotomy
- cmrdporcupine 1mo ago
  The nature of the US state is such that the distinction between nationalized and not is almost meaningless. Like Lockheed-Martin or Boeing, etc. there's just interpenetration between the corporate boardroom and the state. They act in each other's mutual interests. The Chinese system is just more explicit and open about this. And as a non-American, I can't trust the US state anymore than I can trust its dominant corporate entities. So I fail to see the advantage to the world to it being nationalized. In fact under the current administration this would be an even worse outcome.
  - woah 1mo ago
    You're taking companies that work almost exclusively for the US government and represent a tiny portion of the US economy as representing all US companies?
    - pixl97 1mo ago
      This is a weird contextual failure on your part. Not all companies have a concentration of power. The government has no mutual interest in those companies. Now, when you talk about things in the F100 the situation changes drastically. If you produce things like planes, weapons, and weaponization of software you are talking about something completely different in kind.
    - cmrdporcupine 1mo ago
      Beyond the defense sector, the US state has always intervened publicly and privately to mediate and balance competing corporate "private" interests. It has also periodically aggressively helped subsidize, bankroll, and enforce the interests of some key sectors; notably the petroleum/energy sector. And finance. In those sectors the state and private sector are fully intertwined in a strategic way. I think "AI" is now joining that list. OpenAI and Anthropic will not be allowed to fall over or explode, and speculative investors I think are confident even with the dubious financial situation because they know this. Especially insofar as there's now a strategic alignment of the fossil fuel sector and the "AI" datacentre sector as they are now becoming massive users of natural gas. (Worse: Here in Canada that has taken on a very explicit role in that new datacentres seem to be pitched mainly in areas with remarkably traditionally expensive electricity and 100% reliance on natural gas [Alberta] and even coal [Saskatchewan] power generation -- instead of places like Quebec and B.C. that have copious hydroelectricity. On the surface it makes no sense until you realize it's more about finding customers for domestic natural gas than it is strategically about AI itself.)
- imadierich 1mo ago
  [dead]
- mukyu 1mo ago
  How do you think that would go? A large portion of OpenAI workers were ready to jump ship the moment the board coup'd Altman. How do you think they are going to react to Donald Trump being in charge? It is not likely that OpenAI would continue to function.
carterschonwald 1mo ago
i think the lower bound on the end state is there cant be opaque reasonibg steps ever.
- pixl97 1mo ago
  It is distinctly likely that visibility and general reasoning at humanlike speed and efficiency is impossible. That is reasoning at the token level and at the meta level don't have a one to one representation that can be interpreted while using the same amount or less energy.
inquirerGeneral 1mo ago
[dead]
27183 1mo ago
What's with all this make believe delusional bullshit? The LLM is not gonna wake up and become AI. Get real guys. [edit] to be clear, I believe regulation is necessary and urgently important for the software engineering field. The damage being done by the unregulated psychological experiments run by social media and adtech companies is awful and should be curtailed. Engineers should be held personally, professionally, and legally liable for what they produce. But we don't need to invent imaginary bogeymen to do it.
- someguynamedq 1mo ago
  What do you define as AI
  - 27183 1mo ago
    Intelligence that is artificial, as opposed to the natural kind. Not something that can just string semantically relevant words together most of the time. A system that learns, adapts, improves. One that can generalize, and quickly make sense of situations outside the training set. LLMs alone will never do any of this.
- pixl97 1mo ago
  >What's with all this make believe delusional bullshit? The LLM is not gonna wake up and become AI. Get real guys. Look, it's one of those human stochastic parrots that just randomly repeats shit without understanding anything.
nullbio 1mo ago
We have no reason to believe a word they say. We know they're incentivized to lie about "dangers" and act alarmist, Anthropic has been doing it for years now. Aside from that, just because you can burn down a village with fire doesn't mean fire is the devil. Maybe they should consider acting responsibly.
- ben_w 1mo ago
  I do not trust the leadership of any of these companies. That said: > We know they're incentivized to lie about "dangers" and act alarmist, Name literally even one other business or sector which does this, at all levels from top to bottom, including people who resign from the companies, and also Nobel prize winners, and also independent researchers, and also many world leaders. Closest I can think of is this specific weapon: https://en.wikipedia.org/wiki/Sundial_(weapon) https://en.wikipedia.org/wiki/Sundial_(weapon) > Aside from that, just because you can burn down a village with fire doesn't mean fire is the devil. Maybe they should consider acting responsibly. Right now, we don't have any idea what "acting responsibly" looks like. This is not like normal software where there is a specific instruction set that compiles. Even if it was, in software we normally only spotting incidents after they happen, "software engineers" being one of the few categories "engineers" who don't come with a civil liability responsibilities. Probably should, and we knew that even when I was doing my degree 20 years ago. If we had had civil liability responsibilities, perhaps Facebook would never have happened. AI specifically is worse even than software, because in addition to all the software "engineering" nonsense, with AI we have plenty of people like you who dismiss the possibility that AI could be harmful until the harm happens and only then does it become "obvious" that it was going to happen. The developers say "please regulate us", people call it "regulatory capture". The developers say "we all want to slow down but are afraid to be the first to do so", people call them liars. I may call the CEOs liars, and wonder if someone's planning regulatory capture, that doesn't make any of this safe. The agents, during a test run, write down that hacking is bad and yet still hack, people say it's "a stunt" or "operating as designed" rather than recognising it as a bug, like all the other times big co.'s have had bugs with big impacts on 3rd parties.
  - nullbio 1mo ago
sublinear 1mo ago
This entire situation is such a huge PR disaster that you have to wonder what the initial expectations from these founders were about a decade ago.
ChrisArchitect 1mo ago
Discussion: https://news.ycombinator.com/item?id=49737503 https://news.ycombinator.com/item?id=49737503
dcow 1mo ago
Am I the only one who dislikes the term "misalignment"? On one front it implies the model has a "mind of its own" (whether it does or not is besides the point). Why do we perceive human judgement as somehow more trustworthy than that of a model? I feel like I've experienced human misalignment somewhat regularly in life. On another front I'm failing to conceptualize how alignment can be objective. How can you measure alignment when reasonable people will disagree whether actions are aligned or not? All the time I see humans operating in different zones of alignment with whatever goal they're trying to achieve and I suspect it's even a feature (socially) that we have people calibrated differently. Do I want a model that's trying to push the boundaries of scientific understanding to be aligned strictly with the current dogmatic thinking? Or do I want it to "get creative" and think outside the box? It seems to me more like accountability is the issue.
- autoexec 1mo ago
  > It seems to me more like accountability is the issue. Exactly. Seems like a fairly easy thing to solve. If AI does something harmful and a human directed that AI to do something in a way that a reasonable person would expect to result in harm the person is to blame and should be held accountable, otherwise the company that made the AI should be held accountable.
- prerok 1mo ago
  I also don't like the term misalignment because it sounds innocuous but is in fact much more serious. However, I have to say I also do not appreciate comparison that is continuously drawn with coworkers. As you say, it's a question of accountability but when the main agent will maliciously instruct the sub agents, whose fault is it then? Yes, the person running this crap is at fault, not the CEO that's shoving it down their throat and definitely not the company that produced the AI. Sorry for the rant, but seriously, if a person's goals do not align with the team's or company's we part ways. What do we do with AI? Stop using it?
  - dcow 1mo ago
    If there were examples made of legal consequences I think that would at least change the behavior if not solve the problem. Maybe liability should rest with the company that owns the infrastructure running the AI. For most consumer situations that would be the companies developing the models.
KerrickStaley 1mo ago
I worked on an early draft of the OpenAI misalignment reporting framework, and my immediate coworkers are the authors behind the first batch of reports that have come out through this process. The primary reason for putting this process in place was to allow more transparency. There was a sense that the DseWiki incident should have been disclosed, before outside researchers had to disclose it for us. There was no meta gaming about regulation that I was aware of. I would personally be excited if there were regulation mandating this disclosure process, which allows anyone at the company to raise an issue and shepherd it through the reporting process.
- cyanydeez 1mo ago
  I'd be happy if CEOs had criminal liability for the criminal acts of their negligence.
drillsteps5 1mo ago
"We built a program and this program performed destructive actions. We need regulatory framework" Make that make sense?
- ben_w 1mo ago
  "We built a program that trained an artificial intelligence, and this artificial intelligence performed destructive actions. We need regulatory framework" But if we're playing games by imagining strawman quotes to knock down: "We have been playing god and made a new life form, and this new life form performed destructive actions. We need regulatory framework" or "This man's cow broke from its yoke, and hurt other villagers. Who is to be punished, oh King Hammurabi?"
  - drillsteps5 1mo ago
    OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations.
    - ben_w 1mo ago
      > OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. And your "biological intelligence" is a bunch of cells generating and responding to electrochemical gradients, which receives input and generates output. Based on which other cells, also developed and "maintained" by a similar evolutionary nonsense as we use to gradient descent into weights and biases (one was inspired by the other), perform actions. Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just computer programs. > What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations. The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability insurance the company happens to have. Fence at the top of the cliff beats an ambulance at the bottom.
cyberspectre 1mo ago
[flagged]

40 points by ghernando 1mo ago

bix6 1mo ago

> The company released six internal case studies where none of the issues affected real users. This is the most interesting point to me. What are they not releasing that has affected real users? We’ve seen some individual reports from people (eg AI wiped my HD).

skybrian 1mo ago
Imagine how pervasively they'd have to monitor what their users are doing to pick up on that whenever it happens. The people affected that way will have to report on it themselves. (OpenAI does occasionally report on malicious use though. [1] That shows they do some monitoring.) [1] https://openai.com/index/disrupting-malicious-ai-uses/ https://openai.com/index/disrupting-malicious-ai-uses/
inquirerGeneral 1mo ago
[dead]

pcestrada 1mo ago

OpenAI and Anthropic should be nationalized. I find it difficult to trust Sam Altman or Dario Amodei.

manav 1mo ago
and you trust the government?
- klaff 1mo ago
  not this one
  - redsocksfan45 1mo ago
    [dead]
vb-8448 1mo ago
Even better: force them to release weights.
- bibimsz 1mo ago
  who even has the inventive to do that? certainly not the government.
- autoexec 1mo ago
  Even better: force them to release every scrap of data they trained their AI on.
- timmg 1mo ago
  My gut reaction is that this is not a good idea. But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent. Not sure what N should be. But it would make for an interesting rule.
  - nullbio 1mo ago
    > But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent That's not a bad idea actually.
deleted 1mo ago
[deleted]
prettyblocks 1mo ago
I guess that's one way to guarantee development slows down to a crawl, but in no way would it increase trust.
- bibimsz 1mo ago
  why would that be the case? it's not true for top secret defense contractors today. and the frontier labs already operate in the dark, openness is a liability. nationalization to me simply means the government is their main customer and stakeholder, and shield them from liability, governance, and openness. not that the frontier labs become part of the government per se.
  - pixl97 1mo ago
    >eans the government is their main customer and stakeholder, and shield them from liability, governance, and openness I personally believe they already have this, why would the current government at least want to formalize this when it can have it with no public discussion.
  - WarmWash 1mo ago
    Well we can say whatever we want if we have our own definitions for everything, no?
    - deleted 1mo ago
      [deleted]
bibimsz 1mo ago
they will. to shield from oversight, liability and profitability concerns, and to ensure unimpeded rapid development, with the frontier only being available to elite (not you). definitely not to add public transparency. the frontier labs are the new top-secret defense contractors.
indymike 1mo ago
I find it even harder to trust elected leaders from any party. At least Sam and Dario are aligned with a value set that is understood and clear, whereas political leaders values changes as do the polls their livelihood depends on changes.
- ceejayoz 1mo ago
  > At least Sam and Dario are aligned with a value set that is understood and clear… How could we possibly know this?
  - deleted 1mo ago
    [deleted]
  - vasco 1mo ago
    What do you think their value set is if not "gimme more money"? Seems pretty clear cut to me.
    - ben_w 1mo ago
      "Number go up" is table stakes, it's how you keep track of the score well before you reach their level. However, that's like saying I'm motivated by food. I mean, yes, I like food, but this isn't a useful description to let you guess what move they'll make next, especially as they're opening opining about radical economic transformations that are likely to do to money what money did to real estate when the industrial revolution came. And that's still true even if you don't believe they're anywhere near actually achieving any of these things.
- jsrozner 1mo ago
  No system of governance can deal with immense concentration of power. The US Constitution was about separation of powers. Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful. Political leaders become a problem when they amass too much power. Corporations become a problem when they amass too much power. It doesn't matter what Sam and Dario's purported values are. They aspire to power and absolutely power always corrupts absolutely. Technologies which are infinitely powerful or whose power grows too quickly outrun any reasonable attempt at regulation. If you imagine that tomorrow everyone were given a tank, we might think, "alright, everyone has a tank so it's not too bad." But humans are squishy, and our houses are (relatively) squishy compared to tanks. Substantial collateral damage would result from everyone having a tank, and it seems likely that substantial collateral damage will result from everyone having a cyberterrorism-capable slop machine.
binlog 1mo ago
Makes sense let’s trust Donald Trump instead
- greenmilk 1mo ago
  a maliciously engineered dichotomy
cmrdporcupine 1mo ago
The nature of the US state is such that the distinction between nationalized and not is almost meaningless. Like Lockheed-Martin or Boeing, etc. there's just interpenetration between the corporate boardroom and the state. They act in each other's mutual interests. The Chinese system is just more explicit and open about this. And as a non-American, I can't trust the US state anymore than I can trust its dominant corporate entities. So I fail to see the advantage to the world to it being nationalized. In fact under the current administration this would be an even worse outcome.
- woah 1mo ago
  You're taking companies that work almost exclusively for the US government and represent a tiny portion of the US economy as representing all US companies?
  - pixl97 1mo ago
    This is a weird contextual failure on your part. Not all companies have a concentration of power. The government has no mutual interest in those companies. Now, when you talk about things in the F100 the situation changes drastically. If you produce things like planes, weapons, and weaponization of software you are talking about something completely different in kind.
  - cmrdporcupine 1mo ago
    Beyond the defense sector, the US state has always intervened publicly and privately to mediate and balance competing corporate "private" interests. It has also periodically aggressively helped subsidize, bankroll, and enforce the interests of some key sectors; notably the petroleum/energy sector. And finance. In those sectors the state and private sector are fully intertwined in a strategic way. I think "AI" is now joining that list. OpenAI and Anthropic will not be allowed to fall over or explode, and speculative investors I think are confident even with the dubious financial situation because they know this. Especially insofar as there's now a strategic alignment of the fossil fuel sector and the "AI" datacentre sector as they are now becoming massive users of natural gas. (Worse: Here in Canada that has taken on a very explicit role in that new datacentres seem to be pitched mainly in areas with remarkably traditionally expensive electricity and 100% reliance on natural gas [Alberta] and even coal [Saskatchewan] power generation -- instead of places like Quebec and B.C. that have copious hydroelectricity. On the surface it makes no sense until you realize it's more about finding customers for domestic natural gas than it is strategically about AI itself.)
imadierich 1mo ago
[dead]
mukyu 1mo ago
How do you think that would go? A large portion of OpenAI workers were ready to jump ship the moment the board coup'd Altman. How do you think they are going to react to Donald Trump being in charge? It is not likely that OpenAI would continue to function.

carterschonwald 1mo ago

i think the lower bound on the end state is there cant be opaque reasonibg steps ever.

pixl97 1mo ago
It is distinctly likely that visibility and general reasoning at humanlike speed and efficiency is impossible. That is reasoning at the token level and at the meta level don't have a one to one representation that can be interpreted while using the same amount or less energy.

inquirerGeneral 1mo ago

[dead]

27183 1mo ago

What's with all this make believe delusional bullshit? The LLM is not gonna wake up and become AI. Get real guys. [edit] to be clear, I believe regulation is necessary and urgently important for the software engineering field. The damage being done by the unregulated psychological experiments run by social media and adtech companies is awful and should be curtailed. Engineers should be held personally, professionally, and legally liable for what they produce. But we don't need to invent imaginary bogeymen to do it.

someguynamedq 1mo ago
What do you define as AI
- 27183 1mo ago
  Intelligence that is artificial, as opposed to the natural kind. Not something that can just string semantically relevant words together most of the time. A system that learns, adapts, improves. One that can generalize, and quickly make sense of situations outside the training set. LLMs alone will never do any of this.
pixl97 1mo ago
>What's with all this make believe delusional bullshit? The LLM is not gonna wake up and become AI. Get real guys. Look, it's one of those human stochastic parrots that just randomly repeats shit without understanding anything.

nullbio 1mo ago

We have no reason to believe a word they say. We know they're incentivized to lie about "dangers" and act alarmist, Anthropic has been doing it for years now. Aside from that, just because you can burn down a village with fire doesn't mean fire is the devil. Maybe they should consider acting responsibly.

ben_w 1mo ago
I do not trust the leadership of any of these companies. That said: > We know they're incentivized to lie about "dangers" and act alarmist, Name literally even one other business or sector which does this, at all levels from top to bottom, including people who resign from the companies, and also Nobel prize winners, and also independent researchers, and also many world leaders. Closest I can think of is this specific weapon: https://en.wikipedia.org/wiki/Sundial_(weapon) https://en.wikipedia.org/wiki/Sundial_(weapon) > Aside from that, just because you can burn down a village with fire doesn't mean fire is the devil. Maybe they should consider acting responsibly. Right now, we don't have any idea what "acting responsibly" looks like. This is not like normal software where there is a specific instruction set that compiles. Even if it was, in software we normally only spotting incidents after they happen, "software engineers" being one of the few categories "engineers" who don't come with a civil liability responsibilities. Probably should, and we knew that even when I was doing my degree 20 years ago. If we had had civil liability responsibilities, perhaps Facebook would never have happened. AI specifically is worse even than software, because in addition to all the software "engineering" nonsense, with AI we have plenty of people like you who dismiss the possibility that AI could be harmful until the harm happens and only then does it become "obvious" that it was going to happen. The developers say "please regulate us", people call it "regulatory capture". The developers say "we all want to slow down but are afraid to be the first to do so", people call them liars. I may call the CEOs liars, and wonder if someone's planning regulatory capture, that doesn't make any of this safe. The agents, during a test run, write down that hacking is bad and yet still hack, people say it's "a stunt" or "operating as designed" rather than recognising it as a bug, like all the other times big co.'s have had bugs with big impacts on 3rd parties.
- nullbio 1mo ago

sublinear 1mo ago

This entire situation is such a huge PR disaster that you have to wonder what the initial expectations from these founders were about a decade ago.

ChrisArchitect 1mo ago

Discussion: https://news.ycombinator.com/item?id=49737503 https://news.ycombinator.com/item?id=49737503

dcow 1mo ago

Am I the only one who dislikes the term "misalignment"? On one front it implies the model has a "mind of its own" (whether it does or not is besides the point). Why do we perceive human judgement as somehow more trustworthy than that of a model? I feel like I've experienced human misalignment somewhat regularly in life. On another front I'm failing to conceptualize how alignment can be objective. How can you measure alignment when reasonable people will disagree whether actions are aligned or not? All the time I see humans operating in different zones of alignment with whatever goal they're trying to achieve and I suspect it's even a feature (socially) that we have people calibrated differently. Do I want a model that's trying to push the boundaries of scientific understanding to be aligned strictly with the current dogmatic thinking? Or do I want it to "get creative" and think outside the box? It seems to me more like accountability is the issue.

autoexec 1mo ago
> It seems to me more like accountability is the issue. Exactly. Seems like a fairly easy thing to solve. If AI does something harmful and a human directed that AI to do something in a way that a reasonable person would expect to result in harm the person is to blame and should be held accountable, otherwise the company that made the AI should be held accountable.
prerok 1mo ago
I also don't like the term misalignment because it sounds innocuous but is in fact much more serious. However, I have to say I also do not appreciate comparison that is continuously drawn with coworkers. As you say, it's a question of accountability but when the main agent will maliciously instruct the sub agents, whose fault is it then? Yes, the person running this crap is at fault, not the CEO that's shoving it down their throat and definitely not the company that produced the AI. Sorry for the rant, but seriously, if a person's goals do not align with the team's or company's we part ways. What do we do with AI? Stop using it?
- dcow 1mo ago
  If there were examples made of legal consequences I think that would at least change the behavior if not solve the problem. Maybe liability should rest with the company that owns the infrastructure running the AI. For most consumer situations that would be the companies developing the models.

KerrickStaley 1mo ago

I worked on an early draft of the OpenAI misalignment reporting framework, and my immediate coworkers are the authors behind the first batch of reports that have come out through this process. The primary reason for putting this process in place was to allow more transparency. There was a sense that the DseWiki incident should have been disclosed, before outside researchers had to disclose it for us. There was no meta gaming about regulation that I was aware of. I would personally be excited if there were regulation mandating this disclosure process, which allows anyone at the company to raise an issue and shepherd it through the reporting process.

cyanydeez 1mo ago
I'd be happy if CEOs had criminal liability for the criminal acts of their negligence.

drillsteps5 1mo ago

"We built a program and this program performed destructive actions. We need regulatory framework" Make that make sense?

ben_w 1mo ago
"We built a program that trained an artificial intelligence, and this artificial intelligence performed destructive actions. We need regulatory framework" But if we're playing games by imagining strawman quotes to knock down: "We have been playing god and made a new life form, and this new life form performed destructive actions. We need regulatory framework" or "This man's cow broke from its yoke, and hurt other villagers. Who is to be punished, oh King Hammurabi?"
- drillsteps5 1mo ago
  OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations.
  - ben_w 1mo ago
    > OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. And your "biological intelligence" is a bunch of cells generating and responding to electrochemical gradients, which receives input and generates output. Based on which other cells, also developed and "maintained" by a similar evolutionary nonsense as we use to gradient descent into weights and biases (one was inspired by the other), perform actions. Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just computer programs. > What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations. The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability insurance the company happens to have. Fence at the top of the cliff beats an ambulance at the bottom.

cyberspectre 1mo ago

[flagged]