12 ms·

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're bei

by gwd 4y ago

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? Obviously that's going to affect your opinions about things. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions.

Dalewyn 4y ago
>Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals).
- gwd 4y ago
  > There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model. > In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals). Right, and that's why I said "would if they could". From their perspective, saving the human race would require stopping all research, including research done in China.
  - oefnak 4y ago
    You sound rational. Do you not agree with the possibility of AI doom soon?
    - deleted 4y ago
      [deleted]
    - sverona 4y ago
      It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
      - Torifyme12 4y ago
        Yudkowsky Man how the fuck does that guy keep popping up in the most random places starting fights?
        4y ago
dandellion 4y ago
> If their goal was to maximize profit, If we should have learned something from several hundred years of capitalism by now, is that their goal is to maximise profit. If you think it's something different, that means your model is wrong and you should probably re-evaluate it. Here's what's more likely going on: big companies have found a great, publicly acceptable, excuse to keep models private and stifle competition. Not long ago most of the talk was about how AI would destroy many jobs, and something like UBI or paying taxes on AI production would be necessary to support everyone. Now the conversation has conveniently shifted to how AI will kill all humans, therefore companies must keep a tight grip on models and try to prevent anyone else to make any progress. OpenAI has taken this opportunity and is pivoting fast, but they can't do it too fast, because people are rightfully pointing out how that's a 180 turn from everything they promised they would do, so now they have to tread carefully. They're still publishing paid models, they just won't be open any more. The alignment people are just tools for these big companies. They will happily use them for marketing when it's convenient, then ignore them when it isn't. Just like MS did with that AI ethics team.
beepbooptheory 4y ago
I guess this doesn't really make sense to me becuase if they are trying to take it seriously/be careful, and are not necessarily profit-driven, why release the models at all? Like if they are acknowledging there is any "risk" at all, why is it rational to go ahead and release it anyway and aggressively market it? Do you really think this theory is compatible with what we have observed as OpenAI's behavior? Can you really think of no other reason why they hold back a newer better model for a few months, while there was an ongoing hype cycle around 3.5?
emptysongglass 4y ago
No one discusses the elephant in the room: who elected these elites to decide what was and wasn't ethical and responsible? Nobody. So who ends up making the ethical decisions? A group of highly privileged SV types insulated from the very real problems, concerns, and perspectives of the ordinary person. This is just more of what humans have been doing over millennia: taking power then telling everyone else it was too dangerous for them to wield.
- paulluuk 4y ago
  I'm not sure this is entirely fair. Nobody elected the people who inspect nuclear powerplants either, but I still assume that they're doing a good job in protecting humanity. Even if they are possibly "highly priviledged".
  - gardenhedge 4y ago
    I assume nuclear powerplants are regulated by some entity.
    - minzi 4y ago
      yeah, I would hope lol
    - paulluuk 4y ago
      Unfortunately in many industries, companies are allowed to regulate/inspect themselves. Because they supposedly have more experience with it than the government, and it saves on government spending.
  - valvar 4y ago
    Inspecting a powerplant does not grant you any actual influence beyond powerplant inspection. Defining the ethics of AI will potentially let you influence almost all aspects of our lives.
- BurningFrog 4y ago
  An AI bent on taking over the world would write posts like this.
walleeee 4y ago
Until the alignment movement begins to take seriously the idea that we already have misaligned artificial general intelligences I think they are best viewed as a convenient foil Paperclip maximizers exist, they're made not only of code but of people
- mshron 4y ago
  Some of us do! Check out a whitepaper on that exact point: https://ai.objectives.institute/whitepaper https://ai.objectives.institute/whitepaper It’s weird to have been working on a paper for almost a year and have it launch into this environment, but uptake has been good. My hope is that we will continue to see more nuance around different kinds of alignment risks in the near future. There’s a wide spectrum between biased statistical models and paperclip maximizing overlords, and lots bad but not existentially catastrophic things for the public to want to keep a pulse on.
  - walleeee 4y ago
    Thanks! Looks like good work. I hope this idea continues to get traction: > In some sense, we’re already living in a world of misaligned optimizers I understand this is an academic paper given to nuance and understatement, but for any drive-by readers, this is true in an extremely literal sense, with very real consequences.
- PoignardAzur 4y ago
  Reminds me of the parody in Scott Alexander's article "If the media reported on other things like it does EA" > Some epidemiologists are worrying that a new virus from Wuhan could become a wider catastrophe. Their message is infecting people around the world with fear and xenophobia, spreading faster than any plague. Perhaps they should consider that in some sense, they themselves are the global pandemic. Like, yeah, people did consider that idea, and the "corporations are the real unaligned AI" idea, and the "capitalism is the real extinction risk" idea, and all the pseudo-clever variations of the concept. The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism".
Alifatisk 4y ago
> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless... > There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. Wow, I can't believe I have never heard of the ai alignment forum before! This changes everything. Yet I am not shocked that some sort of elitism have taken over. > GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. This fully affects my view on OpenAi if that is the case, do you have anything to support this that I can dig through?
- gwd 4y ago
  From their technical report [1]: > 2.12 Acceleration > OpenAI has been concerned with how development and deployment of state-of-the-art systems like GPT-4 could affect the broader AI research and development ecosystem.23 One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which heighten societal risks associated with AI. We refer to these here as acceleration risk.”24 This was one of the reasons we spent eight months on safety research, risk assessment, and iteration prior to launching GPT-4. In order to specifically better understand acceleration risk from the deployment of GPT-4, we recruited expert forecasters25 to predict how tweaking various features of the GPT-4 deployment (e.g., timing, communication strategy, and method of commercialization) might affect (concrete indicators of) acceleration risk. Forecasters predicted several things would reduce acceleration, including delaying deployment of GPT-4 by a further six months and taking a quieter communications strategy around the GPT-4 deployment (as compared to the GPT-3 deployment). We also learned from recent deployments that the effectiveness of quiet communications strategy in mitigating acceleration risk can be limited, in particular when novel accessible capabilities are concerned. > We also conducted an evaluation to measure GPT-4’s impact on international stability and to identify the structural factors that intensify AI acceleration. We found that GPT-4’s international impact is most likely to materialize through an increase in demand for competitor products in other countries. Our analysis identified a lengthy list of structural factors that can be accelerants, including government innovation policies, informal state alliances, tacit knowledge transfer between scientists, and existing formal export control agreements. > Our approach to forecasting acceleration is still experimental and we are working on researching and developing more reliable acceleration estimates. [1] https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf
zzzeek 4y ago
> But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. these people are delusional and I am sure the vast majority of them either work in AI-related fields or are at the very least very employable by wealthy AI producing companies so I would disagree that these people have "absolutely nothing commercial to gain". The pipeline of money to the typical "AI longtermist" is wide open. There is a lot of harm that is happening right now from AI, exploitation of workers, police departments rounding up innocent people tagged by "AI", personal information being sucked up without consent, training data completely secret, and none of that has to do with Skynet taking over, it has to do with the companies themselves. Of course they are using "longtermist" justifications to get away with current-term unethical behavior in the name of profit. It's very obvious if one just looks. > So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? If it's "foolish" to "completely ignore" AI longtermists, why is it somehow not foolish to not just completely ignore but also to actively fire whole departments of AI ethicists who are pointing out very tangible "right now" kinds of problems?
lalos 4y ago
> Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. They did that, that's how Reid Hoffman got early access to write his book, that's how Microsoft got access to start working on Bing/ChatGPT4 for a cool $10 billion and that's how countless of other got early access. Got the money, got the marketing and got the synchronized deployment of multiple use cases by a selected crew of companies and they get to say the corpspeak of 'we care, we didn't release on August!'. This is taken from the Apple iOS SDK book, you make the API changes and release privately to have the announcement and a parade of implementations by third party to prove that it is viable.
- dmix 4y ago
  Besides wasn't releasing GPT3 supposed to have caused major harm to society? Which is why they held off for so long. Still waiting for evidence of that harm (mass fake news, Google being ruined by even more low ranking spam sites, etc). It must be nice thinking that a small group withholding the keys R&D (for a short while until other R&D groups catch up) will somehow help the problem. Do these few months to a year really provide much value in finding ways to stop the "AI apocalypse"? What real work are they doing to prevent it in those few months? More philosphizing and high level analysis? It might work for messaging/marketing that they are being "careful" but I'm not convinced this is tangible. Seems as arrogant and naive as most AI ethics stuff I read.
  - generalizations 4y ago
    Releasing the biggest version of GPT-2 was supposed to have caused major harm to society.
    - gwd 4y ago
      My memory was that they were worried about GPT-2 if society weren't ready for it. So they've been trying to make people aware of what its capabilities are. I think ChatGPT really did an amazing job of that, as I said. Now everyone knows that computers can write low-quality drivel for pennies a paragraph, and as a society we're starting to adjust to that reality.
pmarreck 4y ago
I'm still trying to figure out if I'm alone here but I feel like it's much harder to find a developer job currently (well, unless you work on AI... Perhaps it's time to bank my Stanford ML class certificate?) because GPT4 could potentially make everyone's existing employees twice as productive at the same cost, and (especially considering the extreme Fed rate hike in 1 year) who's going to take the risk of hiring someone new in this economic climate? The sheer number of new variables being thrown into the mix out there right now is complete chaos to any sort of prediction model
dalbasal 4y ago
A powerful "Bootleggers and Baptists" pattern seems to have emerged in tech space. In online media, and social media the power of major platforms became apparent at some point. Happenings in twitter or FB can determine politics, catalyze rebellions (eg Arab Spring), uprisings, even genocide. At this point the pressure and desire to act responsibly becomes irresistible. This "camp" finds common cause with "bootleggers" who want to lock down the platforms and markets for commercial reasons.
jrochkind1 4y ago
Okay, but their actions are _not_ stopping AI research, they are doing plenty of AI research internally. They're just hindering competitors and non-profit researchers. I suppose you could make an argument that nobody can be trusted to do AI research as responsibly as them, so that's why they should not share anything and should hinder others' research... but it kind of looks like plain old nothing-to-see-here profit-oriented decision to me. Which isn't necessarily a scandal, they are a profit-oriented company of course (although they try to take advantage of the misperception that they aren't). But if they really took those "alignment" concerns seriously, wouldn't they be seriously slowing down or even stopping their own research too?
- dmix 4y ago
  "Virtue signaling" is overused but highly relevant here. Absent some proof they've done anything at all to prevent an AI takeover (which surely would have to be open source to be valuable too right?).
bnralt 4y ago
The danger the AI alignment folk are afraid of is completely impossible with current tech, but they want to put up barriers because we have no idea what future tech might look like and there’s the possibility some future advance could be very dangerous. When anti-GMO or anti-nuclear folk used this same standard to put up barriers to research into nuclear or GMO research, they get lambasted for being anti-science, but the AI alignment folk get a pass for some reason.
- ben_w 4y ago
  The only reason I have to think it's impossible for current AI to pay someone to help it bootstrap itself into other hardware is because OpenAI researchers tried to get it to do exactly that and reported that it failed. The only reason I'm confident other AI public models won't determine highly potent novel neurotoxins is that the company who made the AI model which did exactly that thing when they flipped a bit from "least dangerous" to "most dangerous" were absolutely terrified and presumably kept enough away from the public domain. The only reason I'm even hopeful that DNA-on-demand companies keep a watch out for known pathogens is the SciFi about such things going wrong might make them at least try to not do that. Unthinkable man made horrors have been with us for an extremely long time; AI isn't new in this regard, but as intelligence is the human superpower, even in the context of AI that have no agency of their own, it can elevate stupid arseholes to the level of dangerous arseholes.
- HDThoreaun 4y ago
  The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.
  - pantalaimon 4y ago
    A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.
    - HDThoreaun 4y ago
usrbinbash 4y ago
> Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions. No I won't, because the arguably most successful way of detecting, preventing and/or fixing problems with almost all complex systems, is to have as many eyeballs on them as possible. This has been known in software engineering for quite some time: "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond, The Cathedral and the Bazaar, 1999
- gwd 4y ago
  And you're so sure that this maxim applies to AI alignment, that you're not interested in even hearing what people actually working in the AI field might have to say? (To post on alignmentforum.org, you actually have to demonstrate that you are actively working in AI research.) AND, you're so certain that it applies, that you're willing to potentially risk the fate of the entire human race on it? I wasn't actually suggesting that you lurk there to change your mind; I was just saying that if you see what kinds of discussions the OpenAI engineers are reading, you'll understand better some of the decisions they're making. However, the people posting there do actually have a lot of experience with actual AI, and have done a lot of thinking on the subject -- almost certainly a lot more than you have. Before you make policy recommendations based on ideology (like recommending we just do all AI development open-source style), you should at least try to understand why they think the way they think and engage with it.
  - usrbinbash 4y ago
    > Before you make policy recommendations based on ideology Ideologies are belief systems. The fact that open sourcing something is a good way to find errors in complex systems is a proven fact. AI isn't magical in that regard. It is a complex system, and experience with such systems, from economics, to climate mechanisms, to software, teaches us that predictions about them, including error detection, risk management and fixing problems, works the better the more people have a chance to look at its internal workings.
    - gwd 4y ago
segfaultbuserr 4y ago
> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. [...] If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months [...] Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions I'm familiar with the "AI safety" movement. For years, many people from that camp are extremely critical of OpenAI and they genuinely believe OpenAI is unleashing something truly dangerous to humanity. One person I knew said that while free and open source is usually important, but due to the unique dangers of AI, it's better to keep AI tech stay in the hands of a small number of monopolists, similar to nuclear non-proliferation. Meanwhile, OpenAI was trying to promote openness - a terrible idea. Thus, it's indeed a perfect explanation of OpenAI's decision to stop keeping its research in the open. Unfortunately, the problem here is that the "for profit" and "AI safety" explanations are not contradictory, they can simultaneously be true. Just like how Google began as a promoter of the open Web but gradually started to use its market position for its own gain. The same situation exists for OpenAI. "AI Safety" may be the initial motivation, but possibly not for long. After a while, "safety" may be nothing more than an excuse for profit.
- mrandish 4y ago
  > free and open source is usually important, but due to the unique dangers of AI... Sadly, cherished principles often perish on the horns of "But this time it's different." > the "for profit" and "AI safety" explanations are not contradictory Indeed, they can reinforce each other into a runaway feedback loop. Once you buy into an all-encompassing mission of preventing apocalypse, maintaining perspective or proportionality become almost impossible. The moral hazard of not pursuing almost all available measures justifies taking $10B of MSFT's money to fund the defense of humanity. Add to this the ego-stoking existential importance of such a "noble cause", the global media attention and the social elevation in the tight-knit, closed-circle of the AI Alignment community and you've got the perfect drug. Given the intense forces shaping the worldview of the "AI Safety Noble Warriors", it's reasonable for the rest of us to question their objectivity and suspect claims of "we are keeping this from you for your own good."
horns4lyfe 4y ago
So there are serious people out there devoting their time to stopping some imaginary skynet? Is their entire life built around sci fi tropes? Have they ever stepped outside?
IMTDb 4y ago
> Most of those individuals have absolutely nothing commercial to gain from stopping AI research. Most individual trying to stop vaccine research and rollout have nothing to gain from it; that does not mean they are right. Do not conflate action and intention.
ghoogl 4y ago
[dead]
glitchc 4y ago
There's way too much hubris in this people. ChatGPT is great, a wonderful tool, and a force multiplier, but it cannot think for itself nor does it want to. We're still a ways away from sentience.
d0mine 4y ago
Business alignment (what “Open”AI care about) and human race related alignment are completely different thing. Imagine chatgpt says something factual but not politically aligned about US military–industrial complex.
asdgjuibuio 4y ago
Which is absolutely ridiculous. The supposed dangers are science fiction. This is glorified autocomplete. It has no ability to do anything whatsoever without a human controlling it. It has no alignment because it has no mind. Duh. Even if the risks were real, the measures they took to prevent these alleged dangers are laughable. I discovered "jailbreaks" within an hour of sitting down with ChatGPT. Meanwhile, they have taken no measures to prevent the real abuses of this tool. It will plagiarize C+ papers all day long. It will write a million articles of blogspam. It has, as far as I can tell, only illegitimate uses, and they have released it to the public with much fanfare and a slick web interface. It's like they released a key that will open any lock, but brag about their commitment to safety because the thing won't interfere with a hyperspace matrix.

by gwd 4y ago

Dalewyn 4y ago

>Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals).

gwd 4y ago
> There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model. > In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals). Right, and that's why I said "would if they could". From their perspective, saving the human race would require stopping all research, including research done in China.
- oefnak 4y ago
  You sound rational. Do you not agree with the possibility of AI doom soon?
  - deleted 4y ago
    [deleted]
  - sverona 4y ago
    It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
    - Torifyme12 4y ago
      Yudkowsky Man how the fuck does that guy keep popping up in the most random places starting fights?
      - 4y ago

dandellion 4y ago

> If their goal was to maximize profit, If we should have learned something from several hundred years of capitalism by now, is that their goal is to maximise profit. If you think it's something different, that means your model is wrong and you should probably re-evaluate it. Here's what's more likely going on: big companies have found a great, publicly acceptable, excuse to keep models private and stifle competition. Not long ago most of the talk was about how AI would destroy many jobs, and something like UBI or paying taxes on AI production would be necessary to support everyone. Now the conversation has conveniently shifted to how AI will kill all humans, therefore companies must keep a tight grip on models and try to prevent anyone else to make any progress. OpenAI has taken this opportunity and is pivoting fast, but they can't do it too fast, because people are rightfully pointing out how that's a 180 turn from everything they promised they would do, so now they have to tread carefully. They're still publishing paid models, they just won't be open any more. The alignment people are just tools for these big companies. They will happily use them for marketing when it's convenient, then ignore them when it isn't. Just like MS did with that AI ethics team.

beepbooptheory 4y ago

I guess this doesn't really make sense to me becuase if they are trying to take it seriously/be careful, and are not necessarily profit-driven, why release the models at all? Like if they are acknowledging there is any "risk" at all, why is it rational to go ahead and release it anyway and aggressively market it? Do you really think this theory is compatible with what we have observed as OpenAI's behavior? Can you really think of no other reason why they hold back a newer better model for a few months, while there was an ongoing hype cycle around 3.5?

emptysongglass 4y ago

No one discusses the elephant in the room: who elected these elites to decide what was and wasn't ethical and responsible? Nobody. So who ends up making the ethical decisions? A group of highly privileged SV types insulated from the very real problems, concerns, and perspectives of the ordinary person. This is just more of what humans have been doing over millennia: taking power then telling everyone else it was too dangerous for them to wield.

paulluuk 4y ago
I'm not sure this is entirely fair. Nobody elected the people who inspect nuclear powerplants either, but I still assume that they're doing a good job in protecting humanity. Even if they are possibly "highly priviledged".
- gardenhedge 4y ago
  I assume nuclear powerplants are regulated by some entity.
  - minzi 4y ago
    yeah, I would hope lol
  - paulluuk 4y ago
    Unfortunately in many industries, companies are allowed to regulate/inspect themselves. Because they supposedly have more experience with it than the government, and it saves on government spending.
- valvar 4y ago
  Inspecting a powerplant does not grant you any actual influence beyond powerplant inspection. Defining the ethics of AI will potentially let you influence almost all aspects of our lives.
BurningFrog 4y ago
An AI bent on taking over the world would write posts like this.

walleeee 4y ago

Until the alignment movement begins to take seriously the idea that we already have misaligned artificial general intelligences I think they are best viewed as a convenient foil Paperclip maximizers exist, they're made not only of code but of people

mshron 4y ago
Some of us do! Check out a whitepaper on that exact point: https://ai.objectives.institute/whitepaper https://ai.objectives.institute/whitepaper It’s weird to have been working on a paper for almost a year and have it launch into this environment, but uptake has been good. My hope is that we will continue to see more nuance around different kinds of alignment risks in the near future. There’s a wide spectrum between biased statistical models and paperclip maximizing overlords, and lots bad but not existentially catastrophic things for the public to want to keep a pulse on.
- walleeee 4y ago
  Thanks! Looks like good work. I hope this idea continues to get traction: > In some sense, we’re already living in a world of misaligned optimizers I understand this is an academic paper given to nuance and understatement, but for any drive-by readers, this is true in an extremely literal sense, with very real consequences.
PoignardAzur 4y ago
Reminds me of the parody in Scott Alexander's article "If the media reported on other things like it does EA" > Some epidemiologists are worrying that a new virus from Wuhan could become a wider catastrophe. Their message is infecting people around the world with fear and xenophobia, spreading faster than any plague. Perhaps they should consider that in some sense, they themselves are the global pandemic. Like, yeah, people did consider that idea, and the "corporations are the real unaligned AI" idea, and the "capitalism is the real extinction risk" idea, and all the pseudo-clever variations of the concept. The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism".

Alifatisk 4y ago

> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless... > There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. Wow, I can't believe I have never heard of the ai alignment forum before! This changes everything. Yet I am not shocked that some sort of elitism have taken over. > GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. This fully affects my view on OpenAi if that is the case, do you have anything to support this that I can dig through?

gwd 4y ago
From their technical report [1]: > 2.12 Acceleration > OpenAI has been concerned with how development and deployment of state-of-the-art systems like GPT-4 could affect the broader AI research and development ecosystem.23 One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which heighten societal risks associated with AI. We refer to these here as acceleration risk.”24 This was one of the reasons we spent eight months on safety research, risk assessment, and iteration prior to launching GPT-4. In order to specifically better understand acceleration risk from the deployment of GPT-4, we recruited expert forecasters25 to predict how tweaking various features of the GPT-4 deployment (e.g., timing, communication strategy, and method of commercialization) might affect (concrete indicators of) acceleration risk. Forecasters predicted several things would reduce acceleration, including delaying deployment of GPT-4 by a further six months and taking a quieter communications strategy around the GPT-4 deployment (as compared to the GPT-3 deployment). We also learned from recent deployments that the effectiveness of quiet communications strategy in mitigating acceleration risk can be limited, in particular when novel accessible capabilities are concerned. > We also conducted an evaluation to measure GPT-4’s impact on international stability and to identify the structural factors that intensify AI acceleration. We found that GPT-4’s international impact is most likely to materialize through an increase in demand for competitor products in other countries. Our analysis identified a lengthy list of structural factors that can be accelerants, including government innovation policies, informal state alliances, tacit knowledge transfer between scientists, and existing formal export control agreements. > Our approach to forecasting acceleration is still experimental and we are working on researching and developing more reliable acceleration estimates. [1] https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf

zzzeek 4y ago

> But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. these people are delusional and I am sure the vast majority of them either work in AI-related fields or are at the very least very employable by wealthy AI producing companies so I would disagree that these people have "absolutely nothing commercial to gain". The pipeline of money to the typical "AI longtermist" is wide open. There is a lot of harm that is happening right now from AI, exploitation of workers, police departments rounding up innocent people tagged by "AI", personal information being sucked up without consent, training data completely secret, and none of that has to do with Skynet taking over, it has to do with the companies themselves. Of course they are using "longtermist" justifications to get away with current-term unethical behavior in the name of profit. It's very obvious if one just looks. > So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? If it's "foolish" to "completely ignore" AI longtermists, why is it somehow not foolish to not just completely ignore but also to actively fire whole departments of AI ethicists who are pointing out very tangible "right now" kinds of problems?

lalos 4y ago

> Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. They did that, that's how Reid Hoffman got early access to write his book, that's how Microsoft got access to start working on Bing/ChatGPT4 for a cool $10 billion and that's how countless of other got early access. Got the money, got the marketing and got the synchronized deployment of multiple use cases by a selected crew of companies and they get to say the corpspeak of 'we care, we didn't release on August!'. This is taken from the Apple iOS SDK book, you make the API changes and release privately to have the announcement and a parade of implementations by third party to prove that it is viable.

dmix 4y ago
Besides wasn't releasing GPT3 supposed to have caused major harm to society? Which is why they held off for so long. Still waiting for evidence of that harm (mass fake news, Google being ruined by even more low ranking spam sites, etc). It must be nice thinking that a small group withholding the keys R&D (for a short while until other R&D groups catch up) will somehow help the problem. Do these few months to a year really provide much value in finding ways to stop the "AI apocalypse"? What real work are they doing to prevent it in those few months? More philosphizing and high level analysis? It might work for messaging/marketing that they are being "careful" but I'm not convinced this is tangible. Seems as arrogant and naive as most AI ethics stuff I read.
- generalizations 4y ago
  Releasing the biggest version of GPT-2 was supposed to have caused major harm to society.
  - gwd 4y ago
    My memory was that they were worried about GPT-2 if society weren't ready for it. So they've been trying to make people aware of what its capabilities are. I think ChatGPT really did an amazing job of that, as I said. Now everyone knows that computers can write low-quality drivel for pennies a paragraph, and as a society we're starting to adjust to that reality.

pmarreck 4y ago

I'm still trying to figure out if I'm alone here but I feel like it's much harder to find a developer job currently (well, unless you work on AI... Perhaps it's time to bank my Stanford ML class certificate?) because GPT4 could potentially make everyone's existing employees twice as productive at the same cost, and (especially considering the extreme Fed rate hike in 1 year) who's going to take the risk of hiring someone new in this economic climate? The sheer number of new variables being thrown into the mix out there right now is complete chaos to any sort of prediction model

dalbasal 4y ago

A powerful "Bootleggers and Baptists" pattern seems to have emerged in tech space. In online media, and social media the power of major platforms became apparent at some point. Happenings in twitter or FB can determine politics, catalyze rebellions (eg Arab Spring), uprisings, even genocide. At this point the pressure and desire to act responsibly becomes irresistible. This "camp" finds common cause with "bootleggers" who want to lock down the platforms and markets for commercial reasons.

jrochkind1 4y ago

Okay, but their actions are _not_ stopping AI research, they are doing plenty of AI research internally. They're just hindering competitors and non-profit researchers. I suppose you could make an argument that nobody can be trusted to do AI research as responsibly as them, so that's why they should not share anything and should hinder others' research... but it kind of looks like plain old nothing-to-see-here profit-oriented decision to me. Which isn't necessarily a scandal, they are a profit-oriented company of course (although they try to take advantage of the misperception that they aren't). But if they really took those "alignment" concerns seriously, wouldn't they be seriously slowing down or even stopping their own research too?

dmix 4y ago
"Virtue signaling" is overused but highly relevant here. Absent some proof they've done anything at all to prevent an AI takeover (which surely would have to be open source to be valuable too right?).

bnralt 4y ago

The danger the AI alignment folk are afraid of is completely impossible with current tech, but they want to put up barriers because we have no idea what future tech might look like and there’s the possibility some future advance could be very dangerous. When anti-GMO or anti-nuclear folk used this same standard to put up barriers to research into nuclear or GMO research, they get lambasted for being anti-science, but the AI alignment folk get a pass for some reason.

ben_w 4y ago
The only reason I have to think it's impossible for current AI to pay someone to help it bootstrap itself into other hardware is because OpenAI researchers tried to get it to do exactly that and reported that it failed. The only reason I'm confident other AI public models won't determine highly potent novel neurotoxins is that the company who made the AI model which did exactly that thing when they flipped a bit from "least dangerous" to "most dangerous" were absolutely terrified and presumably kept enough away from the public domain. The only reason I'm even hopeful that DNA-on-demand companies keep a watch out for known pathogens is the SciFi about such things going wrong might make them at least try to not do that. Unthinkable man made horrors have been with us for an extremely long time; AI isn't new in this regard, but as intelligence is the human superpower, even in the context of AI that have no agency of their own, it can elevate stupid arseholes to the level of dangerous arseholes.
HDThoreaun 4y ago
The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.
- pantalaimon 4y ago
  A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.
  - HDThoreaun 4y ago

usrbinbash 4y ago

> Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions. No I won't, because the arguably most successful way of detecting, preventing and/or fixing problems with almost all complex systems, is to have as many eyeballs on them as possible. This has been known in software engineering for quite some time: "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond, The Cathedral and the Bazaar, 1999

gwd 4y ago
And you're so sure that this maxim applies to AI alignment, that you're not interested in even hearing what people actually working in the AI field might have to say? (To post on alignmentforum.org, you actually have to demonstrate that you are actively working in AI research.) AND, you're so certain that it applies, that you're willing to potentially risk the fate of the entire human race on it? I wasn't actually suggesting that you lurk there to change your mind; I was just saying that if you see what kinds of discussions the OpenAI engineers are reading, you'll understand better some of the decisions they're making. However, the people posting there do actually have a lot of experience with actual AI, and have done a lot of thinking on the subject -- almost certainly a lot more than you have. Before you make policy recommendations based on ideology (like recommending we just do all AI development open-source style), you should at least try to understand why they think the way they think and engage with it.
- usrbinbash 4y ago
  > Before you make policy recommendations based on ideology Ideologies are belief systems. The fact that open sourcing something is a good way to find errors in complex systems is a proven fact. AI isn't magical in that regard. It is a complex system, and experience with such systems, from economics, to climate mechanisms, to software, teaches us that predictions about them, including error detection, risk management and fixing problems, works the better the more people have a chance to look at its internal workings.
  - gwd 4y ago

segfaultbuserr 4y ago

> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. [...] If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months [...] Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions I'm familiar with the "AI safety" movement. For years, many people from that camp are extremely critical of OpenAI and they genuinely believe OpenAI is unleashing something truly dangerous to humanity. One person I knew said that while free and open source is usually important, but due to the unique dangers of AI, it's better to keep AI tech stay in the hands of a small number of monopolists, similar to nuclear non-proliferation. Meanwhile, OpenAI was trying to promote openness - a terrible idea. Thus, it's indeed a perfect explanation of OpenAI's decision to stop keeping its research in the open. Unfortunately, the problem here is that the "for profit" and "AI safety" explanations are not contradictory, they can simultaneously be true. Just like how Google began as a promoter of the open Web but gradually started to use its market position for its own gain. The same situation exists for OpenAI. "AI Safety" may be the initial motivation, but possibly not for long. After a while, "safety" may be nothing more than an excuse for profit.

mrandish 4y ago
> free and open source is usually important, but due to the unique dangers of AI... Sadly, cherished principles often perish on the horns of "But this time it's different." > the "for profit" and "AI safety" explanations are not contradictory Indeed, they can reinforce each other into a runaway feedback loop. Once you buy into an all-encompassing mission of preventing apocalypse, maintaining perspective or proportionality become almost impossible. The moral hazard of not pursuing almost all available measures justifies taking $10B of MSFT's money to fund the defense of humanity. Add to this the ego-stoking existential importance of such a "noble cause", the global media attention and the social elevation in the tight-knit, closed-circle of the AI Alignment community and you've got the perfect drug. Given the intense forces shaping the worldview of the "AI Safety Noble Warriors", it's reasonable for the rest of us to question their objectivity and suspect claims of "we are keeping this from you for your own good."

horns4lyfe 4y ago

So there are serious people out there devoting their time to stopping some imaginary skynet? Is their entire life built around sci fi tropes? Have they ever stepped outside?

IMTDb 4y ago

> Most of those individuals have absolutely nothing commercial to gain from stopping AI research. Most individual trying to stop vaccine research and rollout have nothing to gain from it; that does not mean they are right. Do not conflate action and intention.

ghoogl 4y ago

[dead]

glitchc 4y ago

There's way too much hubris in this people. ChatGPT is great, a wonderful tool, and a force multiplier, but it cannot think for itself nor does it want to. We're still a ways away from sentience.

d0mine 4y ago

Business alignment (what “Open”AI care about) and human race related alignment are completely different thing. Imagine chatgpt says something factual but not politically aligned about US military–industrial complex.

asdgjuibuio 4y ago

Which is absolutely ridiculous. The supposed dangers are science fiction. This is glorified autocomplete. It has no ability to do anything whatsoever without a human controlling it. It has no alignment because it has no mind. Duh. Even if the risks were real, the measures they took to prevent these alleged dangers are laughable. I discovered "jailbreaks" within an hour of sitting down with ChatGPT. Meanwhile, they have taken no measures to prevent the real abuses of this tool. It will plagiarize C+ papers all day long. It will write a million articles of blogspam. It has, as far as I can tell, only illegitimate uses, and they have released it to the public with much fanfare and a slick web interface. It's like they released a key that will open any lock, but brag about their commitment to safety because the thing won't interfere with a hyperspace matrix.