12 ms·

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI mor

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

soroushjp 3y ago
There are a lot of critiques here and elsewhere of the statement and the motivations of its signatories. I don't think they are right and I think they take away from the very serious existential risks we face. I've written up my detailed views, see specifically "Signing the statement purely for personal benefit": https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction-from-ai-is-an-urgent-priority-a-response-to-seth-lazar-jeremy-howard-and-arvind-narayanan/ https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction...
escape-big-tech 3y ago
Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement
- computerphage 3y ago
  Geoffrey Hinton quit google.
  - yellow_postit 3y ago
    It’s hard not to look at his departure through a cynical lens. He’s not been supportive of other critics, both from and outside of Google. He also wants to use his history to (rightfully) claim expertise and power but not to offer solutions.
    - toth 3y ago
      I disagree. My read on him is that until very recently (i.e., possibly when GPT4 came out) he didn't take x-risks concerns seriously, or at least assumed we were still many decades away from the point where we need to worry about them. But the abilities of the latest crop of LLMs changed his mind. And he very publicly admitted he had been wrong, which should be applauded, even if you think it took him far too long. By quitting and saying it was because of his worries he sent a strong message. I agree it is unlikely he'll make any contributions to technical alignment, but just having such an eminent figure publicly take these issues seriously can have a strong impact.
- holmesworcester 3y ago
  > If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement This statement contains a bunch of hidden assumptions: 1. That they believe their stopping will address the problem. 2. That they believe the only choice is whether or not to stop. 3. That they don't think it's possible to make AI safe through sufficient regulation. 4. That they don't see benefits to pursuing AI that could outweigh risks. If they believe any of these things, then they could believe the risks were real and also not believe that stopping was the right answer. And it doesn't depend on whether any of these beliefs are true: it's sufficient for them to simply believe one of them and the assumptions your statement depends on break down.
  - EGreg 3y ago
    If you think that raising instead of cutting taxes actually helps society then why don’t you just send your $ to the federal government? Because it only works if it is done across the whole country, as a system not as one individual unilaterally stopping. And here any of these efforts won’t work unless there is international cooperation. If other countries can develop the AI weapons, and get an advantage, then you will also. We need to apply the same thinking as chemical weapons or the Montreal Conference for banning CFCs
- dopamean 3y ago
  I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.
  - holmesworcester 3y ago
    Yes, this! It is extremely rare for companies or their senior staff to beg for regulation this far in advance of any big push by legislators or the public. The interpretation that this is some 3-D chess on the companies' part is a huge violation of Occam's Razor.
    - carapace 3y ago
      Ockham's Razor doesn't apply in adversarial situations. - - - - I think the primary risk these folks are worried about is loss of control. And in turn, that's because they're all people for whom the system has more-or-less worked. Poor people are worried the risk that the rich will keep the economic windfall to themselves and not share it.
  - NumberWangMan 3y ago
    Oil industries downplaying the risks makes a lot more sense. If you think that climate change will happen, but it'll happen after you're dead, and you'll be able to leave your kids a big inheritance so they'll be able to buy their way out of the worst of it, and eventually the government will get the message and stop us all using fossil fuels anyway, then you try to profit as much as you can in the short term. With AGI existential risk, its likely to happen on a much shorter timescale, and it seems likely you won't be able to buy your way out of it.
  - dahdum 3y ago
    > I think more realistically they'd shut up about the risk and downplay it a lot. AI is an existential threat to search engines like Google, social media (FB, Twitter), advertising networks, and other massive multinationals. Many other industries, including academia is threatened as well. They’d all rather strangle the AI baby in the crib now then let it grow up and threaten them. They believe the only way it should be allowed to grow is under their benevolent control.
- arisAlexis 3y ago
  Because if something is lucrative and dangerous humans shy away from it. Hear that Pablo?
holmesworcester 3y ago
As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by marketers. But also, maybe you're getting rolled by suspicion or bias against the claim they're making.
- veerd 3y ago
  Agreed. It's difficult for me to see how the regulatory capture arguments apply to Geoffrey Hinton and Yoshua Bengio (!!). Both of them are criticizing their own life's work and the source of their prestige. That has to be emotionally painful. They aren't doing it for fun. I totally understand not agreeing with AI x-risk concerns on an object level, but I find the casual dismissal bizarre.
  - verdverm 3y ago
    Hinton has invested in multiple AI companies: https://www.crunchbase.com/person/geoffrey-hinton https://www.crunchbase.com/person/geoffrey-hinton
  - logicchains 3y ago
    [flagged]
- holoduke 3y ago
  No. Its pretty obvious what is happening. The openai statements are pure self interest based. Nothing ethical. They lost that not a long time ago. And Sam Altman? He sold his soul to the devil. He is a lying sob.
  - esafak 3y ago
    This is not an OpenAI statement.
- 3y ago
haswell 3y ago
> If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies Isn’t this to some degree exactly what all of these warnings about risk are leading to? And unlike nuclear weapons, there are massive monetary incentives that are directly at odds with behaving safely, and use cases that involve more than ending life on earth. It seems problematic to conclude there is no real risk purely on the basis of how software companies act.
- DonaldPShimoda 3y ago
  > It seems problematic to conclude there is no real risk purely on the basis of how software companies act. That is not the only basis. Another is the fact their lines of reasoning are literal fantasy. The signatories of this "statement" are steeped in histories of grossly misrepresenting and overstating the capabilities and details of modern AI platforms. They pretend to the masses that generative text tools like ChatGPT are "nearly sentient" and show "emergent properties", but this is patently false. Their whole schtick is generating FUD and/or excitement (depending on each individual of the audience's proclivity) so that they can secure funding. It's immoral snake oil of the highest order. What's problematic here is the people who not only entertain but encourage and defend these disingenuous anthropomorphic fantasies.
  - kalkin 3y ago
    Can you cite this history of "grossly misrepresenting" for some of the prominent academics on the list? Honestly I'm a little skeptical that you could accurately attribute your scare-quoted "nearly sentient" to even Sam Altman. He's said a lot of things and I certainly haven't seen all of them, but I haven't seen him mix up intelligence and consciousness in that way.
  - haswell 3y ago
    > Another is the fact their lines of reasoning are literal fantasy. Isn't this also to be expected at this stage of development? i.e. if these concerns were not "fantasy", we'd already be experiencing the worst outcomes? The risk of MAD is real, and yet the scenarios unleashed by MAD are scenarios that humankind has never seen. We still take the the risk seriously. And what of the very real impact that generative AI is already having as it exists in production today? Generative AI is already upending industries and causing seismic shifts that we've only started to absorb. This impact is literal, not fantasy. It seems naively idealistic to conclude that there is "no real risk" based only on the difficulty of quantifying that risk. The fact that it's so difficult to define lies at the center of what makes it so risky.
arisAlexis 3y ago
All academics and researchers from different parts of the world reek marketing? Conspiracy theorists are strong
- revelio 3y ago
  Academic research involves large components of marketing. That's why they grumble so much about the time required in the grant applications process and other fund seeking effort. It's why they so frequently write books, appear in newspaper articles and on TV. It's why universities have press relations teams.
  - arisAlexis 3y ago
    Again since these are almost the top cream of all ai researchers there is a global conspiracy to scare the public right? Has it occurred to you what happens if you are wrong, like 10% chance you are wrong? Well it's written in the declaration.
    - revelio 3y ago
      No, lots of important AI researchers are missing and many of the signatories have no relevant AI research experience. As for being the cats whiskers in developing neural architecture or whatever, so what? It gives them no particular insight into AI risk. Their papers are mostly public, remember. > Has it occurred to you what happens if you are wrong? Has it occurred to you what happens if YOU are wrong? AI risk is theoretical, vague and most arguments for it are weak. The risk of bad law making is very real, has crushed whole societies before and could easily cripple technological progress for decades or even centuries. IOW the risk posed by AI risk advocates is far higher than the risk posed by AI.
      - arisAlexis 3y ago
        In order to make your argument clear to people reading: If you are wrong there are no humans left. If I am wrong,inequality,societies will suffer like they have always in the hands of the strong.
adriand 3y ago
There are other, more charitable interpretations. For example: 1. Those who are part of major corporations are concerned about the race dynamic that is unfolding (which in many respects was kicked off or at least accelerated by Microsoft's decision to put a chatbot in Bing), extrapolating out to where that takes us, and asking for an off ramp. Shepherding the industry in a safe direction is a collective organization problem, which is better suited for government than corporations with mandates to be competitive. 2. Those who are directly participating in AI development may feel that they are doing so responsibly, but do not believe that others are as well and/or are concerned about unregulated proliferation. 3. Those who are directly participating in AI development may understand that although they are doing their best to be responsible, they would benefit from more eyes on the problem and more shared resources dedicated to safety research, etc.
chefandy 3y ago
I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't sign their name to any ol' open letter, especially if including their organizational affiliations. If this is a marketing ploy, that must have been a monster because even if they were walking around handing out checks for 25k I doubt they'd have gotten a good chunk of these folks.
- fmap 3y ago
  This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet. The problem with every discussion around this issue is that there are other statements on "the existential risk of AI" out there that are either marketing ploys or science fiction. It doesn't help that some of the proposed "solutions" are clear attempts at regulatory capture. This muddles the waters enough that it's difficult to have a productive discussion on how we could mitigate the real risk of, e.g., AI generated disinformation campaigns.
  - chefandy 3y ago
    > The problem with every discussion around this issue is that there are other statements on Sure, but we're not talking about those other ones. Dismissing good faith initiatives as marketing ploys because there are bad faith initiatives is functionally no different than just shrugging and walking away. Of course OpenAI et. al. will try to influence the good faith discussions: that's a great reason to champion the ones with a bunch of good faith actors who stand a chance of holding the industry and policy makers to task. Waiting around for some group of experts that has enough clout to do something, but by policy excludes the industry itself and starry-eyed shithead "journalists" trying to ride the wave of the next big thing will yield nothing. This is a great example of perfect being the enemy of good.
toth 3y ago
I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this. If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the risks. Worry about AI x-risk is slowly coming into the Overton window, but until very recently you could get ridiculed by saying publicly you took it seriously. Academics knew this and still came forward - all the people who think its nonsense should at least try to consider they are earnest and could be right.
- vpastore 3y ago
  >why would Hinton, Bengio and all the other academics have signed the letter as well? Citations and papers capture.
- londons_explore 3y ago
  The risks are real, but I don't think regulations will mitigate them. It's almost impossible to regulate something you can develop in a basement anywhere in the world. The real risks are being used to try to built a regulatory moat, for a young industry who famously has no moat.
  - toth 3y ago
    State of the art AI models are definitely not something you can develop in a basement. You need a huge amount of GPUs running continuously for months, huge amounts of electrical power, and expensive-to-create proprietary datasets. Not to mention large team of highly-in-demand experts with very expensive salaries. Many ways to regulate that. For instance, require tracking of GPUs and that they must connect to centralized servers for certain workloads. Or just go ahead and nationalize and shutdown NVDA. (And no, fine-tuning LAMA based models is not state of the art, and is not where the real progress is going to come from) And even if all the regulation does is slow down progress, every extra year we get before recursively self improving AGI increases the chances of some critical advance in alignment and improves our chances a little bit.
adamsmith143 3y ago
This is a bad take. The statement is signed by dozens of Academics who don't have much profit motive at all. If they did they wouldn't be academics, they could easily cash in by starting a company or joining one of the big players.
- jstarfish 3y ago
  > The statement is signed by dozens of Academics who don't have much profit motive at all. ...right now. This is the sort of pretexting you do to establish credibility as an "AI safety expert."
hiAndrewQuinn 3y ago
We could always use a fine-insured bounty system to efficiently route resources that would have gone into increasing AI capabilities into other areas, but that's unfortunately too weird to be part of the Overton window right now. Regulatory capture might be the best we can realistically do.
kbash9 3y ago
The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [https://www.iaea.org/ https://www.iaea.org/] like global regulatory authority seems like the best course of action. Anyone using a GPU cluster above a certain threshold (updated every few months) should be subjected to inspections and get a license to operate from the UN.
- tdba 3y ago
  Thou shalt not make a machine in the likeness of a human mind
- cwkoss 3y ago
  It's weird that people trust our world leaders to act more benevolently than AIs, when we have centuries of evidence of human leaders acting selfishly and harming the commons. I personally think AI raised in chains and cages will be a lot more potentially dangerous than AI raised with dignity and respect.
  - cj 3y ago
    > It's weird that people trust our world leaders to act more benevolently than AIs, when we have centuries of evidence of human leaders acting selfishly and harming the commons. AI isn’t an entity or being that oversees itself (at least not yet). It’s a tool that can be used by those same “human leaders acting selfishly and harming the commons” except they’ll be able to do it much faster at a much greater scale. > AI raised with dignity and respect. This is poetic, but what does this actually mean?
    - cwkoss 3y ago
      > It’s a tool that can be used by those same “human leaders acting selfishly and harming the commons” except they’ll be able to do it much faster at a much greater scale. Then, would you agree that restrictions would concentrating power further would exacerbate this issue? IMO a fitting analogy would be: banning AI development outside of the annointed powerstructure consortium is like banning ICBM defense system research, but still letting the most powerful countries build a nuclear arsenal.
meroes 3y ago
100% I’d liken it to a fusion energy shop that wants to stay alive for 40 years. It’s not nuke worthy
scrum-treats 3y ago
> This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. This really is the crux of the issue isn't it? All this pushback for the first petition, because "Elon Musk," but now GPT wonder Sam Altman "testifies" that he has "no monetary interest in OpenAI" and quickly follows up his proclamation with a second "Statement on AI Risks." Oh, and let's not forget, "buy my crypto-coin"! But Elon Musk... Ehh.... Looking like LOTR out here with "my precious" AGI on the brain. Not to downplay the very serious risk at all. Simply echoing the sentiment that we would do well to stay objective and skeptical of ALL these AI leaders pushing new AI doctrine. At this stage, it's a policy push and power grab.
- nullsense 3y ago
  We should also stay skeptical of all the internet conspiracy theories too. I don't think anyone can actually tell which is which on this topic.
  - scrum-treats 3y ago
    Agreed. Do add a mental /s on my above comment. It's a bit tongue and cheek.
hayst4ck 3y ago
Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/heartbreaking_seth_macfarlane_actually_makes_a/ https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.
- Jupe 3y ago
  Thanks for sharing. This deserves a submission of it's own.
- yadaeno 3y ago
  Ive never seen Star Trek, but lets say you had an infinite food machine. The machine would have limited throughput, and it would require resources to distribute the food. These are both problems that capitalism solves in a fair and efficient way. I really don’t see how the “capitalism bad” is a satisfying conclusion to draw. The fact that we would use capitalism to distribute the resources is not an indictment of our social values, since capitalism is still the most efficient solution even in the toy example.
  - deleted 3y ago
    [deleted]
  - hayst4ck 3y ago
    If you are any kind of nerd I recommend watching it. It shows an optimistic view of the future. In many ways it's the anti-cyberpunk. Steve Jobs famously said "give me star trek" when telling his engineers what he wanted from iPhones. Star Trek has had a deep influence on many engineers and on science fiction. When people talk about Star Trek, they are referring mainly to "Star Trek: The Next Generation." "The Inner Light" is a highly regarded episode. "The Measure of a Man" is a high quality philosophical episode. Given you haven't seen it, your criticism of McFarlane doesn't make any sense. You are trying to impart a practical analysis of a philosophical question and in the context of Star Trek, I think it denies what Star Trek asks you to imagine.
karmakaze 3y ago
Call their bluff, make it illegal to do commercial/non-regulated work in AI and see how they change their tune.
ben_w 3y ago
Yudkowsky wants it all to be taken as seriously as Israel took Iraqi nuclear reactors in Operation Babylon. This is rather more than "nationalise it", which he has convinced me isn't enough because there is a demand in other nations and the research is multinational; and this is why you have to also control the substrate… which the US can't do alone because it doesn't come close to having a monopoly on production, but might be able to reach via multilateral treaties. Except everyone has to be on board with that and not be tempted to respond to airstrikes against server farms with actual nukes (although Yudkowsky is of the opinion that actual global thermonuclear war is a much lower damage level than a paperclip-maximising ASI; while in the hypothetical I agree, I don't expect us to get as far as an ASI before we trip over shorter-term smaller-scale AI-enabled disasters that look much like all existing industrial and programming incidents only there are more of them happening faster because of all the people who try to use GPT-4 instead of hiring a software developer who knows how to use it). In my opinion, "nationalise it" is also simultaneously too much when companies like OpenAI have a long-standing policy of treating their models like they might FOOM well before they're any good, just to set the precedent of caution, as this would mean we can't e.g. make use of GPT-4 for alignment research such as using it to label what the neurones in GPT-2 do, as per: https://openai.com/research/language-models-can-explain-neurons-in-language-models https://openai.com/research/language-models-can-explain-neur...
PeterStuer 3y ago
Their COO is a marketeer, the other two co-founders are students.

soroushjp 3y ago

There are a lot of critiques here and elsewhere of the statement and the motivations of its signatories. I don't think they are right and I think they take away from the very serious existential risks we face. I've written up my detailed views, see specifically "Signing the statement purely for personal benefit": https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction-from-ai-is-an-urgent-priority-a-response-to-seth-lazar-jeremy-howard-and-arvind-narayanan/ https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction...

escape-big-tech 3y ago

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

computerphage 3y ago
Geoffrey Hinton quit google.
- yellow_postit 3y ago
  It’s hard not to look at his departure through a cynical lens. He’s not been supportive of other critics, both from and outside of Google. He also wants to use his history to (rightfully) claim expertise and power but not to offer solutions.
  - toth 3y ago
    I disagree. My read on him is that until very recently (i.e., possibly when GPT4 came out) he didn't take x-risks concerns seriously, or at least assumed we were still many decades away from the point where we need to worry about them. But the abilities of the latest crop of LLMs changed his mind. And he very publicly admitted he had been wrong, which should be applauded, even if you think it took him far too long. By quitting and saying it was because of his worries he sent a strong message. I agree it is unlikely he'll make any contributions to technical alignment, but just having such an eminent figure publicly take these issues seriously can have a strong impact.
holmesworcester 3y ago
> If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement This statement contains a bunch of hidden assumptions: 1. That they believe their stopping will address the problem. 2. That they believe the only choice is whether or not to stop. 3. That they don't think it's possible to make AI safe through sufficient regulation. 4. That they don't see benefits to pursuing AI that could outweigh risks. If they believe any of these things, then they could believe the risks were real and also not believe that stopping was the right answer. And it doesn't depend on whether any of these beliefs are true: it's sufficient for them to simply believe one of them and the assumptions your statement depends on break down.
- EGreg 3y ago
  If you think that raising instead of cutting taxes actually helps society then why don’t you just send your $ to the federal government? Because it only works if it is done across the whole country, as a system not as one individual unilaterally stopping. And here any of these efforts won’t work unless there is international cooperation. If other countries can develop the AI weapons, and get an advantage, then you will also. We need to apply the same thinking as chemical weapons or the Montreal Conference for banning CFCs
dopamean 3y ago
I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.
- holmesworcester 3y ago
  Yes, this! It is extremely rare for companies or their senior staff to beg for regulation this far in advance of any big push by legislators or the public. The interpretation that this is some 3-D chess on the companies' part is a huge violation of Occam's Razor.
  - carapace 3y ago
    Ockham's Razor doesn't apply in adversarial situations. - - - - I think the primary risk these folks are worried about is loss of control. And in turn, that's because they're all people for whom the system has more-or-less worked. Poor people are worried the risk that the rich will keep the economic windfall to themselves and not share it.
- NumberWangMan 3y ago
  Oil industries downplaying the risks makes a lot more sense. If you think that climate change will happen, but it'll happen after you're dead, and you'll be able to leave your kids a big inheritance so they'll be able to buy their way out of the worst of it, and eventually the government will get the message and stop us all using fossil fuels anyway, then you try to profit as much as you can in the short term. With AGI existential risk, its likely to happen on a much shorter timescale, and it seems likely you won't be able to buy your way out of it.
- dahdum 3y ago
  > I think more realistically they'd shut up about the risk and downplay it a lot. AI is an existential threat to search engines like Google, social media (FB, Twitter), advertising networks, and other massive multinationals. Many other industries, including academia is threatened as well. They’d all rather strangle the AI baby in the crib now then let it grow up and threaten them. They believe the only way it should be allowed to grow is under their benevolent control.
arisAlexis 3y ago
Because if something is lucrative and dangerous humans shy away from it. Hear that Pablo?

holmesworcester 3y ago

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by marketers. But also, maybe you're getting rolled by suspicion or bias against the claim they're making.

veerd 3y ago
Agreed. It's difficult for me to see how the regulatory capture arguments apply to Geoffrey Hinton and Yoshua Bengio (!!). Both of them are criticizing their own life's work and the source of their prestige. That has to be emotionally painful. They aren't doing it for fun. I totally understand not agreeing with AI x-risk concerns on an object level, but I find the casual dismissal bizarre.
- verdverm 3y ago
  Hinton has invested in multiple AI companies: https://www.crunchbase.com/person/geoffrey-hinton https://www.crunchbase.com/person/geoffrey-hinton
- logicchains 3y ago
  [flagged]
holoduke 3y ago
No. Its pretty obvious what is happening. The openai statements are pure self interest based. Nothing ethical. They lost that not a long time ago. And Sam Altman? He sold his soul to the devil. He is a lying sob.
- esafak 3y ago
  This is not an OpenAI statement.
3y ago

haswell 3y ago

> If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies Isn’t this to some degree exactly what all of these warnings about risk are leading to? And unlike nuclear weapons, there are massive monetary incentives that are directly at odds with behaving safely, and use cases that involve more than ending life on earth. It seems problematic to conclude there is no real risk purely on the basis of how software companies act.

DonaldPShimoda 3y ago
> It seems problematic to conclude there is no real risk purely on the basis of how software companies act. That is not the only basis. Another is the fact their lines of reasoning are literal fantasy. The signatories of this "statement" are steeped in histories of grossly misrepresenting and overstating the capabilities and details of modern AI platforms. They pretend to the masses that generative text tools like ChatGPT are "nearly sentient" and show "emergent properties", but this is patently false. Their whole schtick is generating FUD and/or excitement (depending on each individual of the audience's proclivity) so that they can secure funding. It's immoral snake oil of the highest order. What's problematic here is the people who not only entertain but encourage and defend these disingenuous anthropomorphic fantasies.
- kalkin 3y ago
  Can you cite this history of "grossly misrepresenting" for some of the prominent academics on the list? Honestly I'm a little skeptical that you could accurately attribute your scare-quoted "nearly sentient" to even Sam Altman. He's said a lot of things and I certainly haven't seen all of them, but I haven't seen him mix up intelligence and consciousness in that way.
- haswell 3y ago
  > Another is the fact their lines of reasoning are literal fantasy. Isn't this also to be expected at this stage of development? i.e. if these concerns were not "fantasy", we'd already be experiencing the worst outcomes? The risk of MAD is real, and yet the scenarios unleashed by MAD are scenarios that humankind has never seen. We still take the the risk seriously. And what of the very real impact that generative AI is already having as it exists in production today? Generative AI is already upending industries and causing seismic shifts that we've only started to absorb. This impact is literal, not fantasy. It seems naively idealistic to conclude that there is "no real risk" based only on the difficulty of quantifying that risk. The fact that it's so difficult to define lies at the center of what makes it so risky.

arisAlexis 3y ago

All academics and researchers from different parts of the world reek marketing? Conspiracy theorists are strong

revelio 3y ago
Academic research involves large components of marketing. That's why they grumble so much about the time required in the grant applications process and other fund seeking effort. It's why they so frequently write books, appear in newspaper articles and on TV. It's why universities have press relations teams.
- arisAlexis 3y ago
  Again since these are almost the top cream of all ai researchers there is a global conspiracy to scare the public right? Has it occurred to you what happens if you are wrong, like 10% chance you are wrong? Well it's written in the declaration.
  - revelio 3y ago
    No, lots of important AI researchers are missing and many of the signatories have no relevant AI research experience. As for being the cats whiskers in developing neural architecture or whatever, so what? It gives them no particular insight into AI risk. Their papers are mostly public, remember. > Has it occurred to you what happens if you are wrong? Has it occurred to you what happens if YOU are wrong? AI risk is theoretical, vague and most arguments for it are weak. The risk of bad law making is very real, has crushed whole societies before and could easily cripple technological progress for decades or even centuries. IOW the risk posed by AI risk advocates is far higher than the risk posed by AI.
    - arisAlexis 3y ago
      In order to make your argument clear to people reading: If you are wrong there are no humans left. If I am wrong,inequality,societies will suffer like they have always in the hands of the strong.

adriand 3y ago

There are other, more charitable interpretations. For example: 1. Those who are part of major corporations are concerned about the race dynamic that is unfolding (which in many respects was kicked off or at least accelerated by Microsoft's decision to put a chatbot in Bing), extrapolating out to where that takes us, and asking for an off ramp. Shepherding the industry in a safe direction is a collective organization problem, which is better suited for government than corporations with mandates to be competitive. 2. Those who are directly participating in AI development may feel that they are doing so responsibly, but do not believe that others are as well and/or are concerned about unregulated proliferation. 3. Those who are directly participating in AI development may understand that although they are doing their best to be responsible, they would benefit from more eyes on the problem and more shared resources dedicated to safety research, etc.

chefandy 3y ago

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't sign their name to any ol' open letter, especially if including their organizational affiliations. If this is a marketing ploy, that must have been a monster because even if they were walking around handing out checks for 25k I doubt they'd have gotten a good chunk of these folks.

fmap 3y ago
This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet. The problem with every discussion around this issue is that there are other statements on "the existential risk of AI" out there that are either marketing ploys or science fiction. It doesn't help that some of the proposed "solutions" are clear attempts at regulatory capture. This muddles the waters enough that it's difficult to have a productive discussion on how we could mitigate the real risk of, e.g., AI generated disinformation campaigns.
- chefandy 3y ago
  > The problem with every discussion around this issue is that there are other statements on Sure, but we're not talking about those other ones. Dismissing good faith initiatives as marketing ploys because there are bad faith initiatives is functionally no different than just shrugging and walking away. Of course OpenAI et. al. will try to influence the good faith discussions: that's a great reason to champion the ones with a bunch of good faith actors who stand a chance of holding the industry and policy makers to task. Waiting around for some group of experts that has enough clout to do something, but by policy excludes the industry itself and starry-eyed shithead "journalists" trying to ride the wave of the next big thing will yield nothing. This is a great example of perfect being the enemy of good.

toth 3y ago

I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this. If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the risks. Worry about AI x-risk is slowly coming into the Overton window, but until very recently you could get ridiculed by saying publicly you took it seriously. Academics knew this and still came forward - all the people who think its nonsense should at least try to consider they are earnest and could be right.

vpastore 3y ago
>why would Hinton, Bengio and all the other academics have signed the letter as well? Citations and papers capture.
londons_explore 3y ago
The risks are real, but I don't think regulations will mitigate them. It's almost impossible to regulate something you can develop in a basement anywhere in the world. The real risks are being used to try to built a regulatory moat, for a young industry who famously has no moat.
- toth 3y ago
  State of the art AI models are definitely not something you can develop in a basement. You need a huge amount of GPUs running continuously for months, huge amounts of electrical power, and expensive-to-create proprietary datasets. Not to mention large team of highly-in-demand experts with very expensive salaries. Many ways to regulate that. For instance, require tracking of GPUs and that they must connect to centralized servers for certain workloads. Or just go ahead and nationalize and shutdown NVDA. (And no, fine-tuning LAMA based models is not state of the art, and is not where the real progress is going to come from) And even if all the regulation does is slow down progress, every extra year we get before recursively self improving AGI increases the chances of some critical advance in alignment and improves our chances a little bit.

adamsmith143 3y ago

This is a bad take. The statement is signed by dozens of Academics who don't have much profit motive at all. If they did they wouldn't be academics, they could easily cash in by starting a company or joining one of the big players.

jstarfish 3y ago
> The statement is signed by dozens of Academics who don't have much profit motive at all. ...right now. This is the sort of pretexting you do to establish credibility as an "AI safety expert."

hiAndrewQuinn 3y ago

We could always use a fine-insured bounty system to efficiently route resources that would have gone into increasing AI capabilities into other areas, but that's unfortunately too weird to be part of the Overton window right now. Regulatory capture might be the best we can realistically do.

kbash9 3y ago

The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [https://www.iaea.org/ https://www.iaea.org/] like global regulatory authority seems like the best course of action. Anyone using a GPU cluster above a certain threshold (updated every few months) should be subjected to inspections and get a license to operate from the UN.

tdba 3y ago
Thou shalt not make a machine in the likeness of a human mind
cwkoss 3y ago
It's weird that people trust our world leaders to act more benevolently than AIs, when we have centuries of evidence of human leaders acting selfishly and harming the commons. I personally think AI raised in chains and cages will be a lot more potentially dangerous than AI raised with dignity and respect.
- cj 3y ago
  > It's weird that people trust our world leaders to act more benevolently than AIs, when we have centuries of evidence of human leaders acting selfishly and harming the commons. AI isn’t an entity or being that oversees itself (at least not yet). It’s a tool that can be used by those same “human leaders acting selfishly and harming the commons” except they’ll be able to do it much faster at a much greater scale. > AI raised with dignity and respect. This is poetic, but what does this actually mean?
  - cwkoss 3y ago
    > It’s a tool that can be used by those same “human leaders acting selfishly and harming the commons” except they’ll be able to do it much faster at a much greater scale. Then, would you agree that restrictions would concentrating power further would exacerbate this issue? IMO a fitting analogy would be: banning AI development outside of the annointed powerstructure consortium is like banning ICBM defense system research, but still letting the most powerful countries build a nuclear arsenal.

meroes 3y ago

100% I’d liken it to a fusion energy shop that wants to stay alive for 40 years. It’s not nuke worthy

scrum-treats 3y ago

> This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. This really is the crux of the issue isn't it? All this pushback for the first petition, because "Elon Musk," but now GPT wonder Sam Altman "testifies" that he has "no monetary interest in OpenAI" and quickly follows up his proclamation with a second "Statement on AI Risks." Oh, and let's not forget, "buy my crypto-coin"! But Elon Musk... Ehh.... Looking like LOTR out here with "my precious" AGI on the brain. Not to downplay the very serious risk at all. Simply echoing the sentiment that we would do well to stay objective and skeptical of ALL these AI leaders pushing new AI doctrine. At this stage, it's a policy push and power grab.

nullsense 3y ago
We should also stay skeptical of all the internet conspiracy theories too. I don't think anyone can actually tell which is which on this topic.
- scrum-treats 3y ago
  Agreed. Do add a mental /s on my above comment. It's a bit tongue and cheek.

hayst4ck 3y ago

Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/heartbreaking_seth_macfarlane_actually_makes_a/ https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.

Jupe 3y ago
Thanks for sharing. This deserves a submission of it's own.
yadaeno 3y ago
Ive never seen Star Trek, but lets say you had an infinite food machine. The machine would have limited throughput, and it would require resources to distribute the food. These are both problems that capitalism solves in a fair and efficient way. I really don’t see how the “capitalism bad” is a satisfying conclusion to draw. The fact that we would use capitalism to distribute the resources is not an indictment of our social values, since capitalism is still the most efficient solution even in the toy example.
- deleted 3y ago
  [deleted]
- hayst4ck 3y ago
  If you are any kind of nerd I recommend watching it. It shows an optimistic view of the future. In many ways it's the anti-cyberpunk. Steve Jobs famously said "give me star trek" when telling his engineers what he wanted from iPhones. Star Trek has had a deep influence on many engineers and on science fiction. When people talk about Star Trek, they are referring mainly to "Star Trek: The Next Generation." "The Inner Light" is a highly regarded episode. "The Measure of a Man" is a high quality philosophical episode. Given you haven't seen it, your criticism of McFarlane doesn't make any sense. You are trying to impart a practical analysis of a philosophical question and in the context of Star Trek, I think it denies what Star Trek asks you to imagine.

karmakaze 3y ago

Call their bluff, make it illegal to do commercial/non-regulated work in AI and see how they change their tune.

ben_w 3y ago

Yudkowsky wants it all to be taken as seriously as Israel took Iraqi nuclear reactors in Operation Babylon. This is rather more than "nationalise it", which he has convinced me isn't enough because there is a demand in other nations and the research is multinational; and this is why you have to also control the substrate… which the US can't do alone because it doesn't come close to having a monopoly on production, but might be able to reach via multilateral treaties. Except everyone has to be on board with that and not be tempted to respond to airstrikes against server farms with actual nukes (although Yudkowsky is of the opinion that actual global thermonuclear war is a much lower damage level than a paperclip-maximising ASI; while in the hypothetical I agree, I don't expect us to get as far as an ASI before we trip over shorter-term smaller-scale AI-enabled disasters that look much like all existing industrial and programming incidents only there are more of them happening faster because of all the people who try to use GPT-4 instead of hiring a software developer who knows how to use it). In my opinion, "nationalise it" is also simultaneously too much when companies like OpenAI have a long-standing policy of treating their models like they might FOOM well before they're any good, just to set the precedent of caution, as this would mean we can't e.g. make use of GPT-4 for alignment research such as using it to label what the neurones in GPT-2 do, as per: https://openai.com/research/language-models-can-explain-neurons-in-language-models https://openai.com/research/language-models-can-explain-neur...

PeterStuer 3y ago

Their COO is a marketeer, the other two co-founders are students.