4 ms·

Very much agree with this. If the signatories believed this, they would shut down development. We can be conveniently distracted from large societal disruption

Very much agree with this. If the signatories believed this, they would shut down development. We can be conveniently distracted from large societal disruption such as huge changes in the job market from automation if there’s a media frenzy on the less likely and still hypothetical extinction AI.

kalkin 3y ago
What do you think of the calls for regulation or licensing of AI?
- JumpCrisscross 3y ago
  > What do you think of the calls for regulation or licensing of AI Misdirection. We see a generic call for regulation, or unrealistic call for a global pause with no answers to how it would be coördinated or enforced. When actual regulations are put forward, they're rejected without a counter-offered solution [1]. [1] https://www.reuters.com/technology/openai-may-leave-eu-if-regulations-bite-ceo-2023-05-24/ https://www.reuters.com/technology/openai-may-leave-eu-if-re...
  - kalkin 3y ago
    Obviously companies are going to try to shape any regulation to their benefit. What I'm wondering is if people who see x-risk worries as distracting from real risks are in agreement with the narrative I've seen a lot on HN recently that calls for regulation are just anti-competitive attempts at regulatory capture.
    - JumpCrisscross 3y ago
      > companies are going to try to shape any regulation to their benefit There is a difference between trying to shape regulation to your advantage and engaging in bad faith. There is no indication there is real regulation that addresses the problem these folks are brining up that they would support.
      - kalkin 3y ago
        Who are "these folks"? OpenAI specifically? I'm not arguing that they're operating in good faith, but it's also not clear to me that them proposing specific regulation would be helpful - there seem to be a lot of people who see anything like that as automatically an attempt to stifle competition. Many other people talking about x-risk signed the letter calling for a pause on training new big models, which seems pretty concrete to me. Does that make sense to you? If not, do you have other regulation you'd prefer?
        JumpCrisscross 3y ago
        > other people talking about x-risk signed the letter calling for a pause on training new big models, which seems pretty concrete to me. Does that make sense to you It's totally unrealistic. How are you going to coördinate a global pause on AI research? Nobody has even bothered proposing a framework because of how ludicrous it is for this to be what Beijing, Washington, Brussels and New Delhi focus on.
        kalkin 3y ago
        Right, you said before that's unrealistic, sorry. But it's not clear to me what you think would be a good idea, which makes it difficult to see what you think would be a serious proposal rather than misdirection. It appears to me that you are taking it as evidence of bad faith if someone proposes regulation either stronger (pause) or weaker (OpenAI amendments to proposed EU rules) than what you prefer - but you also haven't said what you do prefer. What am I missing? Personally, while I don't think the pause is realistic, I don't think advocates necessarily have any obligation to propose realistic ideas when they're not in a position to actually write any laws and are only able to potentially shift the window of what politicians and media consider acceptable.
        JumpCrisscross 3y ago
        > t's not clear to me what you think would be a good idea, which makes it difficult to see what you think would be a serious proposal rather than misdirection Registration. Not even licensing. If you're training a model, you file a form describing who you are, who you work for, who you're working with and what you're doing with which data for what purpose. This lets professionals, policy makers and the public get a sense for the landscape so we can make rules with some sense of reality.
- Freebytes 3y ago
  It will be impossible to regulate and impossible to stop. The code for AGI will not be some monolith of software architecture. The code will likely be simple. This means that someone in their basement could build it. The steps to get there are challenging, though. A single person could have developed the transformer architecture. A single person could have used 4-bit quantization and developed an AI that is just as good as ChatGPT and have it run on their local machine. The difficulties are figuring out the best 'needle in the haystack' to solve the problem. This requires research, and this process happens much faster if you have more people working on it. For years, many people did not put any energy into AI systems because the hardware was not here yet. The hardware is here now. The cat is out of the bag.
hackinthebochs 3y ago
>Very much agree with this. If the signatories believed this, they would shut down development. This just ignores the very real coordination problem. The signatories do not represent the entirety of AI development, nor do they want to unilaterally forgo business opportunities that the next man will exploit. Government is the proper place to coordinate these efforts, and so that is where they appeal.
- rimeice 3y ago
  It’s a wording and media frenzy point, personally if I thought I was doing something that was going to wholly or partly cause the “extinction” of the human race. I would stop doing it. These CEOs signing this statement and running these companies are not despotic psychopaths and have the ability to stop what they’re doing. So to me, it seems this type of wording is hyperbole and will cause us to miss some of the very real, very present and very large risks of AI. Those risks as you say can and should be dealt with government coordination but are distracted from if the media only talk about extinction.
  - JohnFen 3y ago
    This is what bothers me. The risks they're talking about are the most unrealistic ones you can think of. In the meantime, they're completely ignoring much more likely devastating (although not extinction-level) risks. It smells like bullshit being espoused to push an agenda, but I can't tell what the agenda is. My guess: play up the huge unrealistic risks in order to distract from the more realistic ones. Two things about this tech that I'm personally not worried at all about: "evil agi" and "only the elites will control this".
  - hackinthebochs 3y ago
    >personally if I thought I was doing something that was going to wholly or partly cause the “extinction” of the human race. I would stop doing it There are so many things wrong with this line of thinking. First, it mischaracterizes the issues. Few people believe AGI guarantees the extinction of humanity. The issue is that there is a significant potential for extinction and thus we need a coordinated effort to either manage this risk or prevent its creation. It does little to stop the coming calamity to single-handedly abstain from continuing to build. Coordination is a must. Besides, most people will think they stand a greater chance of building it safely than the next guy. The coordination is required to keep other people from being irresponsible. Human hubris knows no limits. The other mistake is misjudging nerd psychology. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it". It is a grave error to discount the motivation of trying to solve a really sweet technical problem. Ultimately these kinds of claims are self-serving, they provide rational cover to justify your predetermined beliefs that those calling for regulation are trying stifle competition. Folks don't want to be left out of the fun. The justification is in service to the motivation.

kalkin 3y ago

What do you think of the calls for regulation or licensing of AI?

JumpCrisscross 3y ago
> What do you think of the calls for regulation or licensing of AI Misdirection. We see a generic call for regulation, or unrealistic call for a global pause with no answers to how it would be coördinated or enforced. When actual regulations are put forward, they're rejected without a counter-offered solution [1]. [1] https://www.reuters.com/technology/openai-may-leave-eu-if-regulations-bite-ceo-2023-05-24/ https://www.reuters.com/technology/openai-may-leave-eu-if-re...
- kalkin 3y ago
  Obviously companies are going to try to shape any regulation to their benefit. What I'm wondering is if people who see x-risk worries as distracting from real risks are in agreement with the narrative I've seen a lot on HN recently that calls for regulation are just anti-competitive attempts at regulatory capture.
  - JumpCrisscross 3y ago
    > companies are going to try to shape any regulation to their benefit There is a difference between trying to shape regulation to your advantage and engaging in bad faith. There is no indication there is real regulation that addresses the problem these folks are brining up that they would support.
    - kalkin 3y ago
      Who are "these folks"? OpenAI specifically? I'm not arguing that they're operating in good faith, but it's also not clear to me that them proposing specific regulation would be helpful - there seem to be a lot of people who see anything like that as automatically an attempt to stifle competition. Many other people talking about x-risk signed the letter calling for a pause on training new big models, which seems pretty concrete to me. Does that make sense to you? If not, do you have other regulation you'd prefer?
      - JumpCrisscross 3y ago
        > other people talking about x-risk signed the letter calling for a pause on training new big models, which seems pretty concrete to me. Does that make sense to you It's totally unrealistic. How are you going to coördinate a global pause on AI research? Nobody has even bothered proposing a framework because of how ludicrous it is for this to be what Beijing, Washington, Brussels and New Delhi focus on.
        kalkin 3y ago
        Right, you said before that's unrealistic, sorry. But it's not clear to me what you think would be a good idea, which makes it difficult to see what you think would be a serious proposal rather than misdirection. It appears to me that you are taking it as evidence of bad faith if someone proposes regulation either stronger (pause) or weaker (OpenAI amendments to proposed EU rules) than what you prefer - but you also haven't said what you do prefer. What am I missing? Personally, while I don't think the pause is realistic, I don't think advocates necessarily have any obligation to propose realistic ideas when they're not in a position to actually write any laws and are only able to potentially shift the window of what politicians and media consider acceptable.
        JumpCrisscross 3y ago
        > t's not clear to me what you think would be a good idea, which makes it difficult to see what you think would be a serious proposal rather than misdirection Registration. Not even licensing. If you're training a model, you file a form describing who you are, who you work for, who you're working with and what you're doing with which data for what purpose. This lets professionals, policy makers and the public get a sense for the landscape so we can make rules with some sense of reality.
Freebytes 3y ago
It will be impossible to regulate and impossible to stop. The code for AGI will not be some monolith of software architecture. The code will likely be simple. This means that someone in their basement could build it. The steps to get there are challenging, though. A single person could have developed the transformer architecture. A single person could have used 4-bit quantization and developed an AI that is just as good as ChatGPT and have it run on their local machine. The difficulties are figuring out the best 'needle in the haystack' to solve the problem. This requires research, and this process happens much faster if you have more people working on it. For years, many people did not put any energy into AI systems because the hardware was not here yet. The hardware is here now. The cat is out of the bag.

hackinthebochs 3y ago

>Very much agree with this. If the signatories believed this, they would shut down development. This just ignores the very real coordination problem. The signatories do not represent the entirety of AI development, nor do they want to unilaterally forgo business opportunities that the next man will exploit. Government is the proper place to coordinate these efforts, and so that is where they appeal.

rimeice 3y ago
It’s a wording and media frenzy point, personally if I thought I was doing something that was going to wholly or partly cause the “extinction” of the human race. I would stop doing it. These CEOs signing this statement and running these companies are not despotic psychopaths and have the ability to stop what they’re doing. So to me, it seems this type of wording is hyperbole and will cause us to miss some of the very real, very present and very large risks of AI. Those risks as you say can and should be dealt with government coordination but are distracted from if the media only talk about extinction.
- JohnFen 3y ago
  This is what bothers me. The risks they're talking about are the most unrealistic ones you can think of. In the meantime, they're completely ignoring much more likely devastating (although not extinction-level) risks. It smells like bullshit being espoused to push an agenda, but I can't tell what the agenda is. My guess: play up the huge unrealistic risks in order to distract from the more realistic ones. Two things about this tech that I'm personally not worried at all about: "evil agi" and "only the elites will control this".
- hackinthebochs 3y ago
  >personally if I thought I was doing something that was going to wholly or partly cause the “extinction” of the human race. I would stop doing it There are so many things wrong with this line of thinking. First, it mischaracterizes the issues. Few people believe AGI guarantees the extinction of humanity. The issue is that there is a significant potential for extinction and thus we need a coordinated effort to either manage this risk or prevent its creation. It does little to stop the coming calamity to single-handedly abstain from continuing to build. Coordination is a must. Besides, most people will think they stand a greater chance of building it safely than the next guy. The coordination is required to keep other people from being irresponsible. Human hubris knows no limits. The other mistake is misjudging nerd psychology. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it". It is a grave error to discount the motivation of trying to solve a really sweet technical problem. Ultimately these kinds of claims are self-serving, they provide rational cover to justify your predetermined beliefs that those calling for regulation are trying stifle competition. Folks don't want to be left out of the fun. The justification is in service to the motivation.