23 ms·

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be c

• Aithr 3y ago
  "Stop new development." is regulatory capture.
  • esafak 3y ago
    Stop existing development too. Carve an exception for academic research.
    • BlueTemplar 3y ago
      «Exception for academic research» seems to be how we got Covid, so...
• rolisz 3y ago
  And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.
  • TeMPOraL 3y ago
    With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing that, bomb it to rubble (because if we don't, some other signatory of the treaty will bomb you for us)".
    • neonihil 3y ago
      The problem with that is that the entity supposed to "bomb it to rubble" and the entity pushing for AI development happens to be the same entity. Maybe the confusion why people can't see this clearly stems from the fact that tech development in the US has mostly been done under the umbrella of a private enterprise. But if one has a look at companies like Palantir, it's going to become quite obvious what is the main driver behind AI development.
    • Dig1t 3y ago
      Are you really suggesting drone striking data centers? What about running AI models on personal hardware? Are we going to require that everyone must have a chip installed in their computer that will report to the government illegal AI software running on the machine? You are proposing an authoritarian nightmare.
      • pixl97 3y ago
        Hence the entire point of this discussion... can we avoid the future nightmares that are coming in one way or another? Make AI = nightmare, stop people from making AI = nightmare. The option that seems to be off the table is people going, hell making AI is a nightmare so I won't voluntarily without the need for a police state.
    • RandomLensman 3y ago
      No-one wants a nuclear war over imaginary risks, so not happening.
      • TeMPOraL 3y ago
        We better find a way to make it happen, because right now, climate change x-risk mitigation is also not happening, for the same reason.
        • RandomLensman 3y ago
          Climate change mitigation is actually happening. Huge investment flows are being redirected and things are being retooled, you might not see it, but don't get caught out by the doomers there.
  • olalonde 3y ago
    This underscores the crux of the issue. Our collective response to curbing emissions has been lackluster, largely due to the fact that the argument for it being beneficial hasn't sufficiently swayed a significant number of people. The case asserting that AI R&D poses an existential risk will be even harder to make. It's hard to enforce a ban when a large part of the population, and perhaps entire countries, do not believe the ban is justified. This contrasts with the near-universal agreement surrounding the prohibition of chemical and biological weapons, or the prevention of nuclear weapons proliferation, where the consensus is considerably stronger.
  • ac29 3y ago
    I think you are really underestimating how much progress has been made with renewable energy. Taking a quick look a CAISO, the California grid is currently 65% powered by renewables (mostly solar). This is a bit higher than average, but it certainly wouldnt have been nearly as high even 5 years ago, much less 25 years ago.
• deleted 3y ago
  [deleted]
• peterth3 3y ago
  Licensing can definitely turn into regulatory capture if it expands enough. It’s effectively a barrier to entry defined by the incumbent.
  • jamilton 3y ago
    They're saying don't do licensing, just make it illegal.
    • peterth3 3y ago
      Sorry, who’s “they” here? The incumbents OpenAI / DeepMind / Anthropic?
      • ben_w 3y ago
        Yudkowsky and similar. (I'm a lot more optimistic than such doomers, but I can at least see their position is coherent; LeCun has the opposite position, but unfortunately, while I want that position to be correct, I find his position to be the same kind of wishful thinking as a factory boss who thinks "common sense" beats health and safety regulations).
      • JoshTriplett 3y ago
        No, the incumbents think it should continue. The xrisk folks are not the AI companies. The AI companies pay lip service to xrisk and try to misdirect towards regulation they think they can work with. Judging by this thread, the misdirection is working.
      • jamilton 3y ago
        In this case I meant the GP comment from JoshTriplett.
• wood_spirit 3y ago
  You can’t _stop_ it. You can make it illegal in some countries. But outlawing it just means some obvious candidate countries or private actors develop it first instead.
  • candiodari 3y ago
    I think these attempts will simply point out that math is a higher power than law. If something becomes possible, and becomes easy, there is nothing law can do to stop it. Just look at the attempts to stop drugs in North West Europe. Huge sentences (so huge they are in fact never fully carried out), tens of dead police officers on a yearly basis, triple that in civilian victims and drugs are easier to get than ever before.
    • FeepingCreature 3y ago
      The aim is to stop it from becoming easy. The drug war would look very different if good cocaine was only produced in one country with extremely costly chemistry machines from one company.
      • upupupandaway 3y ago
        If good cocaine was only produced in one country with extremely costly machines, a good-enough alternative would appear almost immediately. This has happened millions of times in the history of humanity, I refuse to believe that people haven't learned this lesson.
        • _0ffh 3y ago
          > I refuse to believe that people haven't learned this lesson Oh, I believe they absolutely haven't, when not having learnt it buys them money and influence. As for the bulk of the population, they will accept whatever the corporate media tell them to.
        • FeepingCreature 3y ago
          And yet, it doesn't seem to be that easy with chips.
• topspin 3y ago
  > "Stop" is not regulatory capture. "Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.
  • maeil 3y ago
    If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.
    • topspin 3y ago
      Fantasies, bordering on threats.
      • ben_w 3y ago
        Of course it's a threat. Civilians doing that would be bad. But governments do much the same via military intervention on a regular basis, hence Yudkowsky talking about airstrikes that one time.
      • maeil 3y ago
        I'm not suggesting to do that, I'm just showing that it's incredibly possible. You could also go with "Tomorrow a massive storm strikes the Netherlands and huge earthquakes striking Taiwan and the US Westcoast", if you have emotional objections to my first example.
    • Maken 3y ago
      It's delayed at best.
      • maeil 3y ago
        Incredibly delayed, giving at least decades to figure things out.
    • YetAnotherNick 3y ago
• upupupandaway 3y ago
  The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop countries from developing weapons that will destroy us all tomorrow morning and take billions of USD to develop, imagine thinking we can stop matrix multiplication globally. I don't want to derail into an ad hominem, but frankly, it's almost the only option left here.
  • jeffparsons 3y ago
    My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.
    • concordDance 3y ago
      He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.
      • jeffparsons 3y ago
        It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabotage competing efforts. Governments have done much the same in the past with, e.g. conventional weapons, nuclear weapons, and cryptography, with varying levels of success. If we're all dead anyway otherwise, then I don't see how that can possibly be a worse card. Edit: or even try to convince _all_ governments to do this so that they come out on top. At least then there's a greater chance that the bleeding edge of this tech will be under the stewardship of a deliberate attempt for a country to dominate the world, rather than some bored kid who happens to stumble upon the recipe for global paperclips.
• RandomLensman 3y ago
  There is no existential risk now and there might never be. So "stop", especially with the use of violence, is a pretty odd position to take to say it mildly.
• api 3y ago
  Yudkowski is a crackpot. Nobody would take him seriously if he wasn’t one of Peter Thiel’s pet crackpots. I should have done a ton more intellectual edgelording in the oughts and grabbed some Thielbucks. I could have recanted it all later. Of course I probably would have had to at least flirt with eugenics. That seems to be the secret sauce.
• m_a_g 3y ago
  Could you explain to me how AGI is an existential threat to humanity? All the arguments I've read so far feel so far-fetched it's unbelievable. It's in the order of making an AGI and making it control the Pentagon (!?).
  • hollerith 3y ago
    The AGI does not need to control the Pentagon. It might for example invent novel technologies that you and I cannot even imagine because we are much less capable at inventing new technologies than the AGI will be. If members of an alien civilization much more advanced than our own suddenly appeared in orbit around Earth, would that not be dangerous?
    • m_a_g 3y ago
      That I understand, but my thinking automatically goes to control and restrictions. A superintelligent AGI that's as restricted as chatgpt in terms of what it can actually control seems... not so existentially dangerous? In that scenario, that actual problem would rise by not-so-smart humans listening to that AGI blindly, no?
      • hollerith 3y ago
        I'm confused by your "as restricted as chatgpt". Hasn't chatgpt been given unrestricted access to the web? Surely if chatgpt were sufficiently capable, it would be able to use that access to hack web sites and thereby gain control over money. Then it can use the web to buy things with that money just like you and I can. It is a lot harder to prevent the sysadmins who have temporary control over you from noticing that you just hacked some web sites and stole some money, but a sufficiently capable AI would know about the sysadmins and would take pains to hide these activities from them.
        • johnthewise 3y ago
          It doesn't even need to hack or do anything illegal. It can make money trading stocks, predicting or modelling far better than market participants, use money to influence the real world, bribe politicians etc. Like with few billions, i bet you can create a political party, influence campagins and end up with real human followers and lobbying power. It doesn't even need to run on its own, if there was a trading bot that said, in order for it to make me money, I need to give it internet access and run it 24/7, rent gpu clusters and s3 buckets etc, I'd do it. this is the most probable scenario imo, AI creating beneficial scenarios for subset of people so that we comply with its requests. very little convincing is necessary
• EVa5I7bHFq9mnYK 3y ago
  What stop? Will Kim Jong Un stop? Hell no. There is no stop, the genie is out of the bottle.
• birdyrooster 3y ago
  lmao you have to be kidding me
• veidr 3y ago
  I don't want to be rude; I understand where you are coming from (at least in terms of single-minded risk mitigation from a theoretical and unconstrained viewpoint), but this reads like Mike Huckabee's 'just keep an aspirin pill pinned between your knees' advice to teenage girls on how not to get pregnant. There is obviously (!!!!!!!!!) no way in this — or any imaginable — timeline that we are going to just "stop". (And it's hard to imagine any mechanism that could enforce "stop" that isn't even worse than the scenarios we're already worrying about. How does that work? Elon or Xi-Poohbear have to personally sign off on installations of more than 1000 cores?)
  • pixl97 3y ago
    Butlerian Jihad level revolt by the average person? No reason to see that happening these days, and by the time we're willing to it may already be too late.
• j45 3y ago
  Except while saying stop, and saying they will, none have stopped because apparently others won’t. Since developing the danger first to help regulate out everyone who doesn’t cross the line first.. open source alternatives are increasingly important. The regulations announced in the US in the past week effectively make it for the few and not the many. There’s been a few rumblings that OpenAI has achieved AGI recently as well.
• slashdev 3y ago
  Stop is a fantasy pretty much prohibited by game theory in a world with multiple competing states. It’s not going to happen.
• pdonis 3y ago
  But the AI leaders who have been calling for the US government to act are not asking for AI research to stop. Nor are they asking for the kind of international policing that Yudkowsky describes. They are asking for regulations much like the ones the Biden administration just published--which are indeed regulatory capture since all they do is build a moat around current AI companies to protect their profits. They do not stop AI research or make it any less dangerous.
• dragonwriter 3y ago
  Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it poses a greater and more immediate existential risk to humanity than AI ever could; it is largely mitigated (though still a bigger X-Risk than AI) because no one with the capacity to implement it in any substantive way takes him seriously at the moment. But the people selling AI X-Risk that anyone with any policy influence is listening to are AI sellers using it to push regulatory capture, so those are the people being talked about when people are discussing X-Riskers as a substantive force in policy discussions, not the Yudkowsky cult, which, to the extent it is relevant, mostly provides some background fear noise which the regulatory capture crew leverages.
  • 3rd3 3y ago
    > Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it Could you provide a link to where he said this?
    • dragonwriter 3y ago
      I think he's said it many times in slightly different ways, but one notable one: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... The moratorium on new large training runs needs to be indefinite and worldwide. There can be no exceptions, including for governments or militaries. If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S. and in China and on Earth. . . . Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make immediate multinational agreements to prevent the prohibited activities from moving elsewhere. Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.
• 0x00_NULL 3y ago
  No. Sorry, but this is wrong and Yudkowsky is both naïve and mostly exists in the domain of fan fiction. There are way way way too many issues that are addressed with a hand-wave around scenarios like “AI developing super intelligence in secret and spreading itself around decentralized computers while getting forever smarter by reading the internet.” Too many of his arguments depend on stealth for systems that take up datacenters and need whole-city-block scales of power to operate. Physically, it’s just not possible to move these things. Economically, it wouldn’t be either close to plausible either. Power is expensive and it is the single most monitored element in every datacenter. There is nothing large that happens in a datacenter that is not monitored. There is nothing that is going to train on meaningful datasets in secret. “What about as technology increases and we get more efficient at computing?” We use more power for computing every year, not less. Yes, we get more efficient, but we don’t get less energy intensive. We don’t substitute computing capacity. We add more. The same old servers are still there contributing to the compute. Google has data centers filled with old shit. That’s why the standard core in GCP is 2 GHz. Sometimes, your project gets put on a box from 2010, other times, it gets put on a box from 2023. That process is abstracted so you can’t tell the difference. TLDR: Yudkowsky’s arguments are merely fan fiction. People don’t understand ML systems so they imagine an end state without understanding how to get there. These are the same people who imagine Light Speed Steam Trains. “We need faster rail service, so let’s just keep adding steam to our steam trains and accelerate to the speed of light.” That’s exactly what these AI Doom arguments sound like to people in the field. Your AI Doom scenario, although it might be very imaginative, is a Light Speed Steam Train. It falls apart the moment you try to trace a path from today to doom.
  • kubanczyk 3y ago
    Are you saying that the governments/agencies are incapable of hiding any big infra? That's a novel take.