5 ms·

Anyone serious about this should participate in pacing the frontier by quitting their job so as to not potentially contribute to the problematic progress. Be th

Anyone serious about this should participate in pacing the frontier by quitting their job so as to not potentially contribute to the problematic progress. Be the change you want to see! Curious how that isn't what happens.

mofeien 3mo ago
That's an application of the "Tragedy of the commons": One person stopping to push the frontier just means that this person will lose their job, the frontier will be pushed by the others. The solution is to push for a binding rule instead that will make everyone slow down in the racing towards artificial superintelligence.
- fidotron 3mo ago
  And this is how we absolve people of all personal responsibility. Such companies would not pay people that well if they were so fungible. There is no such possible binding rule either, a point that is so obvious that any smart person claiming otherwise is clearly up to something.
  - alach11 3mo ago
    We make binding rules all the time. We didn't fix the ozone layer by encouraging people to quit their jobs producing CFCs. We lobbied for international regulation to ban them.
    - fidotron 3mo ago
      In your example there is nothing to be gained by not complying. Failing to comply with what is in reality arms control is a widespread occurence. It's notable the only historical way to handle this (Schelling covered it at length) is tolerance of mutual espionage on a level that would simply not be tolerable to any of the groups involved. I should add one of Schelling's other pet subjects was "time to weapon being viable". i.e. the distinction between a nuclear power and non-nuclear power isn't quite right, as there's a huge class of countries that could become so fairly fast if they were so inclined, and that this situation is somewhat inevitable. It is this that leads to efforts to denuclearize, even on a civilian level, "problematic" countries. You want to stop frontier AI and you'll have to be rolling back a lot of existing deployed technology.
      - alach11 3mo ago
        Great comment. I haven't read Schelling but will check that out. I think you're right that international cooperation on the topic is likely impossible at present, for the reasons you stated. I'm not sure what it would take to shift the Overton window, but it's probably not pretty.
        fidotron 3mo ago
        So honestly Schelling was utterly brilliant. I cannot recommend all three of the books wikipedia classed as his notable works highly enough: https://en.wikipedia.org/wiki/Thomas_Schelling#Notable_works https://en.wikipedia.org/wiki/Thomas_Schelling#Notable_works There are footnotes in "The Strategy of Conflict" containing more profound insight than whole books that are published today. On this particular topic though "Strategy and Arms Control" is the one to go for.
      - Nition 3mo ago
        The Montreal Protocol has not been perfectly adhered to either. China started emitting quite a lot of banned CFCs again from around 2012[1], which didn't get noticed until 2018! By 2020 it had finally stopped[2]. [1] https://www.nature.com/articles/s41586-018-0106-2 https://www.nature.com/articles/s41586-018-0106-2 [2] https://www.nature.com/articles/d41586-021-00360-0 https://www.nature.com/articles/d41586-021-00360-0
  - polishTar 3mo ago
    Influencing change is more complicated than that. We live in a society that has systems of market competition between companies and economic/military competition between countries. There's not just one country working on frontier AI, and not just a few companies in each country either. The incentives of these systems ensure AI advancement will happen quickly from here on out, no matter what, absent governmental action. An individual researcher leaving the company does nothing, other than give their seat to someone else who likely doesn't share their same concerns. Alternatively, using the influence they do have in their position to directly try to advance AI safety research, advocate for govt action, and make public arguments attempting to persuade the public to pressure the govt to act when they otherwise wouldn't be inclined is far more likely to impact things for the better.
    - fidotron 3mo ago
      > Alternatively, using the influence they do have in their position to directly try to advance AI safety research, advocate for govt action, and make public arguments attempting to persuade the public to pressure the govt to act when they otherwise wouldn't be inclined is far more likely to impact things for the better. No, it just comes across as "I want to get paid as if I was doing the thing I am employed to do by threatening you all about how terrible it would be if I actually did it". They aren't even being semi imaginative and going on strike or anything.
  - teaearlgraycold 3mo ago
    Personal responsibility is a difficult concept for be to apply to massive efforts like this. When there’s one person and you can talk to them personal responsibility is the number one topic. When you’re talking about a multi-trillion dollar bet and hundreds of thousands of people it doesn’t make sense anymore. Same logic goes for other common issues where personal responsibility is lauded. Obesity - real solutions are policy and medicine. Abortion - education and contraceptives.
  - lifeformed 3mo ago
    Hoping everyone chooses to do the right thing is not an effective strategy for mitigating bad things.
- HarHarVeryFunny 3mo ago
  That may be a practical argument, but it's certainly not a moral one - it's like justifying selling drugs on the basis that if you don't do it somebody else will. If these frontier lab employees are actually sincere about wanting to do something about the risk of AI advancing faster than our ability to adapt to it, then what they should really be doing is lobbying their own employers to do something about it. Lead by example, and do something you have control over. Lobbying the government to stop yourself from doing something bad(!!) can't be taken as a serious position - it can only be taken as a plea to get the competition (China - open weights) shut down. It makes no sense to claim that China is only keeping up by copying you, yet China, not yourself, is the problem.
brap 3mo ago
“It is difficult to get a man to understand something, when his salary depends on his not understanding it”
deleted 3mo ago
[deleted]
abecedarius 3mo ago
Some of us considered seeking one of these jobs and decided not to. Some have quit (e.g. Hinton, OpenAI people after OpenAI reneged on safety commitments). Now you're getting this message from people who passed the previous filters. If you think a brake is indicated and not just easing up on the accelerator, you need collective action as the statement said.
- fidotron 3mo ago
  > you need collective action Come on, go on strike then! Have a week of protest and picket the offices in SFO. Let's see what happens. (I'm aware that's not quite what you meant, but it is the classic meaning of that phrase).
  - anon373839 3mo ago
    Mass resignation perhaps?
    - 0xDEAFBEAD 3mo ago
      ...so the only remaining employees give no thought to safety? Unionization might be a little more effective.
0xDEAFBEAD 3mo ago
FWIW, here's an account from a guy who quit his job at Google Deepmind recently: https://turntrout.com/why-i-left-google-deepmind https://turntrout.com/why-i-left-google-deepmind

mofeien 3mo ago

That's an application of the "Tragedy of the commons": One person stopping to push the frontier just means that this person will lose their job, the frontier will be pushed by the others. The solution is to push for a binding rule instead that will make everyone slow down in the racing towards artificial superintelligence.

fidotron 3mo ago
And this is how we absolve people of all personal responsibility. Such companies would not pay people that well if they were so fungible. There is no such possible binding rule either, a point that is so obvious that any smart person claiming otherwise is clearly up to something.
- alach11 3mo ago
  We make binding rules all the time. We didn't fix the ozone layer by encouraging people to quit their jobs producing CFCs. We lobbied for international regulation to ban them.
  - fidotron 3mo ago
    In your example there is nothing to be gained by not complying. Failing to comply with what is in reality arms control is a widespread occurence. It's notable the only historical way to handle this (Schelling covered it at length) is tolerance of mutual espionage on a level that would simply not be tolerable to any of the groups involved. I should add one of Schelling's other pet subjects was "time to weapon being viable". i.e. the distinction between a nuclear power and non-nuclear power isn't quite right, as there's a huge class of countries that could become so fairly fast if they were so inclined, and that this situation is somewhat inevitable. It is this that leads to efforts to denuclearize, even on a civilian level, "problematic" countries. You want to stop frontier AI and you'll have to be rolling back a lot of existing deployed technology.
    - alach11 3mo ago
      Great comment. I haven't read Schelling but will check that out. I think you're right that international cooperation on the topic is likely impossible at present, for the reasons you stated. I'm not sure what it would take to shift the Overton window, but it's probably not pretty.
      - fidotron 3mo ago
        So honestly Schelling was utterly brilliant. I cannot recommend all three of the books wikipedia classed as his notable works highly enough: https://en.wikipedia.org/wiki/Thomas_Schelling#Notable_works https://en.wikipedia.org/wiki/Thomas_Schelling#Notable_works There are footnotes in "The Strategy of Conflict" containing more profound insight than whole books that are published today. On this particular topic though "Strategy and Arms Control" is the one to go for.
    - Nition 3mo ago
      The Montreal Protocol has not been perfectly adhered to either. China started emitting quite a lot of banned CFCs again from around 2012[1], which didn't get noticed until 2018! By 2020 it had finally stopped[2]. [1] https://www.nature.com/articles/s41586-018-0106-2 https://www.nature.com/articles/s41586-018-0106-2 [2] https://www.nature.com/articles/d41586-021-00360-0 https://www.nature.com/articles/d41586-021-00360-0
- polishTar 3mo ago
  Influencing change is more complicated than that. We live in a society that has systems of market competition between companies and economic/military competition between countries. There's not just one country working on frontier AI, and not just a few companies in each country either. The incentives of these systems ensure AI advancement will happen quickly from here on out, no matter what, absent governmental action. An individual researcher leaving the company does nothing, other than give their seat to someone else who likely doesn't share their same concerns. Alternatively, using the influence they do have in their position to directly try to advance AI safety research, advocate for govt action, and make public arguments attempting to persuade the public to pressure the govt to act when they otherwise wouldn't be inclined is far more likely to impact things for the better.
  - fidotron 3mo ago
    > Alternatively, using the influence they do have in their position to directly try to advance AI safety research, advocate for govt action, and make public arguments attempting to persuade the public to pressure the govt to act when they otherwise wouldn't be inclined is far more likely to impact things for the better. No, it just comes across as "I want to get paid as if I was doing the thing I am employed to do by threatening you all about how terrible it would be if I actually did it". They aren't even being semi imaginative and going on strike or anything.
- teaearlgraycold 3mo ago
  Personal responsibility is a difficult concept for be to apply to massive efforts like this. When there’s one person and you can talk to them personal responsibility is the number one topic. When you’re talking about a multi-trillion dollar bet and hundreds of thousands of people it doesn’t make sense anymore. Same logic goes for other common issues where personal responsibility is lauded. Obesity - real solutions are policy and medicine. Abortion - education and contraceptives.
- lifeformed 3mo ago
  Hoping everyone chooses to do the right thing is not an effective strategy for mitigating bad things.
HarHarVeryFunny 3mo ago
That may be a practical argument, but it's certainly not a moral one - it's like justifying selling drugs on the basis that if you don't do it somebody else will. If these frontier lab employees are actually sincere about wanting to do something about the risk of AI advancing faster than our ability to adapt to it, then what they should really be doing is lobbying their own employers to do something about it. Lead by example, and do something you have control over. Lobbying the government to stop yourself from doing something bad(!!) can't be taken as a serious position - it can only be taken as a plea to get the competition (China - open weights) shut down. It makes no sense to claim that China is only keeping up by copying you, yet China, not yourself, is the problem.

brap 3mo ago

“It is difficult to get a man to understand something, when his salary depends on his not understanding it”

deleted 3mo ago

[deleted]

abecedarius 3mo ago

Some of us considered seeking one of these jobs and decided not to. Some have quit (e.g. Hinton, OpenAI people after OpenAI reneged on safety commitments). Now you're getting this message from people who passed the previous filters. If you think a brake is indicated and not just easing up on the accelerator, you need collective action as the statement said.

fidotron 3mo ago
> you need collective action Come on, go on strike then! Have a week of protest and picket the offices in SFO. Let's see what happens. (I'm aware that's not quite what you meant, but it is the classic meaning of that phrase).
- anon373839 3mo ago
  Mass resignation perhaps?
  - 0xDEAFBEAD 3mo ago
    ...so the only remaining employees give no thought to safety? Unionization might be a little more effective.

0xDEAFBEAD 3mo ago

FWIW, here's an account from a guy who quit his job at Google Deepmind recently: https://turntrout.com/why-i-left-google-deepmind https://turntrout.com/why-i-left-google-deepmind