4 ms·

All the worst outcomes involve taking this technology away from the public.

by Buttons840 13d ago

aorloff 13d ago
The very best outcomes involve us intentionally and collectively turning our attention away from this technology.
- amazingman 13d ago
  That is pure fantasy.
  - AngryData 13d ago
    Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person starve to death. And that is if people don't get pissed enough to start burning stuff down and instead try to be peaceful hippie homesteader types.
    - fc417fc802 13d ago
      But notably the amish don't preclude the rest of us from existing. Some subset of humans could turn their attention away from AI but it would presumably still exist and continue to be developed. I can't think of any economically beneficial technologies that we've collectively ignored. If you manage to come up with a counterexample then that's an opportunity to make some money for yourself. It's a fundamentally unstable state given our economic system.
      - Retric 13d ago
        Supersonic passenger aircraft were operated profitably and no longer exist. There’s levels of R&D required for many technologies where the question goes beyond could this be profitable to what are the risk vs reward that this specific project will succeed.
        fc417fc802 13d ago
        But after accounting for maintenance & etc were they more profitable than sinking the equivalent amount of money into something else, such as slower aircraft? Notably there are currently efforts to develop new supersonic passenger liners.
        ben_w 13d ago
        > Supersonic passenger aircraft were operated profitably Were they? IIRC joint project by British and French national airlines, they expected to sell loads, hardly anyone wanted to buy the planes, the two airlines kept them flying out of government embarassment.
        cyanydeez 13d ago
        should probably spell it out: _it wasn't allowed to get to supersonic speed_. Same way we dont _allow_ eugenics.
        fc417fc802 13d ago
        Sure, the regulation that guarded against externalizing certain costs might well have been the only thing precluding profitability. Importantly it wasn't some shared cultural value leading to voluntarily leaving money on the table. Rather the regulator imposed it. We are able to effectively regulate things like the operation of massive aircraft, eugenics, the dumping of toxic waste, or the refinement of nuclear material. But there are also plenty of things that we can't effectively regulate for purely practical reasons.
        ben_w 13d ago
        Over the Atlantic? Nah, the problem was it was very expensive and not a great experience for the people rich enough to afford it. The US company trying to bring supersonic back are not only focussed on the flight speed, but also the entire customer experience.
    - lacy_tinpot 13d ago
      You can't just stop AI without also stopping progress in computer hardware. You have to do both. The real implication of what they're saying extends beyond just AI. You have to stop the entire compute stack in order to prevent or slow down AI progress.
      - bedstefar 13d ago
        And even if we did manage to limit/cull hardware development, what would keep people from continuing to find ever more efficient models runnable on contemporary hardware?
        ben_w 13d ago
        Unclear how much more efficient they can be. Bleeding edge models are putting the desk-job competencies of multiple professional fields into something with as many parameters as a single large rodent has synapses.
    - kelnos 13d ago
      Because "we" embodies a hugely diverse set of beings with different wants, needs, goals, and priorities. Not to mention different morality and ethics. So sure, let's say you get 95% of the world to not use or work on LLMs. 5% is still enough to build something that brings forth the End Times. That's the old checklist trope of "Your idea won't work because: [x] it requires that everyone in the world agrees to do something, all at the same time." As the GP said: pure fantasy.
    - teiferer 13d ago
      > There are numerous technologies we have ignored or abandoned for numerous reasons. Are there? Nukes are a thing. Chemical weapons are a thing. Cluster bombs are a thing. Biological weapons are a thing. What is not a thing that shouldn't be? We say certain things should not be a thing, but then behind the scenes we made them a thing anyway.
      - ant6n 13d ago
        Nukes are barely used. The technology of nuclear detonations was practically abandoned: https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare
        teiferer 13d ago
        I find that a weird argument for saying humanity has "ignored or abandoned" nuclear weapons.
      - int_19h 13d ago
        On chemical weapons, most large state actors have got rid of them. Not because of any moral reasons, though, but simply because they don't actually work all that well in modern conventional warfare. This is then framed as an ethical issue, but you only need to look at where the same countries stand on e.g. landmines to realize how much bullshit it all is. Conversely, where you do still see chemical weapons used, it's usually asymmetric conflicts where "just gas the rebels" actually works much of the time and is much cheaper than other options. Big guys can afford the other options though, someone like Assad, not so much. More on this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/ https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
        teiferer 13d ago
        So, you are actually arguing against yourself and are agreeing with me?
    - hgoel 13d ago
      I'm baffled that people expressing your opinion don't understand what they're advocating for. Current generation smartphones are able to run AI models, people have managed to run very large models (even if slowly) via streaming from disk. The math for the underlying implementation is relatively trivial. Abandoning this technology means a cult-like extremist shift in culture against computers, or a global surveillance state of unprecedented scale and invasiveness. Not even getting into how much it requires us to give up on science, given the shared computational needs.
  - SV_BubbleTime 13d ago
    Watch, very soon this is going to politically divide. They’re sowing the seeds right now with “antiai” campaigns. The right will for a change be for it but will be able to be talked into regulation because you know “small government” and all only when convenient. The left will bitch and moan about fairness and copyright and automation and UBI, they’ll be the doomers and worldwar chicken littles. The Uniparty will be for it, but only against the public having anything good. What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic.
    - ben_w 13d ago
      The world isn't the American political nonsense. > What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic. Already happened, just from agentic coding.
- teiferer 13d ago
  > The very best outcomes involve us intentionally and collectively turning our attention away from this technology. To expand on the "pure fantasy" sister comment: There is just no way this will happen. It's in the spirit of "we can just stop all wars" and "we can just end world hunger". Technically it's very easy to do. Socially it's impossible to do. Unless you ignore realities.
  - calgoo 13d ago
    The danger lies with someone asking the machine to solve those two questions and it decides to cheat the solution. Launching every nuke in the world stops all wars, just like wiping out 99% of humanity ends world hunger.
    - carlob 13d ago
      I don't think killing people solves world hunger, we already produce enough food to feed everyone, it's just not evenly distributed.
      - ako 13d ago
        There can be more than one solution.
      - unrented7977 13d ago
        Can't be hungry if you're dead
    - teiferer 13d ago
      I don't see how that is more dangerous than two crazy head of states deciding that it's time for armageddon. The outcome is the same (99% of humans dead), it's equally easy to technically not do it (don't press the button, don't continue with LLMs), but also equally hard to regulate away given the real world. And that was my point.
  - ako 13d ago
    You forgot “we can stop climate change”…
- locknitpicker 13d ago
  > The very best outcomes involve us intentionally and collectively turning our attention away from this technology. Nowadays even small mom and pop restaurants use these models to generate their menus. Of course no one is going to stop using this technology.
dosisking 13d ago
Not necessarily, it just guarantees that China will win.
- ben_w 13d ago
  Unclear. How much is China learning from open models? Or the other way around, how much is the US losing worldwide mindshare by refusing to allow non-Americans access to bleeding-edge models and letting China fill in the gap? (Playing catchup with Mythos is still catching up, etc.)
  - bigyabai 13d ago
    > How much is China learning from open models? Across all their labs? Probably more than OpenAI and Anthropic learned from Astra/Mythos combined. The American frontier is being driven by an abundance of compute, which doesn't seem to scale efficiently. > how much is the US losing worldwide mindshare They're losing US mindshare. I pay $3/month for a Z.AI subscription and get billions of Opencode tokens. The $20/month price point is insane for the way that Claude and Codex treat their users, and that money doesn't go towards anything good like open-sourcing their models. It's a doomed product.
  - locknitpicker 13d ago
    > How much is China learning from open models? The last batch of open weight models releases by Chinese companies are on par with the performance of US-based frontier models, even though they are designed to run on pretty unimpressive hardware.
    - int_19h 13d ago
      This is very misleading. There is no Chinese model currently that is on par with Fable or Astra. Kimi K3 is pretty good, but it's not that good. Furthermore the Chinese models in that class are appropriately large. K3, for example, is a 2.8T-parameter model. Qwen 3.8 Max, another comparable model, is 2.4T params. Even with MoE, these are not "designed to run on pretty unimpressive hardware". Stuff like Qwen3.7-27B is, but it is also not even in the same ballpark as Opus, never mind frontier.
ben_w 13d ago
The current state of alignment is that we don't know how to make it only as bad as Pol Pot was made by his parents, teachers, genetics, circumstances, etc. This remains true regardless of if it is or isn't kept away from the public. "Helpful, harness, and honest": when used by a bastard, even just helpful and harmless are in direct conflict with each other. you may hate the government, but what about every radical group of extremists that wants to take over your government? Are none of them worse? None of this denies the problems with governments (if or not they take this tech for themselves and refuse it for others), just that it's a lack of imagination to say: > All the worst outcomes involve taking this technology away from the public
- necovek 13d ago
  I agree an argument can be made that not all the worst outcomes include restricting public access, but focusing on niche extremist groups (who might also get access to it through government programs by some of them being government employeed) is not a good argument to protect the public. Governments having unbalanced power can more easily lead to authoritarianism, and thus risk to the public.
  - watwut 13d ago
    At this point, I worry a lot more of billionaires Musk, Altman, Thiel etc then of goverments in general. Thrir project is to get all power and if they succeed, it will be way worst then what we have seen so far.
    - wartywhoa23 13d ago
      False dichotomy.
    - ben_w 13d ago
      There's plenty of things to worry about, and plenty of people to do the worrying.
- int_19h 13d ago
  Honestly? I'd rather al Qaeda have access to something like Fable, than the US government. The US government has capacity to do immensely more damage.
teekert 13d ago
So you’re saying the people should be aligned with the government?

All the worst outcomes involve taking this technology away from the public.

aorloff 13d ago

The very best outcomes involve us intentionally and collectively turning our attention away from this technology.

amazingman 13d ago
That is pure fantasy.
- AngryData 13d ago
  Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person starve to death. And that is if people don't get pissed enough to start burning stuff down and instead try to be peaceful hippie homesteader types.
  - fc417fc802 13d ago
    But notably the amish don't preclude the rest of us from existing. Some subset of humans could turn their attention away from AI but it would presumably still exist and continue to be developed. I can't think of any economically beneficial technologies that we've collectively ignored. If you manage to come up with a counterexample then that's an opportunity to make some money for yourself. It's a fundamentally unstable state given our economic system.
    - Retric 13d ago
      Supersonic passenger aircraft were operated profitably and no longer exist. There’s levels of R&D required for many technologies where the question goes beyond could this be profitable to what are the risk vs reward that this specific project will succeed.
      - fc417fc802 13d ago
        But after accounting for maintenance & etc were they more profitable than sinking the equivalent amount of money into something else, such as slower aircraft? Notably there are currently efforts to develop new supersonic passenger liners.
      - ben_w 13d ago
        > Supersonic passenger aircraft were operated profitably Were they? IIRC joint project by British and French national airlines, they expected to sell loads, hardly anyone wanted to buy the planes, the two airlines kept them flying out of government embarassment.
        cyanydeez 13d ago
        should probably spell it out: _it wasn't allowed to get to supersonic speed_. Same way we dont _allow_ eugenics.
        fc417fc802 13d ago
        Sure, the regulation that guarded against externalizing certain costs might well have been the only thing precluding profitability. Importantly it wasn't some shared cultural value leading to voluntarily leaving money on the table. Rather the regulator imposed it. We are able to effectively regulate things like the operation of massive aircraft, eugenics, the dumping of toxic waste, or the refinement of nuclear material. But there are also plenty of things that we can't effectively regulate for purely practical reasons.
        ben_w 13d ago
        Over the Atlantic? Nah, the problem was it was very expensive and not a great experience for the people rich enough to afford it. The US company trying to bring supersonic back are not only focussed on the flight speed, but also the entire customer experience.
  - lacy_tinpot 13d ago
    You can't just stop AI without also stopping progress in computer hardware. You have to do both. The real implication of what they're saying extends beyond just AI. You have to stop the entire compute stack in order to prevent or slow down AI progress.
    - bedstefar 13d ago
      And even if we did manage to limit/cull hardware development, what would keep people from continuing to find ever more efficient models runnable on contemporary hardware?
      - ben_w 13d ago
        Unclear how much more efficient they can be. Bleeding edge models are putting the desk-job competencies of multiple professional fields into something with as many parameters as a single large rodent has synapses.
  - kelnos 13d ago
    Because "we" embodies a hugely diverse set of beings with different wants, needs, goals, and priorities. Not to mention different morality and ethics. So sure, let's say you get 95% of the world to not use or work on LLMs. 5% is still enough to build something that brings forth the End Times. That's the old checklist trope of "Your idea won't work because: [x] it requires that everyone in the world agrees to do something, all at the same time." As the GP said: pure fantasy.
  - teiferer 13d ago
    > There are numerous technologies we have ignored or abandoned for numerous reasons. Are there? Nukes are a thing. Chemical weapons are a thing. Cluster bombs are a thing. Biological weapons are a thing. What is not a thing that shouldn't be? We say certain things should not be a thing, but then behind the scenes we made them a thing anyway.
    - ant6n 13d ago
      Nukes are barely used. The technology of nuclear detonations was practically abandoned: https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare
      - teiferer 13d ago
        I find that a weird argument for saying humanity has "ignored or abandoned" nuclear weapons.
    - int_19h 13d ago
      On chemical weapons, most large state actors have got rid of them. Not because of any moral reasons, though, but simply because they don't actually work all that well in modern conventional warfare. This is then framed as an ethical issue, but you only need to look at where the same countries stand on e.g. landmines to realize how much bullshit it all is. Conversely, where you do still see chemical weapons used, it's usually asymmetric conflicts where "just gas the rebels" actually works much of the time and is much cheaper than other options. Big guys can afford the other options though, someone like Assad, not so much. More on this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/ https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
      - teiferer 13d ago
        So, you are actually arguing against yourself and are agreeing with me?
  - hgoel 13d ago
    I'm baffled that people expressing your opinion don't understand what they're advocating for. Current generation smartphones are able to run AI models, people have managed to run very large models (even if slowly) via streaming from disk. The math for the underlying implementation is relatively trivial. Abandoning this technology means a cult-like extremist shift in culture against computers, or a global surveillance state of unprecedented scale and invasiveness. Not even getting into how much it requires us to give up on science, given the shared computational needs.
- SV_BubbleTime 13d ago
  Watch, very soon this is going to politically divide. They’re sowing the seeds right now with “antiai” campaigns. The right will for a change be for it but will be able to be talked into regulation because you know “small government” and all only when convenient. The left will bitch and moan about fairness and copyright and automation and UBI, they’ll be the doomers and worldwar chicken littles. The Uniparty will be for it, but only against the public having anything good. What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic.
  - ben_w 13d ago
    The world isn't the American political nonsense. > What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic. Already happened, just from agentic coding.
teiferer 13d ago
> The very best outcomes involve us intentionally and collectively turning our attention away from this technology. To expand on the "pure fantasy" sister comment: There is just no way this will happen. It's in the spirit of "we can just stop all wars" and "we can just end world hunger". Technically it's very easy to do. Socially it's impossible to do. Unless you ignore realities.
- calgoo 13d ago
  The danger lies with someone asking the machine to solve those two questions and it decides to cheat the solution. Launching every nuke in the world stops all wars, just like wiping out 99% of humanity ends world hunger.
  - carlob 13d ago
    I don't think killing people solves world hunger, we already produce enough food to feed everyone, it's just not evenly distributed.
    - ako 13d ago
      There can be more than one solution.
    - unrented7977 13d ago
      Can't be hungry if you're dead
  - teiferer 13d ago
    I don't see how that is more dangerous than two crazy head of states deciding that it's time for armageddon. The outcome is the same (99% of humans dead), it's equally easy to technically not do it (don't press the button, don't continue with LLMs), but also equally hard to regulate away given the real world. And that was my point.
- ako 13d ago
  You forgot “we can stop climate change”…
locknitpicker 13d ago
> The very best outcomes involve us intentionally and collectively turning our attention away from this technology. Nowadays even small mom and pop restaurants use these models to generate their menus. Of course no one is going to stop using this technology.

dosisking 13d ago

Not necessarily, it just guarantees that China will win.

ben_w 13d ago
Unclear. How much is China learning from open models? Or the other way around, how much is the US losing worldwide mindshare by refusing to allow non-Americans access to bleeding-edge models and letting China fill in the gap? (Playing catchup with Mythos is still catching up, etc.)
- bigyabai 13d ago
  > How much is China learning from open models? Across all their labs? Probably more than OpenAI and Anthropic learned from Astra/Mythos combined. The American frontier is being driven by an abundance of compute, which doesn't seem to scale efficiently. > how much is the US losing worldwide mindshare They're losing US mindshare. I pay $3/month for a Z.AI subscription and get billions of Opencode tokens. The $20/month price point is insane for the way that Claude and Codex treat their users, and that money doesn't go towards anything good like open-sourcing their models. It's a doomed product.
- locknitpicker 13d ago
  > How much is China learning from open models? The last batch of open weight models releases by Chinese companies are on par with the performance of US-based frontier models, even though they are designed to run on pretty unimpressive hardware.
  - int_19h 13d ago
    This is very misleading. There is no Chinese model currently that is on par with Fable or Astra. Kimi K3 is pretty good, but it's not that good. Furthermore the Chinese models in that class are appropriately large. K3, for example, is a 2.8T-parameter model. Qwen 3.8 Max, another comparable model, is 2.4T params. Even with MoE, these are not "designed to run on pretty unimpressive hardware". Stuff like Qwen3.7-27B is, but it is also not even in the same ballpark as Opus, never mind frontier.

ben_w 13d ago

The current state of alignment is that we don't know how to make it only as bad as Pol Pot was made by his parents, teachers, genetics, circumstances, etc. This remains true regardless of if it is or isn't kept away from the public. "Helpful, harness, and honest": when used by a bastard, even just helpful and harmless are in direct conflict with each other. you may hate the government, but what about every radical group of extremists that wants to take over your government? Are none of them worse? None of this denies the problems with governments (if or not they take this tech for themselves and refuse it for others), just that it's a lack of imagination to say: > All the worst outcomes involve taking this technology away from the public

necovek 13d ago
I agree an argument can be made that not all the worst outcomes include restricting public access, but focusing on niche extremist groups (who might also get access to it through government programs by some of them being government employeed) is not a good argument to protect the public. Governments having unbalanced power can more easily lead to authoritarianism, and thus risk to the public.
- watwut 13d ago
  At this point, I worry a lot more of billionaires Musk, Altman, Thiel etc then of goverments in general. Thrir project is to get all power and if they succeed, it will be way worst then what we have seen so far.
  - wartywhoa23 13d ago
    False dichotomy.
  - ben_w 13d ago
    There's plenty of things to worry about, and plenty of people to do the worrying.
int_19h 13d ago
Honestly? I'd rather al Qaeda have access to something like Fable, than the US government. The US government has capacity to do immensely more damage.

teekert 13d ago

So you’re saying the people should be aligned with the government?