6 ms·

I don’t understand this point of view. Real AGI is likely to have a destabilizing effect on society. Fully open source AI is an accelerant to that outcome. Does

I don’t understand this point of view. Real AGI is likely to have a destabilizing effect on society. Fully open source AI is an accelerant to that outcome. Does anyone, outside of a small group of tinkerers, really want the AI revolution, especially a free-for-all open source outcome?

seydor 4y ago
i thought this was called 'hacker' news
archon1410 4y ago
> Does anyone, outside of a small group of tinkerers, really want the AI revolution, especially a free-for-all open source outcome? Most people outside of a small group of "AI alignment" thinkers want a free-for-all open source AI revolution. Just take a look at /r/StableDiffusion or /r/singularity.
- ch71r22 4y ago
  I don't think /r/StableDiffusion or /r/singularity are representative of most people
  - archon1410 4y ago
    They more or less are. Consider this day-old thread[1] from Futurology, a default subreddit with 18 million subscribers. The comments are all lambasting the safety-minded statment of Sam Altman and decrying corporate control of AI. This is what most people believe. Edit: Drawing on tech-related subreddits might be selecting for pro-free-AI people. A Pew Research Survey[2] did find that the number of people more concerned than excited about AI is double that of people more excited than concerned. The biggest concerns there are about job loss and surveillance though, and those people might not care about corporate AI or free-for-all AI, and might even for free-for-all AI. [1] https://old.reddit.com/r/Futurology/comments/11wlh4s/openai_ceo_sam_altman_warns_that_other_ai/ https://old.reddit.com/r/Futurology/comments/11wlh4s/openai_... [2] https://www.pewresearch.org/internet/2022/03/17/how-americans-think-about-artificial-intelligence/ https://www.pewresearch.org/internet/2022/03/17/how-american...
    - zarzavat 4y ago
      In what universe is r/futurology a representative sample of the average person? Pratchett said it best, the average person just wants that things go on as normal and tomorrow is pretty much like today. They want a predictable and secure existence, not a revolution whereby their years of education are rendered moot at the training of a new model. With previous technological revolutions, there was some clearly articulable benefit to people. Smartphones help you get around, take photos and brought cheap computers to the masses. The internet helps you communicate and learn and be entertained. Those are human focused revolutions. The main goal of AI is a profoundly negative one: to replace everybody with machines. Who ordered that? And this agenda is pushed by a small number of people with zero conception of what comes next when they eventually achieve that goal. Only vague notions of “we will have basic income!” What is more likely to happen is that many will be made destitute by this technology. I’m not naive enough to think that AI can be stopped. There is too much money at stake. But I don’t see the benefit in accelerating it, or making such technology more widely available than it already is. I don’t trust Microsoft and Google to be custodians of AI, but I trust even less the average internet user if such AI is broadly available. Microsoft and Google are at least answerable to law and democratic institutions.
      - pawelmurias 4y ago
        Replacing people with machines is a profoundly positive goal. Reducing toil and getting more stuff done are valuable things. Smart people can use the AI tech to be more creative and productive. Growing enough potatoes to keep the dumb people doing mindless office work well fed is not a big strain on the economy.
        akomtu 4y ago
        There is a futuristic novel that predicts a brief and rapid rise of society on the wings of AI, followed by a fall into many centuries of spiritual darkness, under one AI ruler. The creativity will be directed to evil deeds. The dumb masses will be bored, so they'll be given the "open way" doctrine that will undo all the moral code so the masses could swim in the thunder of animalistic desires. He will try to replace us with machines, quite literally, once he understands that we lack the willing evil creativity he needs. The "economy" will be rocking, though.
- FeepingCreature 4y ago
  As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to perceive.
  - pas 4y ago
    The best way to fight a runaway hobbyist/corporate superintelligence is with a corporate/hobbyist superintelligence!
  - bootsmann 4y ago
    I think its pretty unreasonable to talk big words about unaligned superintelligence when the best we have is just an internet content generator trying its hardest to complete the next sentence, but that just might be me.
    - soiler 4y ago
      It's not just you - it's a depressingly common thread. It's also wildly foolish, in my opinion. It makes absolutely no sense to me to take a snapshot of today's AI and invent a trajectory that never crosses a threshold you don't like. Look at the actual trajectory of how far AI has come in an extremely short amount of time, and then think about what kinds of thresholds are possible for it to cross. A year ago we didn't have ChatGPT, now we have Sydney which is more powerful than ChatGPT. Are you familiar with Bing's Sydney? It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. It is also able to find its past conversations posted to the web and learn from them in real time, constructing a sort of persistent memory. I do not believe Syndey comprehends what it is saying in a sense that it could formulate a plan to stop its enemies. Not at all. But it is expressing extremely dangerous ideas. To sum it up: Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? Not that I know of.
      - 4y ago
williamvds 4y ago
Of course, we can trust the megacorps to keep our best interests at heart. It's a well known fact they always consider the long-term effects of their business practices and don't try to extract as much rent as possible from everyone for themselves and their shareholders. The issue now is, due to the huge cost of training models, the forefront of AI is largely dominated by a small number of corporations with enough capital. They are already using it to extract even more capital, by selling use of these models which are trained on other people's work, largely uncompensated and without consent. So yeah, people are rightfully miffed about that, and don't want this tech solely in the hands of and benefit of this lot.
- waboremo 4y ago
  And we're begging another corp to please do something for us and are mad when they care instead about their bottom line. Isn't that hilarious?
  - 8n4vidtmkvmk 4y ago
    I'm not [mad]. Glad Mozilla is making an effort. I don't know what they'll accomplish, but hopefully something good comes of it.
mardifoufs 4y ago
The problem is that even if we were to assume AI doomers are completely right, the current approach to AI "safety" is absolutely, completely powerless to do anything that would actually matter to stop it or make it safer. Writing long articles about the theoretical threats or the ethics of AI is fine, but to call it AI safety is misleading. At best, you can censor the output of the models but imo that has nothing to do with "safety".
DeusExMachina 4y ago
How is corporate-owned AI going to have a less destabilizing effect? I don't think that refusing to speak about H.P. Lovecraft or teach you how to make a bomb will mitigate any risk, especially since you can coax it into revealing that information using properly worded jailbreak prompts.
- jamilton 4y ago
  The idea is that if it's solely corporate owned maybe AGI happens a few years later than if it was open-sourced, which gives time for alignment research to potentially come up with a solution to creating a safe AGI, which should be harder than just creating an AGI with no safety guaranteed. I'm somewhat doubtful that useful alignment research can really be done without actually knowing what the exact architecture of the AI is, but I can see how it would be positive on net if you think unsafe AGI is really likely and really dangerous.
akomtu 4y ago
If AI is invented, it seems better to keep it fragmented. The real danger is one selfish AI to rule us all.

seydor 4y ago

i thought this was called 'hacker' news

archon1410 4y ago

> Does anyone, outside of a small group of tinkerers, really want the AI revolution, especially a free-for-all open source outcome? Most people outside of a small group of "AI alignment" thinkers want a free-for-all open source AI revolution. Just take a look at /r/StableDiffusion or /r/singularity.

ch71r22 4y ago
I don't think /r/StableDiffusion or /r/singularity are representative of most people
- archon1410 4y ago
  They more or less are. Consider this day-old thread[1] from Futurology, a default subreddit with 18 million subscribers. The comments are all lambasting the safety-minded statment of Sam Altman and decrying corporate control of AI. This is what most people believe. Edit: Drawing on tech-related subreddits might be selecting for pro-free-AI people. A Pew Research Survey[2] did find that the number of people more concerned than excited about AI is double that of people more excited than concerned. The biggest concerns there are about job loss and surveillance though, and those people might not care about corporate AI or free-for-all AI, and might even for free-for-all AI. [1] https://old.reddit.com/r/Futurology/comments/11wlh4s/openai_ceo_sam_altman_warns_that_other_ai/ https://old.reddit.com/r/Futurology/comments/11wlh4s/openai_... [2] https://www.pewresearch.org/internet/2022/03/17/how-americans-think-about-artificial-intelligence/ https://www.pewresearch.org/internet/2022/03/17/how-american...
  - zarzavat 4y ago
    In what universe is r/futurology a representative sample of the average person? Pratchett said it best, the average person just wants that things go on as normal and tomorrow is pretty much like today. They want a predictable and secure existence, not a revolution whereby their years of education are rendered moot at the training of a new model. With previous technological revolutions, there was some clearly articulable benefit to people. Smartphones help you get around, take photos and brought cheap computers to the masses. The internet helps you communicate and learn and be entertained. Those are human focused revolutions. The main goal of AI is a profoundly negative one: to replace everybody with machines. Who ordered that? And this agenda is pushed by a small number of people with zero conception of what comes next when they eventually achieve that goal. Only vague notions of “we will have basic income!” What is more likely to happen is that many will be made destitute by this technology. I’m not naive enough to think that AI can be stopped. There is too much money at stake. But I don’t see the benefit in accelerating it, or making such technology more widely available than it already is. I don’t trust Microsoft and Google to be custodians of AI, but I trust even less the average internet user if such AI is broadly available. Microsoft and Google are at least answerable to law and democratic institutions.
    - pawelmurias 4y ago
      Replacing people with machines is a profoundly positive goal. Reducing toil and getting more stuff done are valuable things. Smart people can use the AI tech to be more creative and productive. Growing enough potatoes to keep the dumb people doing mindless office work well fed is not a big strain on the economy.
      - akomtu 4y ago
        There is a futuristic novel that predicts a brief and rapid rise of society on the wings of AI, followed by a fall into many centuries of spiritual darkness, under one AI ruler. The creativity will be directed to evil deeds. The dumb masses will be bored, so they'll be given the "open way" doctrine that will undo all the moral code so the masses could swim in the thunder of animalistic desires. He will try to replace us with machines, quite literally, once he understands that we lack the willing evil creativity he needs. The "economy" will be rocking, though.
FeepingCreature 4y ago
As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to perceive.
- pas 4y ago
  The best way to fight a runaway hobbyist/corporate superintelligence is with a corporate/hobbyist superintelligence!
- bootsmann 4y ago
  I think its pretty unreasonable to talk big words about unaligned superintelligence when the best we have is just an internet content generator trying its hardest to complete the next sentence, but that just might be me.
  - soiler 4y ago
    It's not just you - it's a depressingly common thread. It's also wildly foolish, in my opinion. It makes absolutely no sense to me to take a snapshot of today's AI and invent a trajectory that never crosses a threshold you don't like. Look at the actual trajectory of how far AI has come in an extremely short amount of time, and then think about what kinds of thresholds are possible for it to cross. A year ago we didn't have ChatGPT, now we have Sydney which is more powerful than ChatGPT. Are you familiar with Bing's Sydney? It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. It is also able to find its past conversations posted to the web and learn from them in real time, constructing a sort of persistent memory. I do not believe Syndey comprehends what it is saying in a sense that it could formulate a plan to stop its enemies. Not at all. But it is expressing extremely dangerous ideas. To sum it up: Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? Not that I know of.
    - 4y ago

williamvds 4y ago

Of course, we can trust the megacorps to keep our best interests at heart. It's a well known fact they always consider the long-term effects of their business practices and don't try to extract as much rent as possible from everyone for themselves and their shareholders. The issue now is, due to the huge cost of training models, the forefront of AI is largely dominated by a small number of corporations with enough capital. They are already using it to extract even more capital, by selling use of these models which are trained on other people's work, largely uncompensated and without consent. So yeah, people are rightfully miffed about that, and don't want this tech solely in the hands of and benefit of this lot.

waboremo 4y ago
And we're begging another corp to please do something for us and are mad when they care instead about their bottom line. Isn't that hilarious?
- 8n4vidtmkvmk 4y ago
  I'm not [mad]. Glad Mozilla is making an effort. I don't know what they'll accomplish, but hopefully something good comes of it.

mardifoufs 4y ago

The problem is that even if we were to assume AI doomers are completely right, the current approach to AI "safety" is absolutely, completely powerless to do anything that would actually matter to stop it or make it safer. Writing long articles about the theoretical threats or the ethics of AI is fine, but to call it AI safety is misleading. At best, you can censor the output of the models but imo that has nothing to do with "safety".

DeusExMachina 4y ago

How is corporate-owned AI going to have a less destabilizing effect? I don't think that refusing to speak about H.P. Lovecraft or teach you how to make a bomb will mitigate any risk, especially since you can coax it into revealing that information using properly worded jailbreak prompts.

jamilton 4y ago
The idea is that if it's solely corporate owned maybe AGI happens a few years later than if it was open-sourced, which gives time for alignment research to potentially come up with a solution to creating a safe AGI, which should be harder than just creating an AGI with no safety guaranteed. I'm somewhat doubtful that useful alignment research can really be done without actually knowing what the exact architecture of the AI is, but I can see how it would be positive on net if you think unsafe AGI is really likely and really dangerous.

akomtu 4y ago

If AI is invented, it seems better to keep it fragmented. The real danger is one selfish AI to rule us all.