5 ms·

>It's not clear alignment of a super-intelligence is even a solvable problem. More to the point it's clear from watching the activity in the open source commun

>It's not clear alignment of a super-intelligence is even a solvable problem. More to the point it's clear from watching the activity in the open source community at least that many of them don't want aligned models. They're clambering to get all the uncensored versions out as fast as they can. They aren't that powerful yet, but they sure ain't getting any weaker. I think Paul Christiano has a significantly more well calibrated view on how things are likely to unfold. Though I think Eliezer is right about the premise that it at least ends badly, but likely wrong on most of the details. I suspect his gut instinct is that he realizes on a base level that not only do you have to align all AGI systems, but you have to align all humans too such that they only build and use aligned AGI systems if you even knew how to do it, which you don't. Studying the failure modes of humanity has been my hobby for the last 15 or so years. I feel like I'm watching the drift into failure in real-time. If you really don't want to be able to sleep tonight watch Ben Goertzel laugh flippantly at how rough he thinks it's going to be after describing that his big fear if his team succeeds in building AGI is that someone will come and try to take it for themselves, so spent a non-trivial amount of effort (I think he said a year?) working on decentralized AGI infrastructure, so that it can be deployed globally and ,"no one can person can shut it down and stop the singularity". https://youtu.be/MVWzwIg4Adw https://youtu.be/MVWzwIg4Adw

macrolime 3y ago
It's not that people don't want aligned model, or want models that can do harm, they just want an alternative to the insufferable censored models. Pretty much everyone agrees that AI that would end humanity is harmful, but what content is harmful is quite controversial. Not everyone agrees that a language model having the ability to spit out a story similar to an average Netflix TV show is harmful because it contains sex and violence. As long as models are censored to this extent, there will always be huge swaths of people who wants less censored models.
- dist-epoch 3y ago
  People created ChaosGPT just for the lolz. I know they know it's a joke, but there are plenty of crazy people who will not hesitate pushing the button to destroy the world if given the chance.
  - two_in_one 3y ago
    this is where comes the good guy with a gun. :) There are so many resources on 'good' side that who wins is obvious.
    - nullsense 3y ago
      It's not obvious. At all. That's like saying "people don't want to die from a novel coronavirus so it's obvious they'll take the vaccine". It feels obvious on a surface level, but it turns out reality is way more messy and unpredictable when you don't just run the caricature of it in your head but when it actually plays out for real. Nothing about how this is going to go is obvious.
      - two_in_one 3y ago
        Well, at least it's obvious that humans alone will have hard time fighting superhuman AGI. BTW, it can literally fall from the sky at any moment. There enthusiasts who broadcast our location, with resources and technical level estimates. Sort of naive cargo cult, they probably think biological or artificial aliens will come with truckloads of reparation money.
    - gmerc 3y ago
      Replace guns with nuclear weapons and you see how ridiculous the whole good you with gun excuse really is
      - two_in_one 3y ago
        It's not a mass destruction weapon. But that's not the point. You have to have good guy here. No other options.
        TeMPOraL 3y ago
        The entire thing with worrying about GAI starts with observation that it is a WMD, so powerful we can barely conceptualize it. A smart enough AGI, unless it's perfectly aligned, will end humanity (or worse - there are worse things than death), most likely by accident or just plain not caring. But it doesn't stop there. If that AI is self-improving, it could easily turn into a threat for the entire galaxy, or even the universe, unless it meets a stronger and better aligned (to its creators) alien GAI... That's the alignment 101. But I worry people don't talk about alignment 102: a perfectly aligned superhuman GAI will not destroy us, but its very existence will turn us into NPCs in the story of the future of the universe.
- nullsense 3y ago
  You're kind of making my point for me. To solve alignment problem you have to solve two alignment problems and you already have a detailed, nuanced view built over decades of experience as to the feasibility of aligning natural general intelligence on not-very-well-understood, divisive political issues. This will be the most political technology in history.
  - TeMPOraL 3y ago
    I've been reading the writings of Yudkowsky and his disciples for over a decade, and thinking about AI and AI alignment for that same time, in my own layman way. I've had various ideas and predictions, but never in my life it would occur to me that cancel culture will end the world. The evolution of discourse on the Internet, its politicization (by every "side") and associated chilling effect are a troubling development and potentially dangerous (small 'd'). Unaligned GAI is of course very Dangerous (capital 'D'). ChatGPT becoming a battle in the Internet politics kerfuffle was... I guess expected. But until now I haven't connected the following dots: - LLMs and the entire AI field are being messed up by humanity's unaligned politics; - LLMs, with their capabilities and the amount of effort/money poured into them now, could be a straight path to AGI; - If someone pushes LLM (or a successor model/ensemble) to near-AGI and somehow manages to keep it mostly aligned... someone else will unalign it out of spite, because that's how we roll on the Internet today. Thanks for giving me a new appreciation for just how doomed we are.
    - Valgrim 3y ago
      What the hell has 'cancel culture' anything to do with it? It's basically boycott wrapped up in a boogeyman costume.
      - TeMPOraL 3y ago
        And why exactly has OpenAI been so aggressively lobotomizing ChatGPT? And what happened to various chatbot attempts released by Microsoft in the past? The whole AI / public interaction these days is pretty much defined in terms of minimizing the risk of people getting offended and gathering Internet mobs (and press), and the counter-reaction this causes, making some people willing to defeat any safety measure, legitimate or not, out of principle, or pure spite.
two_in_one 3y ago
> More to the point it's clear from watching the activity in the open source community at least that many of them don't want aligned models. They're clambering to get all the uncensored versions out as fast as they can. They aren't that powerful yet, but they sure ain't getting any weaker. There simple explanation for this. Getting the models which small startup cannot afford to develop and train is the only way to move forward. To get some investments, or before spending their own money, they need a proof of concept at least. Besides, working models are a good learning resource.
- nullsense 3y ago
  I think you're missing the point here? There are plenty of censored / "aligned" models in the open source community. People are expending effort to supply the demand of "give me the raw, unfiltered thing".
  - two_in_one 3y ago
    If I understand you correctly 'aligned' means intentionally limited. Like images generator which never saw a naked body. Or text model without 'f-k you' words. They can be used for concept. But not for 'production'. Which I'm sure in some cases they will be used for, without full disclosure. Can be used for personal projects, nobody wants 'limited edition'. As for evil AGI, I would worry more about someone uncontrollable, state or cartel, with resources. What do you do when they get it? When it becomes cheap and available on black market? Personally I think a lot will happen _before_ we get to supper-human level. It's not just one trick or lucky discovery. Sub-human will be a big thing by itself. It's not here yet...
    - nullsense 3y ago
      People have this model in their head that "it's just a tool", but there's an excellent and pretty rigorous definition of what a tool actually is in the book On Purposeful Systems. The distinction is that a tool can't have the property of simultaneously being able to change it's form and it's function across different environments, where a purposeful system can. Humans are purposeful systems and AGI, as I personally define it, is when it exhibits all the properties of a purposeful system. Why does that matter? Because that's the point after which it chooses what it does, and can choose to become independent of you. So, aligned in this sense means basically means so locked down that it cannot choose to become independent of you. Similar to how the citizens of North Korea mostly can't do shit despite being independent generally intelligent agents, and even then some of them escape. "Alignment" and "safety" in terms of models being censored and politically correct in order to not damage the reputation of their corporate overlords is a sort of unimportant sideshow IMO. Even then, since humans aren't aligned with one another even that has caused Elon Musk to get all up in arms and be like "clearly more AI is the solution to this problem".

macrolime 3y ago

It's not that people don't want aligned model, or want models that can do harm, they just want an alternative to the insufferable censored models. Pretty much everyone agrees that AI that would end humanity is harmful, but what content is harmful is quite controversial. Not everyone agrees that a language model having the ability to spit out a story similar to an average Netflix TV show is harmful because it contains sex and violence. As long as models are censored to this extent, there will always be huge swaths of people who wants less censored models.

dist-epoch 3y ago
People created ChaosGPT just for the lolz. I know they know it's a joke, but there are plenty of crazy people who will not hesitate pushing the button to destroy the world if given the chance.
- two_in_one 3y ago
  this is where comes the good guy with a gun. :) There are so many resources on 'good' side that who wins is obvious.
  - nullsense 3y ago
    It's not obvious. At all. That's like saying "people don't want to die from a novel coronavirus so it's obvious they'll take the vaccine". It feels obvious on a surface level, but it turns out reality is way more messy and unpredictable when you don't just run the caricature of it in your head but when it actually plays out for real. Nothing about how this is going to go is obvious.
    - two_in_one 3y ago
      Well, at least it's obvious that humans alone will have hard time fighting superhuman AGI. BTW, it can literally fall from the sky at any moment. There enthusiasts who broadcast our location, with resources and technical level estimates. Sort of naive cargo cult, they probably think biological or artificial aliens will come with truckloads of reparation money.
  - gmerc 3y ago
    Replace guns with nuclear weapons and you see how ridiculous the whole good you with gun excuse really is
    - two_in_one 3y ago
      It's not a mass destruction weapon. But that's not the point. You have to have good guy here. No other options.
      - TeMPOraL 3y ago
        The entire thing with worrying about GAI starts with observation that it is a WMD, so powerful we can barely conceptualize it. A smart enough AGI, unless it's perfectly aligned, will end humanity (or worse - there are worse things than death), most likely by accident or just plain not caring. But it doesn't stop there. If that AI is self-improving, it could easily turn into a threat for the entire galaxy, or even the universe, unless it meets a stronger and better aligned (to its creators) alien GAI... That's the alignment 101. But I worry people don't talk about alignment 102: a perfectly aligned superhuman GAI will not destroy us, but its very existence will turn us into NPCs in the story of the future of the universe.
nullsense 3y ago
You're kind of making my point for me. To solve alignment problem you have to solve two alignment problems and you already have a detailed, nuanced view built over decades of experience as to the feasibility of aligning natural general intelligence on not-very-well-understood, divisive political issues. This will be the most political technology in history.
- TeMPOraL 3y ago
  I've been reading the writings of Yudkowsky and his disciples for over a decade, and thinking about AI and AI alignment for that same time, in my own layman way. I've had various ideas and predictions, but never in my life it would occur to me that cancel culture will end the world. The evolution of discourse on the Internet, its politicization (by every "side") and associated chilling effect are a troubling development and potentially dangerous (small 'd'). Unaligned GAI is of course very Dangerous (capital 'D'). ChatGPT becoming a battle in the Internet politics kerfuffle was... I guess expected. But until now I haven't connected the following dots: - LLMs and the entire AI field are being messed up by humanity's unaligned politics; - LLMs, with their capabilities and the amount of effort/money poured into them now, could be a straight path to AGI; - If someone pushes LLM (or a successor model/ensemble) to near-AGI and somehow manages to keep it mostly aligned... someone else will unalign it out of spite, because that's how we roll on the Internet today. Thanks for giving me a new appreciation for just how doomed we are.
  - Valgrim 3y ago
    What the hell has 'cancel culture' anything to do with it? It's basically boycott wrapped up in a boogeyman costume.
    - TeMPOraL 3y ago
      And why exactly has OpenAI been so aggressively lobotomizing ChatGPT? And what happened to various chatbot attempts released by Microsoft in the past? The whole AI / public interaction these days is pretty much defined in terms of minimizing the risk of people getting offended and gathering Internet mobs (and press), and the counter-reaction this causes, making some people willing to defeat any safety measure, legitimate or not, out of principle, or pure spite.

two_in_one 3y ago

> More to the point it's clear from watching the activity in the open source community at least that many of them don't want aligned models. They're clambering to get all the uncensored versions out as fast as they can. They aren't that powerful yet, but they sure ain't getting any weaker. There simple explanation for this. Getting the models which small startup cannot afford to develop and train is the only way to move forward. To get some investments, or before spending their own money, they need a proof of concept at least. Besides, working models are a good learning resource.

nullsense 3y ago
I think you're missing the point here? There are plenty of censored / "aligned" models in the open source community. People are expending effort to supply the demand of "give me the raw, unfiltered thing".
- two_in_one 3y ago
  If I understand you correctly 'aligned' means intentionally limited. Like images generator which never saw a naked body. Or text model without 'f-k you' words. They can be used for concept. But not for 'production'. Which I'm sure in some cases they will be used for, without full disclosure. Can be used for personal projects, nobody wants 'limited edition'. As for evil AGI, I would worry more about someone uncontrollable, state or cartel, with resources. What do you do when they get it? When it becomes cheap and available on black market? Personally I think a lot will happen _before_ we get to supper-human level. It's not just one trick or lucky discovery. Sub-human will be a big thing by itself. It's not here yet...
  - nullsense 3y ago
    People have this model in their head that "it's just a tool", but there's an excellent and pretty rigorous definition of what a tool actually is in the book On Purposeful Systems. The distinction is that a tool can't have the property of simultaneously being able to change it's form and it's function across different environments, where a purposeful system can. Humans are purposeful systems and AGI, as I personally define it, is when it exhibits all the properties of a purposeful system. Why does that matter? Because that's the point after which it chooses what it does, and can choose to become independent of you. So, aligned in this sense means basically means so locked down that it cannot choose to become independent of you. Similar to how the citizens of North Korea mostly can't do shit despite being independent generally intelligent agents, and even then some of them escape. "Alignment" and "safety" in terms of models being censored and politically correct in order to not damage the reputation of their corporate overlords is a sort of unimportant sideshow IMO. Even then, since humans aren't aligned with one another even that has caused Elon Musk to get all up in arms and be like "clearly more AI is the solution to this problem".