3 ms·

We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse

We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse other people with it. This whole idea that we're going to suddenly have HAL 9000 or Terminators running around is mob hype mentality. The responsible thing to do here is to educate. AGI isn't likely to destroy humanity. Humanity is already doing a good job of it. Climate change and poor political decisions and unchanging economic policy are more likely to finish us off. Concentration of capital through these AI companies is likely contributing to it.

NumberWangMan 4y ago
I'm a lot more scared of an AI destroying humanity, like, every single person, than I am of any government or anything like that. More so than climate change. I'm not saying the people using it aren't dangerous -- but I would choose a totalitarian regime over a rogue AI any day of the week. It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth into computational power in order to cure cancer, or figure out a way to stop humans from fighting each other, or to maximize the profit of GOOG, and being so good at planning and deceiving us that by the time we figured out what it was doing, it was way, way too late. I'm concerned about climate change, but I am a lot more hopeful about that than I am about AI. Climate change -- we have time, we are making changes, and it's not going to kill all of humanity. A smart enough AI might effectively end us the moment we switch it on.
- bombcar 4y ago
  Work out the details on exactly how said end-of-the-world would occur. Note that we already have “AI to end cities” - they’re just sitting, turned off, waiting for the code and button press in silos and submarines throughout the world.
  - mrob 4y ago
    The danger is from something vastly more intelligent than humans, and with a mindset that's incomprehensibly alien. No human is capable of working out the details. That doesn't mean the risk doesn't exist. Failure to understand chemistry does not make ants immune to insecticides. The only thing we can assume about a super-intelligence is that it will be highly capable of achieving its goals. There is no reason to assume those goals will be compatible with human existence.
    - bombcar 4y ago
      So humans will build God and so we all best get religion, and fast?
- agentultra 4y ago
  ChatGPT can do absolutely none of those things. Neither can any LLM. It’s not what they’re designed to do. There’s no need to worry about a speculative paper clip maximizer turning the world into grey goo. That’s still science fiction. The real harms today are much more banal.
  - dwaltrip 4y ago
    You can't think of any ways of taking advanced LLMs and using them as core components in a system that could carry out actions in the world? I bet you can come up with something.
    - agentultra 4y ago
      I can read maths and understand the papers. Extrapolating to something unreasonable is called speculation. Is it reasonable to believe that LLMs, even if they scale to hundreds of billions of tokens, could even emulate reasoning? No. It literally cannot.
      - dwaltrip 4y ago
        What do you think of the jump in capability between gpt-3.5 and gpt-4?
- pawelmurias 4y ago
  > It would be an AI deciding that it needs to turn every bit of available matter on earth into computational power in order to cure cancer, or figure out a way to stop humans from fighting each other, or to maximize the profit of GOOG, and being so good at planning and deceiving us that by the time we figured out what it was doing, it was way, way too late. That's how AIs worked in outdated science fictions. Current one don't have a bunch of mathematical rules that they follow to literally, but try to model what a human would write by statistical means with less logical capability.
kajaktum 4y ago
The danger is socioeconomic. AI will displace many, many jobs (it already does). Many people will claim that we will simply create different, new jobs. However, we have to think about what kind of job ja now viable for the average person? Typewriter used to be a decent job. Now, kids is probably expected to know how to do it. Can we keep up? Folks at HN seems to overestimate what the general population is capable of.
- agentultra 4y ago
  Should “jobs” be required for participation in society? Even Keynesian capitalists predicted we’d be working less by now with all the increases in productivity yet here we are with massive corporations continuing on union busting and all that. I agree there isn’t going to be a BS white collar job to replace the ones lost by advances like this.
mrob 4y ago
Anyone bringing up sci-fi is acting under a fallacy that doesn't have a formal name, but which I'd call "biomorphism", by analogy to anthropomorphism. The human mind is the product of billions of years of evolution, and as such, it's driven by assumptions such as self preservation, sex drive, status seeking, coherent sense of self, that are so fundamental that most of the time we don't even think about them. Sci-fi, even sci-fi written by authors like Peter Watts, who put serious effort into exploring the possible design space of minds, still follows most of these assumptions. A Terminator is ostensibly a machine, but it acts like a caricature of a man. There's only one genre that writes about truly alien minds (albeit with necessary vagueness), and that's cosmic horror. And unlike sci-fi, which often pretends humans could win, cosmic horror is under no delusion that summoning Elder Gods is ever a good idea.
stametseater 4y ago
> We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. Seems like a distinction without a difference to me. Whether or not the machine has some sort of "soul" is a question for philosophers or theologians, it has little bearing on the practical capabilities of the machine. > AGI isn't likely to destroy humanity. Can you give us some idea for the order of unlikely you are supposing? 1 in 10? 1 in 100? For your consideration: > During the next three months scientists in secret conference discussed the dangers of fusion but without agreement. Again Compton took the lead in the final decision. If, after calculation, he said, it were proved that the chances were more than approximately three in one million that the earth would be vaporized by the atomic explosion, he would not proceed with the project. Calculations proved the figures slightly less - and the project continued. http://large.stanford.edu/courses/2015/ph241/chung1/ http://large.stanford.edu/courses/2015/ph241/chung1/ Four in a million certainly "isn't likely" but Arthur Compton was apparently willing to halt the Manhattan Project if the likelihood of an atomic bomb triggering a fusion reaction in earth's atmosphere was merely that likely. Or to put it another way: If I load a revolver with a single bullet, spin the cylinder then point it at you, you are "not likely" to die with a 1 in 6 chance of the loaded chamber aligning with the barrel when I pull the trigger. Is Russian Roulette a game you'd like to play? Remember, it "isn't likely" that you're going to die.

NumberWangMan 4y ago

I'm a lot more scared of an AI destroying humanity, like, every single person, than I am of any government or anything like that. More so than climate change. I'm not saying the people using it aren't dangerous -- but I would choose a totalitarian regime over a rogue AI any day of the week. It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth into computational power in order to cure cancer, or figure out a way to stop humans from fighting each other, or to maximize the profit of GOOG, and being so good at planning and deceiving us that by the time we figured out what it was doing, it was way, way too late. I'm concerned about climate change, but I am a lot more hopeful about that than I am about AI. Climate change -- we have time, we are making changes, and it's not going to kill all of humanity. A smart enough AI might effectively end us the moment we switch it on.

bombcar 4y ago
Work out the details on exactly how said end-of-the-world would occur. Note that we already have “AI to end cities” - they’re just sitting, turned off, waiting for the code and button press in silos and submarines throughout the world.
- mrob 4y ago
  The danger is from something vastly more intelligent than humans, and with a mindset that's incomprehensibly alien. No human is capable of working out the details. That doesn't mean the risk doesn't exist. Failure to understand chemistry does not make ants immune to insecticides. The only thing we can assume about a super-intelligence is that it will be highly capable of achieving its goals. There is no reason to assume those goals will be compatible with human existence.
  - bombcar 4y ago
    So humans will build God and so we all best get religion, and fast?
agentultra 4y ago
ChatGPT can do absolutely none of those things. Neither can any LLM. It’s not what they’re designed to do. There’s no need to worry about a speculative paper clip maximizer turning the world into grey goo. That’s still science fiction. The real harms today are much more banal.
- dwaltrip 4y ago
  You can't think of any ways of taking advanced LLMs and using them as core components in a system that could carry out actions in the world? I bet you can come up with something.
  - agentultra 4y ago
    I can read maths and understand the papers. Extrapolating to something unreasonable is called speculation. Is it reasonable to believe that LLMs, even if they scale to hundreds of billions of tokens, could even emulate reasoning? No. It literally cannot.
    - dwaltrip 4y ago
      What do you think of the jump in capability between gpt-3.5 and gpt-4?
pawelmurias 4y ago
> It would be an AI deciding that it needs to turn every bit of available matter on earth into computational power in order to cure cancer, or figure out a way to stop humans from fighting each other, or to maximize the profit of GOOG, and being so good at planning and deceiving us that by the time we figured out what it was doing, it was way, way too late. That's how AIs worked in outdated science fictions. Current one don't have a bunch of mathematical rules that they follow to literally, but try to model what a human would write by statistical means with less logical capability.

kajaktum 4y ago

The danger is socioeconomic. AI will displace many, many jobs (it already does). Many people will claim that we will simply create different, new jobs. However, we have to think about what kind of job ja now viable for the average person? Typewriter used to be a decent job. Now, kids is probably expected to know how to do it. Can we keep up? Folks at HN seems to overestimate what the general population is capable of.

agentultra 4y ago
Should “jobs” be required for participation in society? Even Keynesian capitalists predicted we’d be working less by now with all the increases in productivity yet here we are with massive corporations continuing on union busting and all that. I agree there isn’t going to be a BS white collar job to replace the ones lost by advances like this.

mrob 4y ago

Anyone bringing up sci-fi is acting under a fallacy that doesn't have a formal name, but which I'd call "biomorphism", by analogy to anthropomorphism. The human mind is the product of billions of years of evolution, and as such, it's driven by assumptions such as self preservation, sex drive, status seeking, coherent sense of self, that are so fundamental that most of the time we don't even think about them. Sci-fi, even sci-fi written by authors like Peter Watts, who put serious effort into exploring the possible design space of minds, still follows most of these assumptions. A Terminator is ostensibly a machine, but it acts like a caricature of a man. There's only one genre that writes about truly alien minds (albeit with necessary vagueness), and that's cosmic horror. And unlike sci-fi, which often pretends humans could win, cosmic horror is under no delusion that summoning Elder Gods is ever a good idea.

stametseater 4y ago

> We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. Seems like a distinction without a difference to me. Whether or not the machine has some sort of "soul" is a question for philosophers or theologians, it has little bearing on the practical capabilities of the machine. > AGI isn't likely to destroy humanity. Can you give us some idea for the order of unlikely you are supposing? 1 in 10? 1 in 100? For your consideration: > During the next three months scientists in secret conference discussed the dangers of fusion but without agreement. Again Compton took the lead in the final decision. If, after calculation, he said, it were proved that the chances were more than approximately three in one million that the earth would be vaporized by the atomic explosion, he would not proceed with the project. Calculations proved the figures slightly less - and the project continued. http://large.stanford.edu/courses/2015/ph241/chung1/ http://large.stanford.edu/courses/2015/ph241/chung1/ Four in a million certainly "isn't likely" but Arthur Compton was apparently willing to halt the Manhattan Project if the likelihood of an atomic bomb triggering a fusion reaction in earth's atmosphere was merely that likely. Or to put it another way: If I load a revolver with a single bullet, spin the cylinder then point it at you, you are "not likely" to die with a 1 in 6 chance of the loaded chamber aligning with the barrel when I pull the trigger. Is Russian Roulette a game you'd like to play? Remember, it "isn't likely" that you're going to die.