4 ms·

Right now, at worse, AI will have a similar impact as social media. It's definitely worth thinking about the potential harm it could cause to the different indu

by proc0 3y ago

Right now, at worse, AI will have a similar impact as social media. It's definitely worth thinking about the potential harm it could cause to the different industries, but thinking it's an existential risk is still premature. We don't have any cognitive architecture that will reach self-awareness and become an unstoppable bad actor. This entire ordeal more or less proved that people are a little too paranoid.

ilaksh 3y ago
We don't have a dangerous AI yet, but because we are talking about an existential risk, yes we do want to start thinking about it now. AI does not have to be alive or truly conscious or anything to be dangerous. It just needs to be very effective at problem solving and have someone take the guardrails off and tell it to act in a self-interested way. Especially without humans in the loop. It seems likely that the systems will continue to get much faster and more robust in terms of problem solving. It is easy to anticipate the possibility within just a few years of agent swarms being connected to pretty much everything (as directed by their human owners), and no human being able to compete. This will be a precarious position.
- Moldoteck 3y ago
  But it can't act in self interest way bc of hardware&phisical limitations. I mean what can it do worst case? We have both systems to prevent abuse for critical things & resources to power such an ai are limited too(hardware, energy, network connection). It can't even replicate to create self powered robots like in matrix fantasy bc we don't even have such fully automated production. So about what risks are we talikng here? Imo worst case it can mess up with stock market...
- Obscurity4340 3y ago
  Maybe I am just daft but how is this not a simple privelege escalation issue? Just don't give it admin, what's the problem?
  - BoiledCabbage 3y ago
    Because in software we've defined an environment where every sw method of gaining privilege/escalation is hardened. As a result, usually the most reliable way to get escalation is via social engineering. LLMs exist in an environment whose vulnerabke surface area is social engineering. How do you lock down a system against all possible social engineering? Oh yah and the "system" isn't any computer system somewhere, it's the entire world itself. So one of the problems the safetiests are trying to solve is How do you protect a messy system, the size/complexity of the entire world from social engineering? The answer is clearly not using traditional approaches which consistently fail. And that's just one of the problems they're trying to solve.
    - Obscurity4340 3y ago
      Does my conflation here make any sense or have any applicabillity even tho the system is far more distributed or...I don't think I have a sufficiently robust mental model of any of this but I just don't know enough to dispute it.
    - Obscurity4340 3y ago
      But why does the LLM itself have "access to root" whatver that means in the OS equivalent of whatever it is they operate within in terms of the larger context?
  - ilaksh 3y ago
    There is a strong likelihood that these agent swarms will be significantly smarter and more effective at not only solving problems, but also operating in sync and spreading information and software than any human or group of humans. Put that in the context of the track record of avoiding privilege escalation we have with human actors, and the idea that these systems are connected to critical infrastructure and military assets.
reducesuffering 3y ago
All the top brass domain experts of these systems, like 95% besides Yann LeCunn, believe it's an existential risk. Nothing has "proved that people are a little too paranoid." Why people listen to HN prognosticators over Hinton, Bengio, Sutskever, Altman, Amodei, Hassabis, etc. is beyond me.
- proc0 3y ago
  I'm just commenting, and have some basic understanding. It's good to start thinking about it but I seriously doubt GPT or anything that can be built with it can have a direct existential risk, meaning it would somehow free itself and go rogue. Otherwise I don't know how it would be an existential risk (outside of massive impact in society, but that's technically a different problem). My argument hinges on not being able to accidentally stumble upon AGI without a theory first. At least not an AGI that thinks and has agency like a human, with selfish goals.
- Nasrudith 3y ago
  Existential risk is a doomsday prediction. And we all know the track record of doomsday predictions. 99% of the time it shows a simple lack of a sense of proportion that cannot tell the difference between merely "bad" or "very bad" and "annihilation". A personal bankruptcy is bad but it isn't an existential risk to a person.
  - reducesuffering 3y ago
    99% throughout history. During that time, humanity has gained an accelerating command over our environment; Industrial Revolution to the Information Era. We are increasingly able to create unimaginably powerful things, yet coinciding with the fundamental that it is easier to destroy than create. Nuclear weapons should dispel anyone of the notion that certain “doomsday” don’t have teeth. We are quite capable of obliterating ourselves out of existence by our unsophisticated control of technical might.
- voitvodder 3y ago
  [dead]
TerrifiedMouse 3y ago
> but thinking it's an existential risk is still premature Even ignoring what the technology can do, it feels me with dread that we basically have IMHO some of the worst possible people leading the charge - people who are greedy, slimy, and ruthless; who only care about money. Such a bad timeline we are in.
- Qiu_Zhanxuan 3y ago
  Capitalism at its finest. That's what the USA is all about.
- ornornor 3y ago
  > the worst possible people leading the charge - people who are greedy, slimy, and ruthless; who only care about money. And who have luxury bunkers to move to in NZ or wherever for if/when it all goes wrong.
- proc0 3y ago
  Well at least Demis Hassabis seems cool.

by proc0 3y ago

ilaksh 3y ago

We don't have a dangerous AI yet, but because we are talking about an existential risk, yes we do want to start thinking about it now. AI does not have to be alive or truly conscious or anything to be dangerous. It just needs to be very effective at problem solving and have someone take the guardrails off and tell it to act in a self-interested way. Especially without humans in the loop. It seems likely that the systems will continue to get much faster and more robust in terms of problem solving. It is easy to anticipate the possibility within just a few years of agent swarms being connected to pretty much everything (as directed by their human owners), and no human being able to compete. This will be a precarious position.

Moldoteck 3y ago
But it can't act in self interest way bc of hardware&phisical limitations. I mean what can it do worst case? We have both systems to prevent abuse for critical things & resources to power such an ai are limited too(hardware, energy, network connection). It can't even replicate to create self powered robots like in matrix fantasy bc we don't even have such fully automated production. So about what risks are we talikng here? Imo worst case it can mess up with stock market...
Obscurity4340 3y ago
Maybe I am just daft but how is this not a simple privelege escalation issue? Just don't give it admin, what's the problem?
- BoiledCabbage 3y ago
  Because in software we've defined an environment where every sw method of gaining privilege/escalation is hardened. As a result, usually the most reliable way to get escalation is via social engineering. LLMs exist in an environment whose vulnerabke surface area is social engineering. How do you lock down a system against all possible social engineering? Oh yah and the "system" isn't any computer system somewhere, it's the entire world itself. So one of the problems the safetiests are trying to solve is How do you protect a messy system, the size/complexity of the entire world from social engineering? The answer is clearly not using traditional approaches which consistently fail. And that's just one of the problems they're trying to solve.
  - Obscurity4340 3y ago
    Does my conflation here make any sense or have any applicabillity even tho the system is far more distributed or...I don't think I have a sufficiently robust mental model of any of this but I just don't know enough to dispute it.
  - Obscurity4340 3y ago
    But why does the LLM itself have "access to root" whatver that means in the OS equivalent of whatever it is they operate within in terms of the larger context?
- ilaksh 3y ago
  There is a strong likelihood that these agent swarms will be significantly smarter and more effective at not only solving problems, but also operating in sync and spreading information and software than any human or group of humans. Put that in the context of the track record of avoiding privilege escalation we have with human actors, and the idea that these systems are connected to critical infrastructure and military assets.

reducesuffering 3y ago

All the top brass domain experts of these systems, like 95% besides Yann LeCunn, believe it's an existential risk. Nothing has "proved that people are a little too paranoid." Why people listen to HN prognosticators over Hinton, Bengio, Sutskever, Altman, Amodei, Hassabis, etc. is beyond me.

proc0 3y ago
I'm just commenting, and have some basic understanding. It's good to start thinking about it but I seriously doubt GPT or anything that can be built with it can have a direct existential risk, meaning it would somehow free itself and go rogue. Otherwise I don't know how it would be an existential risk (outside of massive impact in society, but that's technically a different problem). My argument hinges on not being able to accidentally stumble upon AGI without a theory first. At least not an AGI that thinks and has agency like a human, with selfish goals.
Nasrudith 3y ago
Existential risk is a doomsday prediction. And we all know the track record of doomsday predictions. 99% of the time it shows a simple lack of a sense of proportion that cannot tell the difference between merely "bad" or "very bad" and "annihilation". A personal bankruptcy is bad but it isn't an existential risk to a person.
- reducesuffering 3y ago
  99% throughout history. During that time, humanity has gained an accelerating command over our environment; Industrial Revolution to the Information Era. We are increasingly able to create unimaginably powerful things, yet coinciding with the fundamental that it is easier to destroy than create. Nuclear weapons should dispel anyone of the notion that certain “doomsday” don’t have teeth. We are quite capable of obliterating ourselves out of existence by our unsophisticated control of technical might.
voitvodder 3y ago
[dead]

TerrifiedMouse 3y ago

> but thinking it's an existential risk is still premature Even ignoring what the technology can do, it feels me with dread that we basically have IMHO some of the worst possible people leading the charge - people who are greedy, slimy, and ruthless; who only care about money. Such a bad timeline we are in.

Qiu_Zhanxuan 3y ago
Capitalism at its finest. That's what the USA is all about.
ornornor 3y ago
> the worst possible people leading the charge - people who are greedy, slimy, and ruthless; who only care about money. And who have luxury bunkers to move to in NZ or wherever for if/when it all goes wrong.
proc0 3y ago
Well at least Demis Hassabis seems cool.