4 ms·

I cringe every time I read the words "go rogue" in an AI article. The state of the art is still basically madlibs. Sure, I'm worried about terrorists using AI t

by bloppe 2y ago

I cringe every time I read the words "go rogue" in an AI article. The state of the art is still basically madlibs. Sure, I'm worried about terrorists using AI to generate new bioweapons, but including those concerns in the same breath as something about skynet makes it sound childish.

mistrial9 2y ago
this answer fully misses fundamental aspects of computers and networks. Computers can work 24x7 week after week, month after month. Large computer clouds can answer millions of queries a minute, as long as required. Modern data tech enables the complete and total collection of specialist knowledge of XYZ to be available at all times, with competitive accuracy. This answer is not applying the imagination to circumstances where events happen in real time involving millions of people, their transportation systems, their communication systems and power disparity between those driving the AIs and those relying on public systems or market systems. Since some AIs are built as "goal seeking" they can iterate through whatever resources they have internally, much faster than any person or team of people can track. Much of modern AI is literally a black box when it comes to tracing what happens to reach some abstractly defined goal. Your cringe is a cringe at your own myopia. I am not paranoid nor exaggerating.. Others with credentials in this field, and others with details of system behavior today, are saying things similar to this.
- toasterlovin 2y ago
  Out here in physical reality you still need humans to do things. That isn't changing any time soon.
  - mistrial9 2y ago
    more glossing over the obvious.. automated systems of all kinds are deployed today; monitoring and access control of physical spaces -- rapidly expanding.. locks, access codes, fueling, finance flows.. do not even consider finance flows as "humans need to do things" ?
  - danielmarkbruce 2y ago
    Patronizing and wrong. Drones which return and charge, and can be controlled by software, exist. Automated hacking of systems exists. Various pieces of infrastructure are completely at the control of software systems. LLMs can write software. It takes very little imagination to see things going badly. The straw man of "AI has no goals, can't run by itself blah blah blah" is ridiculous. That's not the threat. The threat is a raving lunatic with some serious skills which gives the LLM goals and a computer from where to execute code to get them done.
    - bloppe 2y ago
      Autonomous killer drones are scary. But that's not AI "going rogue". They're working as designed. That's really the only factor that makes AI scary to me: bad human actors.
      - sabbaticaldev 2y ago
        a LLM with event loop, access to the internet and long term memory will be able to hack autonomous killer drones
        danielmarkbruce 2y ago
        /probably is trying right now.
      - danielmarkbruce 2y ago
        It actually is the real going rogue scenario. A person kicks it off with the set of goals and a little bit of compute resources, and perhaps some financial assets. It can hire people, order things, program things, hack things etc.
- benreesman 2y ago
  It might be reasonable to call AlphaZero “goal-seeking”. The DeepMind people are actually working on stuff that can cope with diverse environments and pursue open-ended strategies. That’s not what anyone means by “AI” in this context.
  - danielmarkbruce 2y ago
    You can give an LLM a goal in the prompt, and set of possible actions which can be taken by calling APIs, and a state which can be called via API, and ask it what to do, then do it, in a loop. I've done it. It's trivial. It's far from trivial to get it to do something especially sophisticated, but... it's not hard to see a couple generations away it will be possible to do some real damage.
  - mistrial9 2y ago
    (I want to course correct the thread back to an inquiry frame) .. very relevant that there is not one definition of "AI" at all, and certainly many flavors and architectures of setups right now that all fall under an umbrella term of "AI" -- agree DeepMind is a peculiar example, since they started by solving video games explicitly (very narrow and goal oriented)
    - benreesman 2y ago
      Yeah and there is no shortage of interview footage where Hassabis made a clear, compelling argument around why Atari was a better starting point for the long road to true digital intelligence than click prediction. There are a number of reasons why I knew OpenAI was a scam including the fact that a metric fuck ton of my former colleagues work there and they all skew a way, but the biggest reason is that I took one look at the Instruct paper and I was like “differentiable loss function on getting humans to click: feed ranking on crack. they’ll make a lot of money but it’ll never work out”.
      - richardatlarge 2y ago
        Just out of curiosity, mathematically, how much more is a fuck ton than just a ton?
- bloppe 2y ago
  Oof... The world is complicated and unforgiving. It's hard enough being a reasonably efficient human being. A super intelligent AI that needs to consume a small town's worth of electricity just to navigate the world through an internet connection is gonna have a hard time staying on once the humans decide to turn it off. Even if humans had given it access to state of the art military drones and it's own personal nuclear reactor (which is difficult to do by accident), it's hard to imagine it putting up a good fight. Drones can be jammed, super intelligence doesn't fit in an autonomous-drone-sized package (again, mostly due to power requirements), and any other kind of power supply is subject to supply chains with way more than enough human choke points to throttle a rogue AI. And it would be difficult to "go distributed" and somehow embed itself in the fabric of the internet itself, although that sounds like a really fun area of research. Make an LLM trojan who's only goal is to to spread itself? Maybe with some sort of evolutionary process to make it interesting? Consider me intrigued. But I run LLMs locally and it would be pretty hard to spread a super basic 10GB model without it getting noticed pretty quick. Let alone a hypothetical super intelligent one that probably needs the latest Nvidia just to avoid page thrashing.

by bloppe 2y ago

mistrial9 2y ago

this answer fully misses fundamental aspects of computers and networks. Computers can work 24x7 week after week, month after month. Large computer clouds can answer millions of queries a minute, as long as required. Modern data tech enables the complete and total collection of specialist knowledge of XYZ to be available at all times, with competitive accuracy. This answer is not applying the imagination to circumstances where events happen in real time involving millions of people, their transportation systems, their communication systems and power disparity between those driving the AIs and those relying on public systems or market systems. Since some AIs are built as "goal seeking" they can iterate through whatever resources they have internally, much faster than any person or team of people can track. Much of modern AI is literally a black box when it comes to tracing what happens to reach some abstractly defined goal. Your cringe is a cringe at your own myopia. I am not paranoid nor exaggerating.. Others with credentials in this field, and others with details of system behavior today, are saying things similar to this.

toasterlovin 2y ago
Out here in physical reality you still need humans to do things. That isn't changing any time soon.
- mistrial9 2y ago
  more glossing over the obvious.. automated systems of all kinds are deployed today; monitoring and access control of physical spaces -- rapidly expanding.. locks, access codes, fueling, finance flows.. do not even consider finance flows as "humans need to do things" ?
- danielmarkbruce 2y ago
  Patronizing and wrong. Drones which return and charge, and can be controlled by software, exist. Automated hacking of systems exists. Various pieces of infrastructure are completely at the control of software systems. LLMs can write software. It takes very little imagination to see things going badly. The straw man of "AI has no goals, can't run by itself blah blah blah" is ridiculous. That's not the threat. The threat is a raving lunatic with some serious skills which gives the LLM goals and a computer from where to execute code to get them done.
  - bloppe 2y ago
    Autonomous killer drones are scary. But that's not AI "going rogue". They're working as designed. That's really the only factor that makes AI scary to me: bad human actors.
    - sabbaticaldev 2y ago
      a LLM with event loop, access to the internet and long term memory will be able to hack autonomous killer drones
      - danielmarkbruce 2y ago
        /probably is trying right now.
    - danielmarkbruce 2y ago
      It actually is the real going rogue scenario. A person kicks it off with the set of goals and a little bit of compute resources, and perhaps some financial assets. It can hire people, order things, program things, hack things etc.
benreesman 2y ago
It might be reasonable to call AlphaZero “goal-seeking”. The DeepMind people are actually working on stuff that can cope with diverse environments and pursue open-ended strategies. That’s not what anyone means by “AI” in this context.
- danielmarkbruce 2y ago
  You can give an LLM a goal in the prompt, and set of possible actions which can be taken by calling APIs, and a state which can be called via API, and ask it what to do, then do it, in a loop. I've done it. It's trivial. It's far from trivial to get it to do something especially sophisticated, but... it's not hard to see a couple generations away it will be possible to do some real damage.
- mistrial9 2y ago
  (I want to course correct the thread back to an inquiry frame) .. very relevant that there is not one definition of "AI" at all, and certainly many flavors and architectures of setups right now that all fall under an umbrella term of "AI" -- agree DeepMind is a peculiar example, since they started by solving video games explicitly (very narrow and goal oriented)
  - benreesman 2y ago
    Yeah and there is no shortage of interview footage where Hassabis made a clear, compelling argument around why Atari was a better starting point for the long road to true digital intelligence than click prediction. There are a number of reasons why I knew OpenAI was a scam including the fact that a metric fuck ton of my former colleagues work there and they all skew a way, but the biggest reason is that I took one look at the Instruct paper and I was like “differentiable loss function on getting humans to click: feed ranking on crack. they’ll make a lot of money but it’ll never work out”.
    - richardatlarge 2y ago
      Just out of curiosity, mathematically, how much more is a fuck ton than just a ton?
bloppe 2y ago
Oof... The world is complicated and unforgiving. It's hard enough being a reasonably efficient human being. A super intelligent AI that needs to consume a small town's worth of electricity just to navigate the world through an internet connection is gonna have a hard time staying on once the humans decide to turn it off. Even if humans had given it access to state of the art military drones and it's own personal nuclear reactor (which is difficult to do by accident), it's hard to imagine it putting up a good fight. Drones can be jammed, super intelligence doesn't fit in an autonomous-drone-sized package (again, mostly due to power requirements), and any other kind of power supply is subject to supply chains with way more than enough human choke points to throttle a rogue AI. And it would be difficult to "go distributed" and somehow embed itself in the fabric of the internet itself, although that sounds like a really fun area of research. Make an LLM trojan who's only goal is to to spread itself? Maybe with some sort of evolutionary process to make it interesting? Consider me intrigued. But I run LLMs locally and it would be pretty hard to spread a super basic 10GB model without it getting noticed pretty quick. Let alone a hypothetical super intelligent one that probably needs the latest Nvidia just to avoid page thrashing.