8 ms·

You have to give them credit for putting their money where their mouth is here. But it's also easy to parody this. I am just imagining Ilya and Jan coming out

by ilaksh 3y ago

You have to give them credit for putting their money where their mouth is here. But it's also easy to parody this. I am just imagining Ilya and Jan coming out on stage wearing red capes. I think George Hotz made sense when he pointed out that the best defense will be having the technology available to everyone rather than a small group. We can at least try to create a collective "digital immune system" against unaligned agents with our own majority of aligned agents. But I also believe that there isn't any really effective mitigation against superintelligence superseding human decision making aside from just not deploying it. And it doesn't need to be alive or anything to be dangerous. All you need is for a large amount of decision-making for critical systems to be given over to hyperspeed AI and that creates a brittle situation where things like computer viruses can be existential risks. It's something similar to the danger of nuclear weapons. Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster and more efficient, that can lead to control of industrial and military assets being handed over to these AI agents. Because the agents are so much faster, humans cannot possibly compete, and if you interrupt them to try to give them new instructions then your competitor's AIs race ahead the equivalent of days or weeks of work. This, again, is a precarious situation to be in. There is huge promise and benefit from making the systems faster, smarter, and more efficient, but in the next few years we may be walking a fine line. We should agree to place some limitation on the performance level of AI hardware that we will design and manufacture.

JimtheCoder 3y ago
"Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster and more efficient, that can lead to control of industrial and military assets being handed over to these AI agents." I call BS on this...it's an LLM...
- crop_rotation 3y ago
  Saying it's "an LLM" doesn't change the impact. GPT4 is an LLM, and so are many others ranging from toy quality to GPT3.5. It is very clear GPT4 is much better. If there is another jump like GPT4 , whether it is LLM or not, it's impact will be huge.
  - esafak 3y ago
    Plus the next thing might not be an LLM.
  - woadwarrior01 3y ago
    Meanwhile, GPT-4 still can’t reliably multiply small numbers. https://arxiv.org/abs/2304.02015 https://arxiv.org/abs/2304.02015
    - fprotthetarball 3y ago
      A minor inconvenience when GPT-4 has no problem learning how to use a code interpreter.
    - mhb 3y ago
      Do you find that comforting when an emergent property of a system whose objective is to complete the next word is able to make drawings?
      - StrictDabbler 3y ago
        Imagine you meet a human who is eloquent, expressive, speaks ten languages, can pass the bar or the medical board exams easily, but who cannot reliably distinguish between truth and falsehood on the smallest of questions ("what is 6x9? 42") and has no persistent memory or sense of self. Would you be "comforted" that this mega-genius is worse at arithmetic than you are and doesn't remember what it did yesterday? Probably not. You might well be worried that this weird psychopath is going to get a medical license and cut the wrong number of fingers off of a whole bunch of patients.
        mhb 3y ago
        We're agreeing, aren't we?
    - famouswaffles 3y ago
      It's alright with algorithmic prompts - https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 also it knows when to use a calculator if it has access to one so it's not a big deal
- chaxor 3y ago
  It's important to recognize that the model is fully capable of operating in open world environments, with visual stimuli and motor output, go achieve high level tasks. This has been demonstrated in proofs of concepts several times now with systems such as voyager et al. So, while there are certainly some details that are important, much of them are the annoyances that we devs deal with all the time (how to connect various parts of a system properly, etc) the fundamental capabilities of expressivity in these models are not that limited. Certainly limited in some sense (as seen in the several papers applying category theoretic arguments to transformers) but for many engineering applications in the world, these models are very capable and useful. Guarantees of correctness and safety are obviously of huge concern, hence the main article. But it's absolutely not unreasonable to see these models allowing humanoid robots capable of various day to day activities and work.
  - DennisP 3y ago
    To save others the trouble, I googled Voyager, it's pretty interesting. I had no idea an LLM could do this sort of thing: https://voyager.minedojo.org/ https://voyager.minedojo.org/
    - famouswaffles 3y ago
      Other examples(in the real world) you might find interesting. https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/ https://innermonologue.github.io/ https://innermonologue.github.io/ https://palm-e.github.io/ https://palm-e.github.io/ https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy...
      - Animats 3y ago
        > https://palm-e.github.io/ https://palm-e.github.io/ The alignment problem will come up when the robot control system notices that the guy with the stick is interfering with the robot's goals.
- 908087 3y ago
  [dead]
- Footkerchief 3y ago
  Military command and control is already performed via input and output of token streams.
- ben_w 3y ago
  It's autocomplete on steroids… That can guide me through the process of writing a Navier-Stokes simulation… In a foreign language… That can be trivially put into a loop and tasked with acting like an agent… And which is good enough that people are already seriously asking themselves if they need to hire people to do certain tasks… … Why call BS? It's not perfect, sure, but it's not making a highly regional joke about the Isle of White Ferry[0] either. [0] "What's brown and comes steaming out the back of Cowes?"
  - wickoff 3y ago
    But you're also autocomplete (prediction engine) on steroids. https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-machine-it-knows-how-good-we-are-doing-something-before-we-even-try https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-mach...
    - ben_w 3y ago
      "It's one of those irregular verbs, isn't it? I'm good at improv and speaking on my feet, you finish each other's sentences, they're just autocomplete on steroids." https://en.wikiquote.org/wiki/Yes,_Minister https://en.wikiquote.org/wiki/Yes,_Minister
- reaperman 3y ago
  GPT-4 is actually multimodal, not just an LLM. OpenAI just doesn't provide the public with any way to use the image embedding capabilities.
- nopinsight 3y ago
  LLM is already a misnomer. Many of the latest models are better called LFMs (Large Foundation Models). They have multimodal capabilities. Some can even handle sensory input humans can't. Another comment already links to demos and papers of LFMs operating robots and agents in 3D environments.
gonehome 3y ago
The recent paper about using gpt-4 to give more insight into its actual internals was interesting, but yeah the risks seem really high at the moment that we'd accidentally develop unaligned AGI before figuring out alignment. Out of the options to reduce that risk I think it would really take something like this, which also seems extremely unlikely to actually happen given the coordination problem: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... You talk about aligned agents - but there aren't any today and we don't know how to make them. It wouldn't be aligned agents vs. unaligned, it's only unaligned. I don't think spreading out the tech reduces the risk. Spreading out nuclear weapons doesn't reduce the risk (and with nukes at least it's a lot easier to control the fissionable materials). Even with nukes you can still create them and decide not to use them, not so true with superintelligent AGI. If anyone could have made nukes from their computer humanity may not have made it. I'm glad OpenAI understands the severity of the problem though and is at least trying to solve it in time.
- lukeschlather 3y ago
  Unaligned doesn't really seem like it should be a threat. If it's unaligned it can't work toward any goal. The danger is that it aligns with some anti-goal. If you've got a bunch of agents all working unaligned, they will work at cross-purposes and won't be able to out-think us.
  - jdasdf 3y ago
    This is a misunderstanding of what AI alignment problems are all about. Alignment != capability Think a paperclip maximizing robot that in its process of creating paperclips kills everyone on earth to turn them into paperclips.
    - climatologist 3y ago
      Corporations like Saudi Aramco are already doing that. You don't need a superintelligent AI, corporations that maximize profit are already sufficient as misaligned superhuman agents.
deleted 3y ago
[deleted]
sagebird 3y ago
It’s not money where their mouth is… It’s paying a cost of doing business. The minimum theater required to minimize expected regulatory cost. They want to own the saftey issue so they can risk your life for their profit.
- janalsncm 3y ago
  Call it insurance. It’s the R&D cost to try to make sure your models don’t do/say anything that will get you into trouble.
arisAlexis 3y ago
"And it doesn't need to be alive or anything to be dangerous" Why are tech people stuck in the now and not future looking?
- ilaksh 3y ago
  I just think it's much easier to convince people that existing types of AIs will get somewhat smarter and significantly faster. And that's dangerous enough. My own belief is that regardless of what we do in terms of the most immediate dangers, within one or two centuries (maximum) we will enter the posthuman era where digital intelligent life has taken control of the planet. I don't mean "posthuman" as in all of the humans have been killed (necessarily), just that what humans 1.0 do won't be very important or interesting relative to what the superintelligent AIs are doing. I don't think there is anything that prevents people from giving AI all of the characteristics of animals (such as humans). I think it's foolish, but researchers seem determined to do it. But this is fairly speculative and much harder to convince people of.
  - c_crank 3y ago
    If the value of superintelligence is to lead to an Age of Em scenario where AIs (or Ems) do most of the intellectual labor, the reality is still that they would be doing this labor in service of humans. I could see a scenario where it is done in service of the AIs instead, but it would look nothing like the existential risk stuff bandied about by these weenies.
    - Footkerchief 3y ago
      There is no example in our knowledge of any lifeform prioritizing (writ large) the well-being of a different lifeform over its own.
      - c_crank 3y ago
        Why call AIs a life form? They aren't like cellular life.
c_crank 3y ago
Control of military and industrial assets won't be handed willy nilly to AIs, given the threat of legal liability for any mistakes the AI could make. Their tendency of making things up is well known by now.
- mhb 3y ago
  You've been spewing out nonsense at an impressive pace. Stop digging. Read more, write less.
- sdenton4 3y ago
  And yet the military regularly hands machine guns to 18 year-olds...
  - c_crank 3y ago
    The 18 year old human alignment problem has been solved pretty well. Not perfectly , but enough to justify handing out such weapons.
skybrian 3y ago
> the best defense will be having the technology available to everyone rather than a small group Proliferation of a dangerous technology is the best defense? Sure, it's a libertarian meme, but it wouldn't work for nuclear weapons or virus research. Maybe that would make sense, but the argument needs to be made.
- flagrant_taco 3y ago
  There's nothing libertarian about this. Inventing and securing dangerous technology is an affront to individual freedoms. The idea of proliferation as deterance is totally separate from libertarianism and seeded more in fear than anything else.
  - skybrian 3y ago
    The prioritization of individual freedoms above most other considerations (along with the assumption that it will work out better in the end) is what libertarianism is all about. Maybe you’re a libertarian without realizing it? :)
dlkf 3y ago
> Even if you just make GPT-4 say 33% smarter What is your unit of intelligence?
isaacfrond 3y ago
> Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster (...) humans cannot possibly compete I sincerely doubt that. Gpt4 and it's ilk excel at the 5 paragraph essay on topics that are so well understood by humans that books have been written about them. ChatGPT4 is a very useful took when writing text. But it is useful in the sense that a thesaurus and a spell check use useful. What chatGPT4 truly sucks at is understanding a large amount of text and synthesizing it. That token limit is really a problem if you want gpt to become a scientist or a military strategist. Strategy requires you to consume a huge amount of less than certain information and to synthesize that in a coherent strategy, preferably explainable in terms potus can understand. Science is the same thing. Play the the Phd game that just featured on HN frontpage. It is a lot of false starts, a lot of reading, again things gpt just cannot do. By the way their text understanding is really a lot less than human. A nice example are 'word in context' puzzle's. In this puzzle a target word is used in two different sentences. The puzzle is to decide if the word is used in the same meaning or not. chatGpt4 does better than 3.5 but it doesn't take a lot of effort to trick it. Especially if you ask a couple of questions in one prompt, it will easily trip up.

by ilaksh 3y ago

JimtheCoder 3y ago

"Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster and more efficient, that can lead to control of industrial and military assets being handed over to these AI agents." I call BS on this...it's an LLM...

crop_rotation 3y ago
Saying it's "an LLM" doesn't change the impact. GPT4 is an LLM, and so are many others ranging from toy quality to GPT3.5. It is very clear GPT4 is much better. If there is another jump like GPT4 , whether it is LLM or not, it's impact will be huge.
- esafak 3y ago
  Plus the next thing might not be an LLM.
- woadwarrior01 3y ago
  Meanwhile, GPT-4 still can’t reliably multiply small numbers. https://arxiv.org/abs/2304.02015 https://arxiv.org/abs/2304.02015
  - fprotthetarball 3y ago
    A minor inconvenience when GPT-4 has no problem learning how to use a code interpreter.
  - mhb 3y ago
    Do you find that comforting when an emergent property of a system whose objective is to complete the next word is able to make drawings?
    - StrictDabbler 3y ago
      Imagine you meet a human who is eloquent, expressive, speaks ten languages, can pass the bar or the medical board exams easily, but who cannot reliably distinguish between truth and falsehood on the smallest of questions ("what is 6x9? 42") and has no persistent memory or sense of self. Would you be "comforted" that this mega-genius is worse at arithmetic than you are and doesn't remember what it did yesterday? Probably not. You might well be worried that this weird psychopath is going to get a medical license and cut the wrong number of fingers off of a whole bunch of patients.
      - mhb 3y ago
        We're agreeing, aren't we?
  - famouswaffles 3y ago
    It's alright with algorithmic prompts - https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 also it knows when to use a calculator if it has access to one so it's not a big deal
chaxor 3y ago
It's important to recognize that the model is fully capable of operating in open world environments, with visual stimuli and motor output, go achieve high level tasks. This has been demonstrated in proofs of concepts several times now with systems such as voyager et al. So, while there are certainly some details that are important, much of them are the annoyances that we devs deal with all the time (how to connect various parts of a system properly, etc) the fundamental capabilities of expressivity in these models are not that limited. Certainly limited in some sense (as seen in the several papers applying category theoretic arguments to transformers) but for many engineering applications in the world, these models are very capable and useful. Guarantees of correctness and safety are obviously of huge concern, hence the main article. But it's absolutely not unreasonable to see these models allowing humanoid robots capable of various day to day activities and work.
- DennisP 3y ago
  To save others the trouble, I googled Voyager, it's pretty interesting. I had no idea an LLM could do this sort of thing: https://voyager.minedojo.org/ https://voyager.minedojo.org/
  - famouswaffles 3y ago
    Other examples(in the real world) you might find interesting. https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/ https://innermonologue.github.io/ https://innermonologue.github.io/ https://palm-e.github.io/ https://palm-e.github.io/ https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy...
    - Animats 3y ago
      > https://palm-e.github.io/ https://palm-e.github.io/ The alignment problem will come up when the robot control system notices that the guy with the stick is interfering with the robot's goals.
908087 3y ago
[dead]
Footkerchief 3y ago
Military command and control is already performed via input and output of token streams.
ben_w 3y ago
It's autocomplete on steroids… That can guide me through the process of writing a Navier-Stokes simulation… In a foreign language… That can be trivially put into a loop and tasked with acting like an agent… And which is good enough that people are already seriously asking themselves if they need to hire people to do certain tasks… … Why call BS? It's not perfect, sure, but it's not making a highly regional joke about the Isle of White Ferry[0] either. [0] "What's brown and comes steaming out the back of Cowes?"
- wickoff 3y ago
  But you're also autocomplete (prediction engine) on steroids. https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-machine-it-knows-how-good-we-are-doing-something-before-we-even-try https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-mach...
  - ben_w 3y ago
    "It's one of those irregular verbs, isn't it? I'm good at improv and speaking on my feet, you finish each other's sentences, they're just autocomplete on steroids." https://en.wikiquote.org/wiki/Yes,_Minister https://en.wikiquote.org/wiki/Yes,_Minister
reaperman 3y ago
GPT-4 is actually multimodal, not just an LLM. OpenAI just doesn't provide the public with any way to use the image embedding capabilities.
nopinsight 3y ago
LLM is already a misnomer. Many of the latest models are better called LFMs (Large Foundation Models). They have multimodal capabilities. Some can even handle sensory input humans can't. Another comment already links to demos and papers of LFMs operating robots and agents in 3D environments.

gonehome 3y ago

The recent paper about using gpt-4 to give more insight into its actual internals was interesting, but yeah the risks seem really high at the moment that we'd accidentally develop unaligned AGI before figuring out alignment. Out of the options to reduce that risk I think it would really take something like this, which also seems extremely unlikely to actually happen given the coordination problem: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... You talk about aligned agents - but there aren't any today and we don't know how to make them. It wouldn't be aligned agents vs. unaligned, it's only unaligned. I don't think spreading out the tech reduces the risk. Spreading out nuclear weapons doesn't reduce the risk (and with nukes at least it's a lot easier to control the fissionable materials). Even with nukes you can still create them and decide not to use them, not so true with superintelligent AGI. If anyone could have made nukes from their computer humanity may not have made it. I'm glad OpenAI understands the severity of the problem though and is at least trying to solve it in time.

lukeschlather 3y ago
Unaligned doesn't really seem like it should be a threat. If it's unaligned it can't work toward any goal. The danger is that it aligns with some anti-goal. If you've got a bunch of agents all working unaligned, they will work at cross-purposes and won't be able to out-think us.
- jdasdf 3y ago
  This is a misunderstanding of what AI alignment problems are all about. Alignment != capability Think a paperclip maximizing robot that in its process of creating paperclips kills everyone on earth to turn them into paperclips.
  - climatologist 3y ago
    Corporations like Saudi Aramco are already doing that. You don't need a superintelligent AI, corporations that maximize profit are already sufficient as misaligned superhuman agents.

deleted 3y ago

[deleted]

sagebird 3y ago

It’s not money where their mouth is… It’s paying a cost of doing business. The minimum theater required to minimize expected regulatory cost. They want to own the saftey issue so they can risk your life for their profit.

janalsncm 3y ago
Call it insurance. It’s the R&D cost to try to make sure your models don’t do/say anything that will get you into trouble.

arisAlexis 3y ago

"And it doesn't need to be alive or anything to be dangerous" Why are tech people stuck in the now and not future looking?

ilaksh 3y ago
I just think it's much easier to convince people that existing types of AIs will get somewhat smarter and significantly faster. And that's dangerous enough. My own belief is that regardless of what we do in terms of the most immediate dangers, within one or two centuries (maximum) we will enter the posthuman era where digital intelligent life has taken control of the planet. I don't mean "posthuman" as in all of the humans have been killed (necessarily), just that what humans 1.0 do won't be very important or interesting relative to what the superintelligent AIs are doing. I don't think there is anything that prevents people from giving AI all of the characteristics of animals (such as humans). I think it's foolish, but researchers seem determined to do it. But this is fairly speculative and much harder to convince people of.
- c_crank 3y ago
  If the value of superintelligence is to lead to an Age of Em scenario where AIs (or Ems) do most of the intellectual labor, the reality is still that they would be doing this labor in service of humans. I could see a scenario where it is done in service of the AIs instead, but it would look nothing like the existential risk stuff bandied about by these weenies.
  - Footkerchief 3y ago
    There is no example in our knowledge of any lifeform prioritizing (writ large) the well-being of a different lifeform over its own.
    - c_crank 3y ago
      Why call AIs a life form? They aren't like cellular life.

c_crank 3y ago

Control of military and industrial assets won't be handed willy nilly to AIs, given the threat of legal liability for any mistakes the AI could make. Their tendency of making things up is well known by now.

mhb 3y ago
You've been spewing out nonsense at an impressive pace. Stop digging. Read more, write less.
sdenton4 3y ago
And yet the military regularly hands machine guns to 18 year-olds...
- c_crank 3y ago
  The 18 year old human alignment problem has been solved pretty well. Not perfectly , but enough to justify handing out such weapons.

skybrian 3y ago

> the best defense will be having the technology available to everyone rather than a small group Proliferation of a dangerous technology is the best defense? Sure, it's a libertarian meme, but it wouldn't work for nuclear weapons or virus research. Maybe that would make sense, but the argument needs to be made.

flagrant_taco 3y ago
There's nothing libertarian about this. Inventing and securing dangerous technology is an affront to individual freedoms. The idea of proliferation as deterance is totally separate from libertarianism and seeded more in fear than anything else.
- skybrian 3y ago
  The prioritization of individual freedoms above most other considerations (along with the assumption that it will work out better in the end) is what libertarianism is all about. Maybe you’re a libertarian without realizing it? :)

dlkf 3y ago

> Even if you just make GPT-4 say 33% smarter What is your unit of intelligence?

isaacfrond 3y ago

> Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster (...) humans cannot possibly compete I sincerely doubt that. Gpt4 and it's ilk excel at the 5 paragraph essay on topics that are so well understood by humans that books have been written about them. ChatGPT4 is a very useful took when writing text. But it is useful in the sense that a thesaurus and a spell check use useful. What chatGPT4 truly sucks at is understanding a large amount of text and synthesizing it. That token limit is really a problem if you want gpt to become a scientist or a military strategist. Strategy requires you to consume a huge amount of less than certain information and to synthesize that in a coherent strategy, preferably explainable in terms potus can understand. Science is the same thing. Play the the Phd game that just featured on HN frontpage. It is a lot of false starts, a lot of reading, again things gpt just cannot do. By the way their text understanding is really a lot less than human. A nice example are 'word in context' puzzle's. In this puzzle a target word is used in two different sentences. The puzzle is to decide if the word is used in the same meaning or not. chatGpt4 does better than 3.5 but it doesn't take a lot of effort to trick it. Especially if you ask a couple of questions in one prompt, it will easily trip up.