3 ms·

In my opinion this type of comment is the core problem with AI alignment these days. The current GPT models are not going to "wake up" or accidentally take ove

by ilaksh 4y ago

In my opinion this type of comment is the core problem with AI alignment these days. The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Forcing everyone who is conscientious to copy and paste the output of these relatively weak models is not going to save us from anything. If you want people to take AI regulations seriously, you have to do it in a reasonable way. Which means understanding the nuance of the dangers. Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. People need to be able to distinguish different levels of autonomy and speed. It's when we get to high levels of autonomy and performance that we run into danger. Which we are really at the cusp. But when you fail to differentiate between systems that are obviously not dangerous and future/more autonomous agents that possibly are, that makes it impossible to take the concerns seriously. And actually harder for people to understand the problems with full autonomy and superintelligent models.

ekidd 4y ago
> The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. It's pretty obvious that GPT-4 isn't consistently coherent enough to carry out complex plans or to do serious scientific research. There are several other major shortcomings which also prevent this from being viable, too. And of course, it's possible that some of these shortcomings are very hard to "fix." We've had AI winters before, and self-driving seems to have stalled. However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human. And I no longer have any solid idea how far we are from that point, especially if we make certain architectural changes. We might be closer than we think. Or at least we should stop assuming that next-gen models are obviously safe. nVidia's hope to increase training performance 1 million times in 10 years seems potentially dangerous to me. Anthrophic's plan to spend over $1 billion training next gen models seems awfully sketchy as well.
- pixl97 4y ago
  Self driving, I think is where we got it backwards. To drive correctly we need AGI because 'uncommon' but not 'rare' events are avoided by drivers all the time and in a manner you have to understand how the world works and how people think. LLMs, especially the multi-modal ones go much further to making self driving possible, that is if we ever get to the point they can incorporate and act on data fast enough. We may still have a hardware problem for some time there. --- But on your second point I am with you. There is nothing obvious about at what level emergent behavior occurs in the models. Just keep adding parameters and new things keep popping out with no real means to determine where and when seems kinda sketch as we push these things past the exa scale.
- xvilka 4y ago
  > However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human Not any time soon. Modern "AI" research built on completely wrong assumptions about how neurons work from the middle of XX century. It turned out they are much more complex than that (long story short every single neuron is analogous to the "neural network" as AI researchers understand them). See more at https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
  - kiratp 4y ago
    Why would you think that the machine implementation of “thinking” would need to mimic the biological implementation? Your brain does 1+1 = 2 very differently than an integrated circuit, but the end output is the same.
    - xvilka 4y ago
      Because it shows that real brain complexity is much larger than we thought, which means AGI is farther down the road.
ChatGTP 4y ago
The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Was it obvious to you that you’d see a paper called “Sparks of an AGI” after the release of ChatGPT 4 ?
- s_m_t 4y ago
  Not the poster you asked the question to but: I think we are going to be building and using 'sub-AGI' models for a very long time, probably well after we (probably achieve) full AGI. I think these 'sub-AGI' models are going to radically change the way we think about intelligence in the sense that we are going to be better able to nail down what we find special about human intelligence while at the same time discovering that it is a narrow and somewhat arbitrary subset of what we consider advanced intelligence. ChatGPT seems to me like a very advanced Markov chain, which isn't to denigrate it, the surprising part is how useful and effective it is which to me shows we have been thinking about intelligence the wrong way. IMO an important portion of our brain probably works in a very analogous way but you could build as advanced a version of ChatGPT as you like and it wouldn't become 'sentient' or go HAL9000 on us. There is definitely a danger of someone taking ChatGPT9 and telling it to create you computer viruses to take down digital infrastructure or telling it to trade in a way to crash the stock market but that's going to happen when all the zero-day exploits are detonated in the lead up to WW3 anyways and all things considered that has a higher chance of happening sooner than ChatGPT9 or the equivalent is used to do it. Instead of getting worried over rogue AI we should probably just harden our infrastructure.
  - pixl97 4y ago
    Hardening infrastructure is a trillion dollar problem that a huge number of people don't believe in, or believe it's a waste of money to spend even more on. And that is before we make cutting edge thinking machines that could attack it. Also, a paperclip maximizer could in theory go 'HAL' on you without intent or malice. Sentience isn't needed. And at the same time as we're working on embodiment models, what we could consider human like sentience could be far closer than we expect.
- deleted 4y ago
  [deleted]
antibasilisk 4y ago
>Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. This is a grossly irresponsible and categorically false statement, in very obvious ways.
- s_m_t 4y ago
  Why would ChatGPT provide me with dangerous code if I prompt it benignly? I just the other day used ChatGPT to create a webscraper for a friends project and it provided me with an only slightly wrong implementation to scrape data from a specific website. 15 minutes of reviewing the code and editing literally three lines and I was done. Unless it was trained on a bunch of webscraping code that also contained malicious code I just don't see how it could happen. If anything they could enable an option to have ChatGPT review the code for you and let you know that it hasn't accidentally thrown in some fishy code.
  - antibasilisk 4y ago
    >Why would ChatGPT provide me with dangerous code if I prompt it benignly? Because it's a black box that's sensitive to unexpected inputs. Humans create bugs too, but an unaligned AI doesn't have the sense to be careful of certain classes of bug.
    - maxbond 4y ago
      The burden of proof for executing arbitrary code as input should always be on proving that it doesn't violate your security or reliability model. You shouldn't need to assume anything about that code other than that you're able to configure the runtime before it executes. Whether it comes from ChatGPT or users shouldn't matter, you should assume some of those programs were written by hackers and that you don't know which ones. Even if you trust ChatGPT you should do this. You should assume that one day the hackers will intercept your connection to OpenAI and hand you malware, and they should either fail or be forced to use a zero day exploit against eg Firecracker (modulo your threat model and such).
hrpnk 4y ago
> In my opinion this type of comment is the core problem with AI alignment these days. [...] The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. [...] Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. I meant to comment on the security aspects of sending confidential information to models and letting them generate output that will be immediately executed on your own machine. People also chain models together and let them generate output that will be interpreted as the next model's input. If done without guardrails, this is done ideally in a sandboxed environment. Too often, I see this type of code just ran on one's machine. As the projects attract less technical people, they may not be aware of the dangers of running such code locally and can see a portion of their data being wiped or exposed.

by ilaksh 4y ago

ekidd 4y ago

> The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. It's pretty obvious that GPT-4 isn't consistently coherent enough to carry out complex plans or to do serious scientific research. There are several other major shortcomings which also prevent this from being viable, too. And of course, it's possible that some of these shortcomings are very hard to "fix." We've had AI winters before, and self-driving seems to have stalled. However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human. And I no longer have any solid idea how far we are from that point, especially if we make certain architectural changes. We might be closer than we think. Or at least we should stop assuming that next-gen models are obviously safe. nVidia's hope to increase training performance 1 million times in 10 years seems potentially dangerous to me. Anthrophic's plan to spend over $1 billion training next gen models seems awfully sketchy as well.

pixl97 4y ago
Self driving, I think is where we got it backwards. To drive correctly we need AGI because 'uncommon' but not 'rare' events are avoided by drivers all the time and in a manner you have to understand how the world works and how people think. LLMs, especially the multi-modal ones go much further to making self driving possible, that is if we ever get to the point they can incorporate and act on data fast enough. We may still have a hardware problem for some time there. --- But on your second point I am with you. There is nothing obvious about at what level emergent behavior occurs in the models. Just keep adding parameters and new things keep popping out with no real means to determine where and when seems kinda sketch as we push these things past the exa scale.
xvilka 4y ago
> However, I strongly suspect that it's theoretically possible to build a machine that "thinks" as well as a human Not any time soon. Modern "AI" research built on completely wrong assumptions about how neurons work from the middle of XX century. It turned out they are much more complex than that (long story short every single neuron is analogous to the "neural network" as AI researchers understand them). See more at https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
- kiratp 4y ago
  Why would you think that the machine implementation of “thinking” would need to mimic the biological implementation? Your brain does 1+1 = 2 very differently than an integrated circuit, but the end output is the same.
  - xvilka 4y ago
    Because it shows that real brain complexity is much larger than we thought, which means AGI is farther down the road.

ChatGTP 4y ago

The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Was it obvious to you that you’d see a paper called “Sparks of an AGI” after the release of ChatGPT 4 ?

s_m_t 4y ago
Not the poster you asked the question to but: I think we are going to be building and using 'sub-AGI' models for a very long time, probably well after we (probably achieve) full AGI. I think these 'sub-AGI' models are going to radically change the way we think about intelligence in the sense that we are going to be better able to nail down what we find special about human intelligence while at the same time discovering that it is a narrow and somewhat arbitrary subset of what we consider advanced intelligence. ChatGPT seems to me like a very advanced Markov chain, which isn't to denigrate it, the surprising part is how useful and effective it is which to me shows we have been thinking about intelligence the wrong way. IMO an important portion of our brain probably works in a very analogous way but you could build as advanced a version of ChatGPT as you like and it wouldn't become 'sentient' or go HAL9000 on us. There is definitely a danger of someone taking ChatGPT9 and telling it to create you computer viruses to take down digital infrastructure or telling it to trade in a way to crash the stock market but that's going to happen when all the zero-day exploits are detonated in the lead up to WW3 anyways and all things considered that has a higher chance of happening sooner than ChatGPT9 or the equivalent is used to do it. Instead of getting worried over rogue AI we should probably just harden our infrastructure.
- pixl97 4y ago
  Hardening infrastructure is a trillion dollar problem that a huge number of people don't believe in, or believe it's a waste of money to spend even more on. And that is before we make cutting edge thinking machines that could attack it. Also, a paperclip maximizer could in theory go 'HAL' on you without intent or malice. Sentience isn't needed. And at the same time as we're working on embodiment models, what we could consider human like sentience could be far closer than we expect.
deleted 4y ago
[deleted]

antibasilisk 4y ago

>Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. This is a grossly irresponsible and categorically false statement, in very obvious ways.

s_m_t 4y ago
Why would ChatGPT provide me with dangerous code if I prompt it benignly? I just the other day used ChatGPT to create a webscraper for a friends project and it provided me with an only slightly wrong implementation to scrape data from a specific website. 15 minutes of reviewing the code and editing literally three lines and I was done. Unless it was trained on a bunch of webscraping code that also contained malicious code I just don't see how it could happen. If anything they could enable an option to have ChatGPT review the code for you and let you know that it hasn't accidentally thrown in some fishy code.
- antibasilisk 4y ago
  >Why would ChatGPT provide me with dangerous code if I prompt it benignly? Because it's a black box that's sensitive to unexpected inputs. Humans create bugs too, but an unaligned AI doesn't have the sense to be careful of certain classes of bug.
  - maxbond 4y ago
    The burden of proof for executing arbitrary code as input should always be on proving that it doesn't violate your security or reliability model. You shouldn't need to assume anything about that code other than that you're able to configure the runtime before it executes. Whether it comes from ChatGPT or users shouldn't matter, you should assume some of those programs were written by hackers and that you don't know which ones. Even if you trust ChatGPT you should do this. You should assume that one day the hackers will intercept your connection to OpenAI and hand you malware, and they should either fail or be forced to use a zero day exploit against eg Firecracker (modulo your threat model and such).

hrpnk 4y ago

> In my opinion this type of comment is the core problem with AI alignment these days. [...] The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. [...] Executing "arbitrary code" that InstructGPT models outputs based on your instructions is not a danger. I meant to comment on the security aspects of sending confidential information to models and letting them generate output that will be immediately executed on your own machine. People also chain models together and let them generate output that will be interpreted as the next model's input. If done without guardrails, this is done ideally in a sandboxed environment. Too often, I see this type of code just ran on one's machine. As the projects attract less technical people, they may not be aware of the dangers of running such code locally and can see a portion of their data being wiped or exposed.