4 ms·

The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Was it obvious to you that you’d see a paper calle

The current GPT models are not going to "wake up" or accidentally take over the world. This is very obvious. Was it obvious to you that you’d see a paper called “Sparks of an AGI” after the release of ChatGPT 4 ?

s_m_t 4y ago
Not the poster you asked the question to but: I think we are going to be building and using 'sub-AGI' models for a very long time, probably well after we (probably achieve) full AGI. I think these 'sub-AGI' models are going to radically change the way we think about intelligence in the sense that we are going to be better able to nail down what we find special about human intelligence while at the same time discovering that it is a narrow and somewhat arbitrary subset of what we consider advanced intelligence. ChatGPT seems to me like a very advanced Markov chain, which isn't to denigrate it, the surprising part is how useful and effective it is which to me shows we have been thinking about intelligence the wrong way. IMO an important portion of our brain probably works in a very analogous way but you could build as advanced a version of ChatGPT as you like and it wouldn't become 'sentient' or go HAL9000 on us. There is definitely a danger of someone taking ChatGPT9 and telling it to create you computer viruses to take down digital infrastructure or telling it to trade in a way to crash the stock market but that's going to happen when all the zero-day exploits are detonated in the lead up to WW3 anyways and all things considered that has a higher chance of happening sooner than ChatGPT9 or the equivalent is used to do it. Instead of getting worried over rogue AI we should probably just harden our infrastructure.
- pixl97 4y ago
  Hardening infrastructure is a trillion dollar problem that a huge number of people don't believe in, or believe it's a waste of money to spend even more on. And that is before we make cutting edge thinking machines that could attack it. Also, a paperclip maximizer could in theory go 'HAL' on you without intent or malice. Sentience isn't needed. And at the same time as we're working on embodiment models, what we could consider human like sentience could be far closer than we expect.
deleted 4y ago
[deleted]
ilaksh 4y ago
I think unfortunately my comment got some upvotes based on an imprecise interpretation of what I actually said. Probably if people understood what I was actually saying it would have been buried. Also part of the problem is the word AGI and having total conflation inside of that. I meant SPECIFICALLY the current GPT versions and architecture. In my opinion, GPT-4 is extremely intelligent in a very general way and this is incredibly obvious. What GPT-4 is NOT, by itself, is autonomous, or many other human/animal characteristics that people conflate with intelligence. GPT does not have full autonomy. It is not a person. It is not alive. It does not feel. It does not form its own goals. It does not feel, or have survival instincts, or a high bandwidth stream of sensory data. In these and other ways it is quite different from animals and people. It is not a digital person. However, people are quite stupidly moving forward with trying to create digital people that imitate all animal characteristics, apparently with the idea that they will enslave them, with the thought process being that they will not be able to do useful work without having all of these animal (human) characteristics. Besides that problem, people are not taking into account the different levels of speed and other performance measurements of the future systems. Future quite possibly being this year. Or especially I think it is critical for the new compute-in-memory substrates or any advances that increase the speed or other IQ measurements beyond say one order of magnitude of human to be highly regulated. Systems that are twice as smart and say 100 times faster at thinking than humans absolutely can be dangerous. We achieved the architecture for general purpose intelligence in 2017 and in 2018 I suggested we would have AGI in 2019. Unfortunately now AGI is quite vague and means things like digital person or digital God..I wrote quite a lot but unfortunately deleted some of the explanation. General intelligence versus personhood or Godhood or alive, or autonomous or having survival instincts or instant adaptability etc. are not the same thing.

s_m_t 4y ago

Not the poster you asked the question to but: I think we are going to be building and using 'sub-AGI' models for a very long time, probably well after we (probably achieve) full AGI. I think these 'sub-AGI' models are going to radically change the way we think about intelligence in the sense that we are going to be better able to nail down what we find special about human intelligence while at the same time discovering that it is a narrow and somewhat arbitrary subset of what we consider advanced intelligence. ChatGPT seems to me like a very advanced Markov chain, which isn't to denigrate it, the surprising part is how useful and effective it is which to me shows we have been thinking about intelligence the wrong way. IMO an important portion of our brain probably works in a very analogous way but you could build as advanced a version of ChatGPT as you like and it wouldn't become 'sentient' or go HAL9000 on us. There is definitely a danger of someone taking ChatGPT9 and telling it to create you computer viruses to take down digital infrastructure or telling it to trade in a way to crash the stock market but that's going to happen when all the zero-day exploits are detonated in the lead up to WW3 anyways and all things considered that has a higher chance of happening sooner than ChatGPT9 or the equivalent is used to do it. Instead of getting worried over rogue AI we should probably just harden our infrastructure.

pixl97 4y ago
Hardening infrastructure is a trillion dollar problem that a huge number of people don't believe in, or believe it's a waste of money to spend even more on. And that is before we make cutting edge thinking machines that could attack it. Also, a paperclip maximizer could in theory go 'HAL' on you without intent or malice. Sentience isn't needed. And at the same time as we're working on embodiment models, what we could consider human like sentience could be far closer than we expect.

deleted 4y ago

[deleted]

ilaksh 4y ago

I think unfortunately my comment got some upvotes based on an imprecise interpretation of what I actually said. Probably if people understood what I was actually saying it would have been buried. Also part of the problem is the word AGI and having total conflation inside of that. I meant SPECIFICALLY the current GPT versions and architecture. In my opinion, GPT-4 is extremely intelligent in a very general way and this is incredibly obvious. What GPT-4 is NOT, by itself, is autonomous, or many other human/animal characteristics that people conflate with intelligence. GPT does not have full autonomy. It is not a person. It is not alive. It does not feel. It does not form its own goals. It does not feel, or have survival instincts, or a high bandwidth stream of sensory data. In these and other ways it is quite different from animals and people. It is not a digital person. However, people are quite stupidly moving forward with trying to create digital people that imitate all animal characteristics, apparently with the idea that they will enslave them, with the thought process being that they will not be able to do useful work without having all of these animal (human) characteristics. Besides that problem, people are not taking into account the different levels of speed and other performance measurements of the future systems. Future quite possibly being this year. Or especially I think it is critical for the new compute-in-memory substrates or any advances that increase the speed or other IQ measurements beyond say one order of magnitude of human to be highly regulated. Systems that are twice as smart and say 100 times faster at thinking than humans absolutely can be dangerous. We achieved the architecture for general purpose intelligence in 2017 and in 2018 I suggested we would have AGI in 2019. Unfortunately now AGI is quite vague and means things like digital person or digital God..I wrote quite a lot but unfortunately deleted some of the explanation. General intelligence versus personhood or Godhood or alive, or autonomous or having survival instincts or instant adaptability etc. are not the same thing.