3 ms·

This is a very delusional idea: "He thinks ChatGPT just might be conscious (if you squint)" It's a technology with literally no intelligence or understanding

This is a very delusional idea: "He thinks ChatGPT just might be conscious (if you squint)" It's a technology with literally no intelligence or understanding of the world of any kind. Its just statistics on data. It is as conscious as a calculator.

deleted 3y ago
[deleted]
stevenhuang 3y ago
I often observe that those dismissing this idea tend to be less informed about current insights into human cognition, philosophy, and concepts such as the information theoretic view of consciousness, neural correlates of consciousness, the free energy principle, and predictive coding. The human mind is "just statistics on data". People more informed than you are taking this seriously. You should pay attention and start inquiring why that's the case.
- cactusplant7374 3y ago
  How can it be an insight when those people don't actually understand consciousness or the brain?
- lovich 3y ago
  > People more informed than you are taking this seriously As a heuristic for why I don’t believe anyone saying llm type AI is reaching sentience I point to the fact that the same set of people are usually philosophically opposed to slavery. If you thought that this was actually AGI or sapient, then that would imply personhood and you would stop using the technology immediately since it’s forces the model to do work. Instead, everyone I’ve seen claim that these models are reaching AGI levels are also trying to figure out how to automate using them as fast as possible. There is a possibility that the set of people who’ve identified AGI accurately and early are the same set of people who are fine with slavery, but I don’t know if I could handle that happening as the default situation
  - Vecr 3y ago
    I'm actually not sure how much people in general dislike slavery. They probably do to some extent, but the actual reasons slavery is not generally done (of humans) are quite complex, including hundreds of years of various philosophical arguments, economics, and logistics. If you ask someone if they support human slavery, they probably know or can imagine the conditions involved, put themselves into that situation, and dislike it. It's much harder to put yourself into a model running on a computer somewhere, even if you intellectually think it should have rights to some extent.
MVissers 3y ago
I very much disagree that it's not intelligent. GPT-4 is clearly intelligent, and we use the lobotomized version of it. So imagine what GPT-5 can do internally. Conscious is hard to say, partly because we can't define it either, so it means something different for you and me.
kromem 3y ago
This line of nonsense has become the new "Tell me you aren't current on the past 12 months of LLM research without telling me." Harvard/MIT's Othello-GPT paper showing the development of what turned out to be linear representations of world models from training data that didn't explicitly contain that modeling is over a year old now. That in turn inspired research showing linear representations in geographical mapping and in more traditional text models around truthiness vs falsehoods. So we already have an increasing research trend that is showing over and over linear representations of more abstract modeling than "just statistics." So you are wrong that LLMs with sufficient network complexity don't develop an understanding of the world (in parts). And I'd encourage looking more into the difference between understanding the difference between training for next token prediction and the overall capabilities of the network with the smallest loss at that training task, particularly as network complexity increases.
Cacti 3y ago
OpenAI is a religious institution that self-selects for fundamentalism. This is why so many very good researchers passed on working there, they simply didn’t believe in The Mission Where Ilya Saves Humanity. This is why you need to take classes other than computers and math, kids.
packet_nerd 3y ago
Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: * "Understanding": It understood that "didn't see" implies he didn't count. * "Knowledge": It knew enough about the world to know that frogs are often green. * "Reasoning": It was able to correctly reason about how many should be subtracted from the final result. * "Math: It successfully did some basic additions and subtractions arriving at the correct answer. Crucially, I made this up right here on the spot, and used a dice for some of the numbers. This question does not exist anywhere in the training corpus! I think this demonstrates an impressive level of intelligence, for what up until about a year ago I thought a computer would ever be capable of in my lifetime. Now in absolute terms of course current gen ChatGPT is clearly far less good at reasoning and understanding than most people (well, specifically it seems to me that it's knowledge and reasoning are super-humanly broad, but child-level deep). Can future improvements to this architecture improve the depth up to "AGI", whatever that means? I have no idea. It doesn't automatically seem impossible, but maybe what we see now is already near the limit? I guess only time will tell.
- krapp 3y ago
  It answered 16 for me. Then 10 when I tried again. Then 12. And 15.
  - packet_nerd 3y ago
    ChatGPT 3.5? I'm using 4 and get 11 most times but other numbers occasionally.
    - it_citizen 3y ago
      Tried with GPT4, I got 12.
  - 2-718-281-828 3y ago
nlh 3y ago
The problem with this line of reasoning is that the exact same thing can be said about our meat-calculator brains. This gets to the philosophical heart of a debate that I can already foresee will NEVER be settled: I guarantee you - with 100% certainty - that when we get to a point where AI is "AGI", there will be a continuous and massive political debate (akin to the abortion debate we face today) where one side argues that a given AGI is conscious and must be given rights and cannot be shut off and the other side argues that it's just a calculator and a computer program and computers can be turned off at will, erased, experimented on, and whatever. We have the same debate today all the time! There are those who believe every human life is sacrosanct (from age 0-100+) and others who believe human life is disposable (from age 0-100+!). There's no reason to believe this debate won't extend to AGI.

deleted 3y ago

[deleted]

stevenhuang 3y ago

I often observe that those dismissing this idea tend to be less informed about current insights into human cognition, philosophy, and concepts such as the information theoretic view of consciousness, neural correlates of consciousness, the free energy principle, and predictive coding. The human mind is "just statistics on data". People more informed than you are taking this seriously. You should pay attention and start inquiring why that's the case.

cactusplant7374 3y ago
How can it be an insight when those people don't actually understand consciousness or the brain?
lovich 3y ago
> People more informed than you are taking this seriously As a heuristic for why I don’t believe anyone saying llm type AI is reaching sentience I point to the fact that the same set of people are usually philosophically opposed to slavery. If you thought that this was actually AGI or sapient, then that would imply personhood and you would stop using the technology immediately since it’s forces the model to do work. Instead, everyone I’ve seen claim that these models are reaching AGI levels are also trying to figure out how to automate using them as fast as possible. There is a possibility that the set of people who’ve identified AGI accurately and early are the same set of people who are fine with slavery, but I don’t know if I could handle that happening as the default situation
- Vecr 3y ago
  I'm actually not sure how much people in general dislike slavery. They probably do to some extent, but the actual reasons slavery is not generally done (of humans) are quite complex, including hundreds of years of various philosophical arguments, economics, and logistics. If you ask someone if they support human slavery, they probably know or can imagine the conditions involved, put themselves into that situation, and dislike it. It's much harder to put yourself into a model running on a computer somewhere, even if you intellectually think it should have rights to some extent.

MVissers 3y ago

I very much disagree that it's not intelligent. GPT-4 is clearly intelligent, and we use the lobotomized version of it. So imagine what GPT-5 can do internally. Conscious is hard to say, partly because we can't define it either, so it means something different for you and me.

kromem 3y ago

This line of nonsense has become the new "Tell me you aren't current on the past 12 months of LLM research without telling me." Harvard/MIT's Othello-GPT paper showing the development of what turned out to be linear representations of world models from training data that didn't explicitly contain that modeling is over a year old now. That in turn inspired research showing linear representations in geographical mapping and in more traditional text models around truthiness vs falsehoods. So we already have an increasing research trend that is showing over and over linear representations of more abstract modeling than "just statistics." So you are wrong that LLMs with sufficient network complexity don't develop an understanding of the world (in parts). And I'd encourage looking more into the difference between understanding the difference between training for next token prediction and the overall capabilities of the network with the smallest loss at that training task, particularly as network complexity increases.

Cacti 3y ago

OpenAI is a religious institution that self-selects for fundamentalism. This is why so many very good researchers passed on working there, they simply didn’t believe in The Mission Where Ilya Saves Humanity. This is why you need to take classes other than computers and math, kids.

packet_nerd 3y ago

Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: * "Understanding": It understood that "didn't see" implies he didn't count. * "Knowledge": It knew enough about the world to know that frogs are often green. * "Reasoning": It was able to correctly reason about how many should be subtracted from the final result. * "Math: It successfully did some basic additions and subtractions arriving at the correct answer. Crucially, I made this up right here on the spot, and used a dice for some of the numbers. This question does not exist anywhere in the training corpus! I think this demonstrates an impressive level of intelligence, for what up until about a year ago I thought a computer would ever be capable of in my lifetime. Now in absolute terms of course current gen ChatGPT is clearly far less good at reasoning and understanding than most people (well, specifically it seems to me that it's knowledge and reasoning are super-humanly broad, but child-level deep). Can future improvements to this architecture improve the depth up to "AGI", whatever that means? I have no idea. It doesn't automatically seem impossible, but maybe what we see now is already near the limit? I guess only time will tell.

krapp 3y ago
It answered 16 for me. Then 10 when I tried again. Then 12. And 15.
- packet_nerd 3y ago
  ChatGPT 3.5? I'm using 4 and get 11 most times but other numbers occasionally.
  - it_citizen 3y ago
    Tried with GPT4, I got 12.
- 2-718-281-828 3y ago

nlh 3y ago

The problem with this line of reasoning is that the exact same thing can be said about our meat-calculator brains. This gets to the philosophical heart of a debate that I can already foresee will NEVER be settled: I guarantee you - with 100% certainty - that when we get to a point where AI is "AGI", there will be a continuous and massive political debate (akin to the abortion debate we face today) where one side argues that a given AGI is conscious and must be given rights and cannot be shut off and the other side argues that it's just a calculator and a computer program and computers can be turned off at will, erased, experimented on, and whatever. We have the same debate today all the time! There are those who believe every human life is sacrosanct (from age 0-100+) and others who believe human life is disposable (from age 0-100+!). There's no reason to believe this debate won't extend to AGI.