Agreed, and this is where there needs to be a line drawn. GPT is trained to emulate the patterns in the text it was trained in, and it is very likely learning h

Agreed, and this is where there needs to be a line drawn. GPT is trained to emulate the patterns in the text it was trained in, and it is very likely learning higher level relations between concepts/states where it needs to in order to make better predictions. And this is amazing in of itself. But that doesn’t mean it ‘feels emotions’ related to these concepts because it hasn’t had the billions of years of reinforcement learning that we have to tell us some of these concepts should induce fear/desire etc. I have no doubt that AGI is possible but I really don’t expect the intelligence that results to resemble human intelligence. I would expect dolphin intelligence to be more ‘similar’ to human intelligence since at least we have a common ancestor.