Came here to say this. And to add: we don't know to what degree human behaviour is regular. It could be that in practice an AI can't really generalise well to

Came here to say this. And to add: we don't know to what degree human behaviour is regular. It could be that in practice an AI can't really generalise well to All People compared to humans, because the humans are so diverse and the humans who are doing some jobs generalise to just the small set of people they are working with. Right now it does look like prompts like "answer this as if you're one of Napoleon's generals" work fairly well, but I wonder if in practice we'll find out that specific human experiences which are hard to document will turn out to create niches which AI is very very poor at dealing with. Potential examples: * what does it feel like to fall in love? humans can't describe well. * what does it feel like to summit Everest for the first time? ditto. * what are spiritual experiences like? Probably many other areas which are clearly important to people, particularly falling in love, but language is weak, and at the end of the day... It's a large language model.