> The thing we need to be concerned about pretty soon is the truly lifelike AI that is hyperspeed and hyper connected. "Thinking" X times faster than any human,

> The thing we need to be concerned about pretty soon is the truly lifelike AI that is hyperspeed and hyper connected. "Thinking" X times faster than any human, instantly exchanging info with a swarm of peers. I've gone from doomerish on AGI safety to slightly more optimistic for two reasons. There's only so much you can do with this hyperspeed. Most intellectual progress requires interaction with the physical world, which entails time constraints on the level of humans. The second reason is that I believe AGI will asymptote to the level of intelligence present in its corpus of training data. AGI isn't going to become much better at writing or programming than the best humans because it doesn't have examples to go off. Unlike with board games, this kind of data can't be easily generated. So AGI will hit a wall at a certain level of competence within the realm of human intelligence. After hitting that wall progress will become more incremental. Hopefully.