3 ms·

He had already disappointed me in 2011 with this post https://www.antipope.org/charlie/blog-static/2011/06/reality-check-1.html https://www.antipope.org/charli

• amanaplanacanal 3mo ago
  Why is that disappointing? Everything he said there seems perfectly reasonable.
  • ben_w 3mo ago
    A large part of that goes via arguing about consciousness. Starts of with a quite defensible: First: super-intelligent AI is unlikely because, if you pursue Vernor's program, you get there incrementally by way of human-equivalent AI, and human-equivalent AI is unlikely. But the moment he hits that (parenthetical) paragraph about consciousness, he blends human capabilities with human-like consciousness. If human-like capabilities required human-like consciousness, early models (like GPT-4) passing all those exams would suggest those models being "as conscious as" a university student. (My inner Douglas Adams wants to invert this, but the best I can do is "The Xarloxians did not consider university students to be conscious beings; their reasoning being…", which you may recognise as a riff on Babel fish and the non-existence of God).