The open letter is so foolish it invalidates itself. For one thing, it's pouring fuel on the fire. The obvious reaction to such a letter is to accelerate and

The open letter is so foolish it invalidates itself. For one thing, it's pouring fuel on the fire. The obvious reaction to such a letter is to accelerate and hide one's own progress, eh? Yudkowsky's fear-mongering about sci-fi Skynet is also unmoving to me (for metaphysical reasons I won't go into here) however, his position is at least logically consistent, as Aaronson points out. These machines can already talk and think better than many humans, and the ratio (of mentally sub-computer to super-computer humans) will only go up. Ergo, the only intellectual problem left is how to use them. I've been banging on about this for a couple of weeks now, so apologies to those who've seen it before, please read "Augmenting Human Intellect: A Conceptual Framework" SRI Summary Report AFOSR-3223 by Douglas C. Engelbart, October 1962 https://dougengelbart.org/pubs/augment-3906.html https://dougengelbart.org/pubs/augment-3906.html And then at least skim "An Introduction to Cybernetics" by W. Ross Ashby. http://pespmc1.vub.ac.be/books/IntroCyb.pdf http://pespmc1.vub.ac.be/books/IntroCyb.pdf The thing you're looking for is "intelligence amplifier". Engelbart references it, the book explains all the math you need to understand and build one. The next piece of the puzzle is something called "Neurolinguistic Programming" (you may have heard of it, some people will tell you it's pseudoscience, ignore those people, they mean well but they just don't know what they're talking about.) It turns out there's a rigorous repeatable psychology that more-or-less works. It's been under development for about half a century. The particular thing to look at is a specific algorithm called "Core Transformation Process" ( https://www.coretransformation.org/ https://www.coretransformation.org/ ) which is kind of like the "Five Why's" technique but for the mind. It functions to reduce yak-shaving and harmonize intent. Anyway, the upshot of it all is that these machines can be made into automatic perfect therapists, using off-the-shelf technology.