3 ms·
That was better said in some ways than my own comment.
by api 1y ago
That was better said in some ways than my own comment.
- mrandish 1y agoThanks! And I just saw in a parallel post currently on the HN home page, John Carmack also said a similar thing in his lecture notes. > "offline training can bootstrap itself off into a coherent fantasy untested by reality." "a coherent fantasy untested by reality" is a lovely turn of phrase.
- api 1y agoThe Yudkowskiites are all about coherent fantasies untested by reality, as are… really… a lot of philosophers throughout history. Maybe most. I fundamentally do not believe in knowing without sensing or learning without experiencing. Of course it need not be direct experience. You can “download” information. But that information had to be gathered somehow at some point. There is only so much training data. As I said — I don’t dismiss the idea of AI challenging human intellect or replacing human jobs. An AI “merely” as smart as a human but tireless and faster could seem superhuman. I am just intensely skeptical of the idea of something learning to self improve and then magically taking off into some godlike superintelligence realm far beyond what is latent in or implied by its training data. That would be an informatic perpetual motion machine. It would, in fact, be magic, in the fantasy sense.