Yeah, that's the third response to cite the halting problem. I think that's a pretty good response. The guarantee that it is not an output from possible progr

Yeah, that's the third response to cite the halting problem. I think that's a pretty good response. The guarantee that it is not an output from possible programs part is an assumption. Actually, it isn't an assumption. I've constructed the requested response such that it can't be a response from any of the programs on the list. It's like the Cantor diagonalization argument. Since it is also an assumption to take for granted that even though the possible programs is countable it is a practically enumerable finity. Not really. If I take the mapping to be the trivial one -- any number maps to the program the binary representation of that number would be in machine code -- then getting a runnable program from a number in input is pretty trivial. Of course, most of them would do nothing but crash. It is also an assumption that there is no 1:1 mapping from the list of programs to something that could perfectly emulate a human. I neither assume, affirm, attempt to prove, or attempt to deny that. I haven't said anything about programs that can emulate a human other than that there can't be any. ;) It is also does not follow that just because the AI could theoretically be stymied by this conundrum, the set of its computations would not include those that utterly outperformed humanity in all their cognitive useful tasks. Yes, I suppose not. But Sal can't comprehend and execute such a simple request, I don't really have much hope for her ability to do things like advance set theory. ;) It is also an assumption to believe that a human could not just construct such a program but construct one that gave an output Well, actually, I just asked them to give a response different than what the program gave. I would expect most humans, after a few seconds, to say, "It seems to be hanging. I have better things to do." Which is probably not what the AI was going to say. Or if I really want to go down that path, a human can talk long enough that its response is vanishingly unlikely to be what the program was going to say. But I think the halting really is a pretty good objection. Especially because the problematic case -- feeding the program its own source code -- really would hang on infinite recursion. I'm still thinking about that. But I do have this to say: I wouldn't consider hanging on a simple question to be a very human response. It's definitely something done in science fiction to identify the AIs. ;)