4 ms·

Startup founder to VC: We're going to build an AI that can do X! 1 billion dollars later: We discovered we can't do X because there's this thing called "undeci

Startup founder to VC: We're going to build an AI that can do X! 1 billion dollars later: We discovered we can't do X because there's this thing called "undecidability." Undecidability puts fundamental limits on what we can do with computers. You can't even do simple things like write a program which can { determine if two programs are equal, determine if two programs would have equal outputs given any input, if any program will return a "1", if any program would halt, ... } This feels kinda important considering what people might try to build using AI.

haltist 3y ago
For practical purposes it doesn't really matter because most real world computation is an approximation anyway. AI researchers don't care much about impossibility results because their goal is human level artificial intelligence and they consider people to be existence proofs that such computation is possible to engineer artificially. Basically, AI researchers believe that people are just walking computers/robots so they're not at all concerned about theoretical obstructions to constructing generally intelligent software systems.
- haltist 3y ago
  Something else to consider is that if it is possible to truly engineer human level intelligence for operating autonomous machines then ethical concerns would require treating such systems with full human rights. Some are already claiming the ethical thresholds for human rights for AI have already been passed in internal research projects at Google.[1] 1: https://www.bbc.com/news/technology-62275326 https://www.bbc.com/news/technology-62275326
cochne 3y ago
You can certainly write a program which can determine if a program would halt. It just won't be able to output a yes or no for _every_ program. In some cases it would have to output 'undecidable'. Most of the practical programs you might want to feed into this program will have an answer.
- trehalose 3y ago
  If I may nitpick, it shouldn't ever output "undecidable" but rather something like "undecided" or "I give up". A program that can be trivially seen to halt if we let it run for a quadrillion years is only "undecidable" in the sense that we might get impatient and die.

haltist 3y ago

For practical purposes it doesn't really matter because most real world computation is an approximation anyway. AI researchers don't care much about impossibility results because their goal is human level artificial intelligence and they consider people to be existence proofs that such computation is possible to engineer artificially. Basically, AI researchers believe that people are just walking computers/robots so they're not at all concerned about theoretical obstructions to constructing generally intelligent software systems.

haltist 3y ago
Something else to consider is that if it is possible to truly engineer human level intelligence for operating autonomous machines then ethical concerns would require treating such systems with full human rights. Some are already claiming the ethical thresholds for human rights for AI have already been passed in internal research projects at Google.[1] 1: https://www.bbc.com/news/technology-62275326 https://www.bbc.com/news/technology-62275326

cochne 3y ago

You can certainly write a program which can determine if a program would halt. It just won't be able to output a yes or no for _every_ program. In some cases it would have to output 'undecidable'. Most of the practical programs you might want to feed into this program will have an answer.

trehalose 3y ago
If I may nitpick, it shouldn't ever output "undecidable" but rather something like "undecided" or "I give up". A program that can be trivially seen to halt if we let it run for a quadrillion years is only "undecidable" in the sense that we might get impatient and die.