2 ms·
The problem is that unlike with "regular" engineering, there are mathematical theorems that basically say that writing bug-free code is somewhere between prohib
by pron 11y ago
The problem is that unlike with "regular" engineering, there are mathematical theorems that basically say that writing bug-free code is somewhere between prohibitively expensive and impossible. The cost of proving a non-trivial program property has a proven lower bound of O(S) where S is the number of states the program may take; alternatively, we say that proving correctness is at least PSPACE-complete (if not EXPTIME) in the size of the program. Not saying we can't do our best and use various tools, but this cost cannot -- in general -- be reduced to anything acceptable, and therefore we will never be shocked when a bug does occur.
It is easier to achieve string AI than to write bug-free code, and even that won't help: a super-intelligence will still have bugs in the code it writes.