4 ms·

Can you give an example of how the halting problem allows one to make it impossible/difficult to detect malicious code? I sort of get what you are saying, that

Can you give an example of how the halting problem allows one to make it impossible/difficult to detect malicious code? I sort of get what you are saying, that Turing completeness makes analysis more difficult, but I still don't see what it has to do with security.

yk 11y ago
Well, if you can somehow inject arbitrary code, then you can inject an infinite loop. So a program will only halt on all inputs if you can not inject an infinite loop.
- radarsat1 11y ago
  Sure.. so how is an infinite loop a security vulnerability?
  - JadeNB 11y ago
    If you are performing data processing on user-supplied inputs (as online compilers do), then an infinite loop can amount to a DoS attack.
  - yk 11y ago
    The argument is the other way around, if a program halts on all inputs, then there is no possibility to inject arbitrary code. ( There may still be important security vulnerabilities, but there are some limits. Consider a debugger that executes the first 10 000 operations of a program, it would halt, but 10 000 operations is a lot of space to do something malicious.)
  - coldtea 11y ago
    Security doesn't end with "privilege escalation". "Killing" (DoS) a system is also an attack.
yaps8 11y ago
The halting problem makes it impossible to consistently answer the question "Will this instruction be executed ?" This is why you can't know if the code is malicious : you can't (always) infer meaningful properties about the code (Will it write something on the disk ?...).
- JadeNB 11y ago
  > The halting problem makes it impossible to consistently answer the question "Will this instruction be executed ?" In fact, Rice's theorem says that it's impossible more generally to answer any question about arbitrary code—so no work-arounds like "OK, I can't tell if a particular instruction is executed, but I can just test whether this program is 'safe'" (say, performs no I/O).
  - 9mit3t2m9h9a 11y ago
    And the reason why you cannot check that the program performs no I/O is because you cannot check if the I/O instructions are behind an infinite loop or an impossible condition or not. It is definitely easy to verify that a program has no references to I/O instructions. This will reject programs like «if(false) then write("Hello")», so it is not a violation of Rice theorem (functionally equivalent programs are not treated in the same way). But you don't need to accept everything that is actually benign, you just need to make it easy to write benign code that passes the check.
  - chriswarbo 11y ago
    Technically, Rice's theorem only applies to "non-trivial" questions, i.e. those which are true for some programs and false for others, which are properties of the function being computed. Hence we can work around the theorem, by either turning the questions we care about into trivial questions, or by turning questions of the function into questions about the implementation. For example, we might ask the question "Will program P fire the missiles?". If the answer is yes for some P and false for others, then we cannot answer it for arbitrary P. We can work around it by weakening the question to become trivial, e.g. "Is program P capable of firing missiles?". If the language allows missiles to be fired, then the answer is trivially yes for all P; if not then the answer is trivially no. Alternatively, we can change the question to one of implementation, e.g. "Does program P contain the <fire missiles> instruction?". Assuming that our programming language contains such an instruction, this is a non-trivial question. However, since it's a question about the implementation (does that instruction appear) rather than the function being computed (which may or may not fire missiles), Rice's theorem doesn't forbid us from answering it.
logicrook 11y ago
This is more from the perspective of the language designer, not from a "you get a binary, is it malware" one, in which case Turing-completeness basically has no importance. So for an example... Many exploits use stackoverflows to get the execution of arbitrary code. You can prove that in some bounded languages such things cannot happen, regardless of the inputs the program gets. Obviously you still rely on assumptions such as "the OS is not going to do random things with the memory", but that's outside of your power anyway.
coldtea 11y ago
>I sort of get what you are saying, that Turing completeness makes analysis more difficult, but I still don't see what it has to do with security. If analysis was easy you could prove (as in mathematically, e.g. with type theory and such) that the program could never get in certain insecure states. In fact, lots of highly secure environments (NASA etc), require just such proofs from certain programming modules in order to use them.
- adrianN 11y ago
  Turing completeness of the language is only a problem if you don't want to reject safe programs. If you're fine with false positives you can just check (using a regex!) that no \write happens. Static analysis (not using regexes) is a thing for Turing complete languages and can prove non-trivial things.
robert_tweed 11y ago
A turing complete program could, by definition, generate any other turing complete program. In theory, you can statically analyse any program to prove that it doesn't contain malicious code (for some definition of malicious). But if a program is turing complete, you cannot predict all of its possible outputs and therefore, cannot rule out the possibility of a malicious output. The halting problem is what prevents you doing the analysis, not what introduces the possibility of maliciousness.

yk 11y ago

Well, if you can somehow inject arbitrary code, then you can inject an infinite loop. So a program will only halt on all inputs if you can not inject an infinite loop.

radarsat1 11y ago
Sure.. so how is an infinite loop a security vulnerability?
- JadeNB 11y ago
  If you are performing data processing on user-supplied inputs (as online compilers do), then an infinite loop can amount to a DoS attack.
- yk 11y ago
  The argument is the other way around, if a program halts on all inputs, then there is no possibility to inject arbitrary code. ( There may still be important security vulnerabilities, but there are some limits. Consider a debugger that executes the first 10 000 operations of a program, it would halt, but 10 000 operations is a lot of space to do something malicious.)
- coldtea 11y ago
  Security doesn't end with "privilege escalation". "Killing" (DoS) a system is also an attack.

yaps8 11y ago

The halting problem makes it impossible to consistently answer the question "Will this instruction be executed ?" This is why you can't know if the code is malicious : you can't (always) infer meaningful properties about the code (Will it write something on the disk ?...).

JadeNB 11y ago
> The halting problem makes it impossible to consistently answer the question "Will this instruction be executed ?" In fact, Rice's theorem says that it's impossible more generally to answer any question about arbitrary code—so no work-arounds like "OK, I can't tell if a particular instruction is executed, but I can just test whether this program is 'safe'" (say, performs no I/O).
- 9mit3t2m9h9a 11y ago
  And the reason why you cannot check that the program performs no I/O is because you cannot check if the I/O instructions are behind an infinite loop or an impossible condition or not. It is definitely easy to verify that a program has no references to I/O instructions. This will reject programs like «if(false) then write("Hello")», so it is not a violation of Rice theorem (functionally equivalent programs are not treated in the same way). But you don't need to accept everything that is actually benign, you just need to make it easy to write benign code that passes the check.
- chriswarbo 11y ago
  Technically, Rice's theorem only applies to "non-trivial" questions, i.e. those which are true for some programs and false for others, which are properties of the function being computed. Hence we can work around the theorem, by either turning the questions we care about into trivial questions, or by turning questions of the function into questions about the implementation. For example, we might ask the question "Will program P fire the missiles?". If the answer is yes for some P and false for others, then we cannot answer it for arbitrary P. We can work around it by weakening the question to become trivial, e.g. "Is program P capable of firing missiles?". If the language allows missiles to be fired, then the answer is trivially yes for all P; if not then the answer is trivially no. Alternatively, we can change the question to one of implementation, e.g. "Does program P contain the <fire missiles> instruction?". Assuming that our programming language contains such an instruction, this is a non-trivial question. However, since it's a question about the implementation (does that instruction appear) rather than the function being computed (which may or may not fire missiles), Rice's theorem doesn't forbid us from answering it.

logicrook 11y ago

This is more from the perspective of the language designer, not from a "you get a binary, is it malware" one, in which case Turing-completeness basically has no importance. So for an example... Many exploits use stackoverflows to get the execution of arbitrary code. You can prove that in some bounded languages such things cannot happen, regardless of the inputs the program gets. Obviously you still rely on assumptions such as "the OS is not going to do random things with the memory", but that's outside of your power anyway.

coldtea 11y ago

>I sort of get what you are saying, that Turing completeness makes analysis more difficult, but I still don't see what it has to do with security. If analysis was easy you could prove (as in mathematically, e.g. with type theory and such) that the program could never get in certain insecure states. In fact, lots of highly secure environments (NASA etc), require just such proofs from certain programming modules in order to use them.

adrianN 11y ago
Turing completeness of the language is only a problem if you don't want to reject safe programs. If you're fine with false positives you can just check (using a regex!) that no \write happens. Static analysis (not using regexes) is a thing for Turing complete languages and can prove non-trivial things.

robert_tweed 11y ago

A turing complete program could, by definition, generate any other turing complete program. In theory, you can statically analyse any program to prove that it doesn't contain malicious code (for some definition of malicious). But if a program is turing complete, you cannot predict all of its possible outputs and therefore, cannot rule out the possibility of a malicious output. The halting problem is what prevents you doing the analysis, not what introduces the possibility of maliciousness.