3 ms·
>> The same logic means that people shouldn't use static code analysis tools, or valgrind, or even debuggers, until they acquire deep technical knowledge. While
by dsacco 12y ago
>> The same logic means that people shouldn't use static code analysis tools, or valgrind, or even debuggers, until they acquire deep technical knowledge. While I think all of these tools help reinforce the principles.
This probably sounds controversial, but I agree. I don't think you should use Valgrind until you understand how Valgrind works. This doesn't mean early C programmers shouldn't use Valgrind - you can read the documentation and theory behind Valgrind in a day. But definitely do that. I think maybe "deep technical knowledge" wasn't the right term for me to use. A better term would be "technical understanding" - know how it works, and know how to find the different classes of bugs it can find, but you do not need to be capable of writing the tool yourself.
Now let me clarify this, and my earlier point about afl - I think you should use them, and generously, and pretty much always once you know what you're doing. But if you use them without understanding the fundamentals, you will get caught up in false positives/negatives. Always use it as a supplement, not a crutch.
However, I agree with what you're saying about someone's level of dedication to security analysis. Using afl is better than not using afl, so if you're not a dedicated security guy, then you're definitely right that someone should use it.
I would still caution anyone that using an automated tool without fully understanding how it works will lead to an incomplete picture of the application's security posture.
- dalke 12y agoWell in that case, I have only a surface level understanding of Valgrind, but I use it for those rare times I code in C. I also don't know how afl works at the code modification level, other than that it asks gcc to add some instrumentation, but I've used it to find bugs. :) I completely agree with your last statement, with one proviso. You will always have an incomplete picture of the application's security posture. The fundamentals include hardware and OS behaviors, power consumption, RF emissions, and other fields which are very far from purity of the programming language model. Acquisition of all of this knowledge is very difficult, but is also rarely needed. I think for most cases it's okay to have an incomplete understanding of the tools. It depends on the threat model.
- tptacek 12y agoThis idea of people not using tools until they understand how they work (or in the Matasano shorthand, until they can implement them, in "build your own light saber" fashion) is I think very true of professional testers, but not true for developers. If your full-time job is going to be about developing a radar for assumptions that developers make and a deviousness about breaking those assumptions, tools --- and most especially opaque tools --- totally hamstring you. We both know plenty of testers that develop a kind of blindness from relying on tools too much. But I'm guessing most of HN doesn't really understand the mindset that's involved in pentesting full-time. I certainly wouldn't tell a developer that they should avoid valgrind.