5 ms·

> If a test suite covers all aspects of an implementation, it's basically a second implementation, and then the question becomes whether the time spent writing

• cLeEOGPw 10y ago
  And it if I understand OpenBSD goal correctly, they take any kind of edge they can get to find those missed parts. Their formula for what is worth essentially boils down to "if this approach can find at least one bug no other approach would do it, we will use it".
• runeks 10y ago
  > The OpenBSD code has been audited many times by very experienced and careful C experts. Fuzzing apparently found the very few things they hadn't already found by reading the code. That's definitely comforting, but I guess my point is that the fact that bugs can even hide in plain sight in the first place is the root cause of the problem. If we can look at a specification of what something is supposed to do - the code - and not see that it does something entirely different, then the problem is with the programming/specification language. The very purpose of a programming language is to make the specification of computer programs readable (rather than reading assembly). If a specification language needs a a computer program to go through the specification and see if it specifies things that we didn't intend to, then that specification language misses the very point of being a language in which to specify things in the first place.
  • et1337 10y ago
    I don't think it's that simple. A lot of bugs are the result of disparate pieces of code -- each of which is fairly easy to verify by reading it -- interacting in an unforeseen way.
    • runeks 10y ago
      I would argue that this effect is not a product of code but of global state. If you have a pure function that transforms some input to an output, all you can do wrong is use the wrong tool for the wrong job, which the type system should, preferably, prevent you from doing. Using the wrong function for the wrong job means we haven't restricted well enough the input the function takes, through clearly defining the types of the arguments, not allowing for meaningless invariants. If a specification language requires the writer/reader to be aware of the entire specification in order to verify whether an isolated piece of specification is valid, that specification language is not of much value, I would argue. It occurs to me that the problem with traditional languages is that they do not allow the direct reference to information (values); only to variables which contain values (information). In Haskell, if you want a place to store information you create a TVar (variable), while you are forced to do this, always, in traditional languages, if you want to work with information/values properly. You're forced to keep information in a register if you want to manipulate it, and always refer to the register, when the information it contains is what you're really concerned with. Having to both think about the values/information and where you stored this information is double work. Why not reference the information directly?
      • et1337 10y ago
        I agree that many bugs occur due to state. I disagree that languages, particularly imperative languages (which you seem to be attacking here), are significantly responsible for bugs. Syntax is not really an issue for any programmer with a reasonable amount of experience. I very rarely have trouble accurately transcribing ideas to code. More often, my ideas are faulty. Physics engines in games are a perfect example. They're a perfect use case for functional programming, there's not a lot of complex state ideally, and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands of YouTube videos of physics glitches in games.
        • runeks 10y ago
          > Syntax is not really an issue for any programmer with a reasonable amount of experience. > They're a perfect use case for functional programming, there's not a lot of complex state ideally, > and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands > of YouTube videos of physics glitches in games. I'm not sure I'm following. Are you arguing that because physics engines are a perfect use case for functional programming, that they are implemented as pure functions? I don't know of any physics engines that are implemented this way, hence the buggyness. If an implementations of 3D scene rasterization contains glitches, the language used is most definitely the issue, because producing glitch-free 3D animations is a solved problem. No programmer writes a physics engine to produce glitches, so if they magically appear, even though they aren't visible in the spec, the spec language is faulty. I guess I'm arguing that the problem is that most popular languages allow you to write programs/specs that are invalid (will crash when executed). So we're writing specifications in languages where we can't even say whether the specification is valid or not (actually implementable). If humans produce buggy programs, that is proof that we're using the wrong tool, I'm arguing, because no one intends to produce bugs. Haskell is a bit scary, at least it was for me in the beginning, because it can seem so hard just to get your program to compile. But that's because the compiler actually verifies that your program is valid - that no unhandled exception will occur. If people are having writing valid, purely functional programs, perhaps this is, in part, because their idea is unimplementable, and Haskell is telling them this via not being able to compile your spec, while other languages inform you of this via an unhandled exception that crops up a year later, after the code is in production.