6 ms·

The Dark Path

21 points by runesoerensen 10y ago

AnimalMuppet 10y ago
Every check that you can add to a language means ways that the programmer cannot mess up. That's a gain. It also means things that programmers cannot do. That's a loss. For some restrictions, the gains outweigh the losses. But that won't be true for every restriction; kitchen knives are sharp for a reason. Restricting everything so that nobody can do anything wrong is unworkable; it is, if not the dark path, at least a dark path. But so is relying solely on testing for safety. There's always a path through the code that you didn't test. Even if your tests execute every path through every function, there's always a combination of paths that wasn't, or an input value that wasn't, or, or or. You cannot achieve quality solely by tests. You need a balance. Use language features that prevent errors - not that prevent every error, but at least that prevent some. Use good design. Use static code analysis. Use thorough testing. It all matters; no one thing will save you.
- kazinator 10y ago
  A static check adds a way in which the programmer can mess up. A code path which fails a check was obviously not tested (at least, not since the change was introduced which triggers the check). The programmer can massage the code so the check goes away, yet leave the code path still untested, due to the "safyness" feeling that that the machine "got his/her back", so all is well.
  - AnimalMuppet 10y ago
    > A static check adds a way in which the programmer can mess up. True, though it is one that you can ignore. You can't ignore the compiler. Reading through the output of a static checker is 95% "Stupid hyper-picky static checker", and 5% "Oh #&%$@, we'd better fix that right away". Ignore the ones that should be ignored. > A code path which fails a check was obviously not tested (at least, not since the change was introduced which triggers the check). Or not tested with the right combination of inputs, or not tested looking for the right thing... > The programmer can massage the code so the check goes away, yet leave the code path still untested, due to the "safyness" feeling that that the machine "got his/her back", so all is well. Sure, they can, if they aren't trying to understand what the checker is telling them, and instead just try to get it to shut up. And they can do the same with a test.
    - kazinator 10y ago
      > Or not tested with the right combination of inputs, or not tested looking for the right thing... There's the thing. Unless the code is exploiting dynamism, probing large input spaces is not required to catch the same issues. Just enough variety to achieve path coverage. We don't need to probe the space of integers to catch that a string operation is applied; running that expression with any value will do. I added a static check for uses of undefined functions and variables to a dynamic language quite recently. It didn't find anything in any of my application code. One instance was found in the language's standard library. The function which had that error was never tested. Calling that function with any argument at all triggered the dynamic check. The static check only pointed out this fact: hey, you didn't test this! That's a poor substitute for an actual path coverage tool. You can develop untested code which passes various static checks; only a coverage tool will show you the untested paths.
      - Jweb_Guru 10y ago
        A coverage tool will not come remotely close to showing you untested paths. Array indexing, for example, is an implicit branch between every possible element in the array, but most coverage tools I've seen count just evaluating the indexing expression as covering the line. On the other hand, judicious use of indexed data types can guarantee a data path for all indices into the array. Simple sum types like Maybe and Either guarantee a data path for the more simple use cases of null or errors. Uses of undefined functions and variables are not what people are talking about here (though those do happen in dynamic languages); we're talking about stuff that can't be checked locally (like "is a null allowed to come into the function at all, and if it does are you covering it") that are hard or impossible to catch with coverage tools.
        kazinator 10y ago
        If for some crazy reason we choose to regard an array access as if it were a big switch or case statement with a branch for each index, we are wrong in concluding that this creates a path coverage problem. These cases are machine-generated and do not require separate coverage treatment. An analogy might be found in, say, machine-unrolled loops. I don't care about coverage of multiple copies of a loop body caused by a compiler optimization.
seertaak 10y ago
I think this an excellent article, and mostly agree with the author's somewhat controversial thesis. Broadly speaking, I have noticed that language features fall into two categories. The first gives you some kind of new ability. For example, late binding allows polymorphism. On the other hand, the second category entails deliberate restrictions and hurdles imposed upon the programmer with the putative goal of reducing errors. An example of this category is the checked exception mechanism of e.g. Java. For reasons I don't deign to fully understand, many programmers seem to be rather attracted to the features from the second category, whereas I myself am more attracted to features in the first category. Languages with a strong emphasis on the second category include Haskell and Rust. My criticism of those languages is that they seem to neglect that language design almost always involves tradeoffs. This is because there exists a complexity budget -- related to the our ability to hold multiple pieces of information in our very imperfect brains. During my forays into Haskell -- which I tested on simple scripts in the hedge fund I was working -- I always was struck by the beauty and power of the language. That said, it was hard not to notice how concept-heavy even simple scripts were; scripts that would be absolutely trivial in other languages now involved tortuous use of monads within monads. C++, the language I use for my current app (https://zenaud.io https://zenaud.io), is a bit of a Jekyll and Hyde language in this sense. In my opinion, it is telling that the category 1 feature that is templates was wildly successful. By contrast, the category 2 feature which is concepts appears to be struggling (although in fairness, the jury is still out [and anyone who has read the Design and Evolution of C++ knows that it's a brave man who bets against the language-design savvy of Bjarne Stroustroup]). Last point: Rust seems to me a category 2 heavy language. For this reason, I'm steering clear. What I really want is C with templates, destructors, late-binding (but with value types a la Sean Parent), and operators, and very little of the rest of the crap you get in C++.
- AnimalMuppet 10y ago
  > What I really want is C with templates, destructors, late-binding (but with value types a la Sean Parent), and operators, and very little of the rest of the crap you get in C++. Then just use those parts of C++. It's explicitly intended that you can use just the parts you want. (Until you have to interface with someone else's code...)
- 10y ago
qubyte 10y ago
I like the emphasis this article places on testing. Languages which emphasise testing or make testing easy are my favourites to code in. For example, JS being dynamic and having mature testing and assertion libraries lowers the barrier to getting stuff tested, Go has testing in its standard library, and Rust has amazing support for tests (including testing examples in comments) both in the language and from Cargo. Perhaps it is more important to provide tooling and libraries for testing a language than it is to provide new features in the language proper to avoid foot-guns. It's much harder to back out of the latter.
gumby 10y ago
I like the sentiment but don't at all agree with the conclusion. > Why are these languages adopting all these features? Because programmers are not testing their code. I have a more uncharitable view. Most programmers don't really understand their tools and don't really take a lot of time to think about what they are doing (to be fair, management, especially in enterprise environments, discourage programmers from thinking about what they are doing). Languages like Swift and Go are designed to constrain the programmer to make it harder for them to make mistakes. The strong type systems also make it easier for programmers to provide information to the compilers to help them generate better code. Testing happens so late in the process I don't think it has anything to do with it. There are powerful grownup tools like Lisp and C++ which are paradigm-neutral, but which also hand you enough rope to asphyxiate yourself before you can even get as far as hanging yourself.
- kazinator 10y ago
  > Testing happens so late in the process I don't think it has anything to do with it. Most testing happens as late as possible, in fact. Which is to say, never.
DannyB2 10y ago
The article asserts that authors are not testing their code. The problem is that you can't ensure that every possible bug is exercised in tests. Think of the static typing of the compiler as the very first level of testing. If you can have an NPE at this line of code, then it won't compile. The programmer is forced to deal with it. It has failed the first round of 'testing' done by the compiler. Who's job is it, the programmer or the language? In this case, it is the programmer, who is being forced to fix it by the language. We don't want code with bugs. Don't 'fix' it in testing when it is way cheaper to put that null test into the code when it is written. The compiler can absolutely ensure that no NPEs can be generated. The testing absolutely CAN NOT make this guarantee. There are other languages that allow sloppy programming. Use them if you like. Encourage their use if you like. If there were one perfect programming language then everyone would be using it already.
spion 10y ago
The main argument here apparently is "lets not add safety features because we have a limited pool of features before the language becomes too complex". I don't think thats a convincing argument. Its a question of which features you value. For example, I'd totally give up `public, private and protected` to be able to get rid of exception and `null` pointer problems forever. > It is programmers who create defects – not languages. In the case of nulls in typed languages, its the language's fault. The language is literally lying to you. Lets say that Java is claiming, for example, that a method always returns a `string`. What is a string? Its a value of a certain class, that supports certain methods that return other values. In structurally typed ("duck typed") languages, the type can be defined by the methods you can call on the value (the interface it supports) But this claim is just not true. The method may return null, and if you call the string methods on that value, your code may die a violent death. Its clear that the value `null` doesn't belong to the type `string`, since it doesn't support the contract that a `string` provides. So why is the programmers fault to expect that the type system doesn't lie to them? I think the programmer has the right to expect an actual string - not a number, not an `StringLike`, and definitely not null, all of which don't fully satisfy the `string` contract. Unfortunately, the language doesn't give you the ability to discern between a string that cannot be null and a string-or-null (could be string, but also could be null). Because of this, you need to track all the places that might be null in your head to make sure you check for them, or look up the test code to remind yourself all the time. Its a similar problem with exceptions. The language is claiming that functions that don't cause exceptions act the same way as those that do. But thats painfully untrue, and you would find that out the moment you try to `fopen` a file, read something from it, and then `fclose` it. f = fopen('file.txt') data = fread(f) fclose(f) Oops but the `read` call may throw, and you now have a dangling file handle. Good thing you knew that about `fread`! Now how about the other thousand methods in your code? Do you know whether any of those throw? Whats that? You should be using `try-with-resources`, I hear? Okay, then what do I do with this code: atomicIncrement(inFlightRequests) result = doRequest() atomicDecrement(inFlightRequests) I should use try-finally. Thats true, but what if doRequest() doesn't throw? Then I don't need to. Why do those two look the same? Why is the language lying that they are the same? Now lets see whats available in Swift: * `try!` means - I don't expect this piece of code to throw, even though it "might". If it does, crash horribly. * `try` means - I know that this piece of code throws, but I can't handle that exception here, so propagate it. * no keyword - This code should not throw. If it returns a Result that may be an error, I want to handle it right here. * `try?` similar as above, but discards the specific error and returns an Optional. Thats it. You manage the risk, the language only makes that explicit. It also makes it really clear whats going on during code review. Here is how the problematic piece of code looks when `doRequest` might throw: atomicIncrement(inFlightRequests) result = try doRequest() atomicDecrement(inFlightRequests) The possible bug is now very easy to spot. We don't need to remember whether `doRequest` throws or to look that up. Its clear that a finally or defer block is necessary here. Whats even better is how the code looks like when `doRequest` doesn't throw! atomicIncrement(inFlightRequests) result = doRequest() atomicDecrement(inFlightRequests) Since we know the compiler would force a `try` keyword there if there was the possibility of exception, we know this code is correct just by glancing it. No need for `defer` or `finally`. So are these two features not worth the cost? Maybe, if you are arguing that we don't need types as a tool to prevent errors. In that case, it would indeed be the programmer's fault for not writing a test for it. But if you are going to have types, you might as well have types that tell the truth. Otherwise they're useless. Finally, IMO this is not a good reason to hate on types. Stop complaining that the compiler wont accept your buggy code and take the time to do the changes. I'm happy when the compiler points out all the places that will be affected by the change - thats something that unit tests with mocks will never catch! Imagine that a change in doRequest caused it to throw, where previously it did not. The compiler will now tell you about ALL places that may be affected. What would happen if it didn't? The `inFlightRequests` counter will suddenly start behaving strange and increasing indefinitely. What would happen if above a certain value new requests don't get queued? A bug that leads to denial of service. Is it worth the hassle of changing all affected code? I don't know, what do you think? There are much better complaints against type systems, like the fact that most are not able to understand certain advanced styles of metaprogramming yet. Or that it doesn't do control flow analysis, e.g. if I type if (x != null) { doCode(x); } then `x` should have its nullability removed in the block or if (x == null) { x = nonNullValue; } then below this line, x should not be considered nulable. TypeScript for example does both of the above.

21 points by runesoerensen 10y ago

Every check that you can add to a language means ways that the programmer cannot mess up. That's a gain. It also means things that programmers cannot do. That's a loss. For some restrictions, the gains outweigh the losses. But that won't be true for every restriction; kitchen knives are sharp for a reason. Restricting everything so that nobody can do anything wrong is unworkable; it is, if not the dark path, at least a dark path. But so is relying solely on testing for safety. There's always a path through the code that you didn't test. Even if your tests execute every path through every function, there's always a combination of paths that wasn't, or an input value that wasn't, or, or or. You cannot achieve quality solely by tests. You need a balance. Use language features that prevent errors - not that prevent every error, but at least that prevent some. Use good design. Use static code analysis. Use thorough testing. It all matters; no one thing will save you.

kazinator 10y ago
A static check adds a way in which the programmer can mess up. A code path which fails a check was obviously not tested (at least, not since the change was introduced which triggers the check). The programmer can massage the code so the check goes away, yet leave the code path still untested, due to the "safyness" feeling that that the machine "got his/her back", so all is well.
- AnimalMuppet 10y ago
  > A static check adds a way in which the programmer can mess up. True, though it is one that you can ignore. You can't ignore the compiler. Reading through the output of a static checker is 95% "Stupid hyper-picky static checker", and 5% "Oh #&%$@, we'd better fix that right away". Ignore the ones that should be ignored. > A code path which fails a check was obviously not tested (at least, not since the change was introduced which triggers the check). Or not tested with the right combination of inputs, or not tested looking for the right thing... > The programmer can massage the code so the check goes away, yet leave the code path still untested, due to the "safyness" feeling that that the machine "got his/her back", so all is well. Sure, they can, if they aren't trying to understand what the checker is telling them, and instead just try to get it to shut up. And they can do the same with a test.
  - kazinator 10y ago
    > Or not tested with the right combination of inputs, or not tested looking for the right thing... There's the thing. Unless the code is exploiting dynamism, probing large input spaces is not required to catch the same issues. Just enough variety to achieve path coverage. We don't need to probe the space of integers to catch that a string operation is applied; running that expression with any value will do. I added a static check for uses of undefined functions and variables to a dynamic language quite recently. It didn't find anything in any of my application code. One instance was found in the language's standard library. The function which had that error was never tested. Calling that function with any argument at all triggered the dynamic check. The static check only pointed out this fact: hey, you didn't test this! That's a poor substitute for an actual path coverage tool. You can develop untested code which passes various static checks; only a coverage tool will show you the untested paths.
    - Jweb_Guru 10y ago
      A coverage tool will not come remotely close to showing you untested paths. Array indexing, for example, is an implicit branch between every possible element in the array, but most coverage tools I've seen count just evaluating the indexing expression as covering the line. On the other hand, judicious use of indexed data types can guarantee a data path for all indices into the array. Simple sum types like Maybe and Either guarantee a data path for the more simple use cases of null or errors. Uses of undefined functions and variables are not what people are talking about here (though those do happen in dynamic languages); we're talking about stuff that can't be checked locally (like "is a null allowed to come into the function at all, and if it does are you covering it") that are hard or impossible to catch with coverage tools.
      - kazinator 10y ago
        If for some crazy reason we choose to regard an array access as if it were a big switch or case statement with a branch for each index, we are wrong in concluding that this creates a path coverage problem. These cases are machine-generated and do not require separate coverage treatment. An analogy might be found in, say, machine-unrolled loops. I don't care about coverage of multiple copies of a loop body caused by a compiler optimization.

seertaak 10y ago

I think this an excellent article, and mostly agree with the author's somewhat controversial thesis. Broadly speaking, I have noticed that language features fall into two categories. The first gives you some kind of new ability. For example, late binding allows polymorphism. On the other hand, the second category entails deliberate restrictions and hurdles imposed upon the programmer with the putative goal of reducing errors. An example of this category is the checked exception mechanism of e.g. Java. For reasons I don't deign to fully understand, many programmers seem to be rather attracted to the features from the second category, whereas I myself am more attracted to features in the first category. Languages with a strong emphasis on the second category include Haskell and Rust. My criticism of those languages is that they seem to neglect that language design almost always involves tradeoffs. This is because there exists a complexity budget -- related to the our ability to hold multiple pieces of information in our very imperfect brains. During my forays into Haskell -- which I tested on simple scripts in the hedge fund I was working -- I always was struck by the beauty and power of the language. That said, it was hard not to notice how concept-heavy even simple scripts were; scripts that would be absolutely trivial in other languages now involved tortuous use of monads within monads. C++, the language I use for my current app (https://zenaud.io https://zenaud.io), is a bit of a Jekyll and Hyde language in this sense. In my opinion, it is telling that the category 1 feature that is templates was wildly successful. By contrast, the category 2 feature which is concepts appears to be struggling (although in fairness, the jury is still out [and anyone who has read the Design and Evolution of C++ knows that it's a brave man who bets against the language-design savvy of Bjarne Stroustroup]). Last point: Rust seems to me a category 2 heavy language. For this reason, I'm steering clear. What I really want is C with templates, destructors, late-binding (but with value types a la Sean Parent), and operators, and very little of the rest of the crap you get in C++.

AnimalMuppet 10y ago
> What I really want is C with templates, destructors, late-binding (but with value types a la Sean Parent), and operators, and very little of the rest of the crap you get in C++. Then just use those parts of C++. It's explicitly intended that you can use just the parts you want. (Until you have to interface with someone else's code...)
10y ago

qubyte 10y ago

I like the emphasis this article places on testing. Languages which emphasise testing or make testing easy are my favourites to code in. For example, JS being dynamic and having mature testing and assertion libraries lowers the barrier to getting stuff tested, Go has testing in its standard library, and Rust has amazing support for tests (including testing examples in comments) both in the language and from Cargo. Perhaps it is more important to provide tooling and libraries for testing a language than it is to provide new features in the language proper to avoid foot-guns. It's much harder to back out of the latter.

gumby 10y ago

I like the sentiment but don't at all agree with the conclusion. > Why are these languages adopting all these features? Because programmers are not testing their code. I have a more uncharitable view. Most programmers don't really understand their tools and don't really take a lot of time to think about what they are doing (to be fair, management, especially in enterprise environments, discourage programmers from thinking about what they are doing). Languages like Swift and Go are designed to constrain the programmer to make it harder for them to make mistakes. The strong type systems also make it easier for programmers to provide information to the compilers to help them generate better code. Testing happens so late in the process I don't think it has anything to do with it. There are powerful grownup tools like Lisp and C++ which are paradigm-neutral, but which also hand you enough rope to asphyxiate yourself before you can even get as far as hanging yourself.

kazinator 10y ago
> Testing happens so late in the process I don't think it has anything to do with it. Most testing happens as late as possible, in fact. Which is to say, never.

DannyB2 10y ago

The article asserts that authors are not testing their code. The problem is that you can't ensure that every possible bug is exercised in tests. Think of the static typing of the compiler as the very first level of testing. If you can have an NPE at this line of code, then it won't compile. The programmer is forced to deal with it. It has failed the first round of 'testing' done by the compiler. Who's job is it, the programmer or the language? In this case, it is the programmer, who is being forced to fix it by the language. We don't want code with bugs. Don't 'fix' it in testing when it is way cheaper to put that null test into the code when it is written. The compiler can absolutely ensure that no NPEs can be generated. The testing absolutely CAN NOT make this guarantee. There are other languages that allow sloppy programming. Use them if you like. Encourage their use if you like. If there were one perfect programming language then everyone would be using it already.

spion 10y ago

The main argument here apparently is "lets not add safety features because we have a limited pool of features before the language becomes too complex". I don't think thats a convincing argument. Its a question of which features you value. For example, I'd totally give up `public, private and protected` to be able to get rid of exception and `null` pointer problems forever. > It is programmers who create defects – not languages. In the case of nulls in typed languages, its the language's fault. The language is literally lying to you. Lets say that Java is claiming, for example, that a method always returns a `string`. What is a string? Its a value of a certain class, that supports certain methods that return other values. In structurally typed ("duck typed") languages, the type can be defined by the methods you can call on the value (the interface it supports) But this claim is just not true. The method may return null, and if you call the string methods on that value, your code may die a violent death. Its clear that the value `null` doesn't belong to the type `string`, since it doesn't support the contract that a `string` provides. So why is the programmers fault to expect that the type system doesn't lie to them? I think the programmer has the right to expect an actual string - not a number, not an `StringLike`, and definitely not null, all of which don't fully satisfy the `string` contract. Unfortunately, the language doesn't give you the ability to discern between a string that cannot be null and a string-or-null (could be string, but also could be null). Because of this, you need to track all the places that might be null in your head to make sure you check for them, or look up the test code to remind yourself all the time. Its a similar problem with exceptions. The language is claiming that functions that don't cause exceptions act the same way as those that do. But thats painfully untrue, and you would find that out the moment you try to `fopen` a file, read something from it, and then `fclose` it. f = fopen('file.txt') data = fread(f) fclose(f) Oops but the `read` call may throw, and you now have a dangling file handle. Good thing you knew that about `fread`! Now how about the other thousand methods in your code? Do you know whether any of those throw? Whats that? You should be using `try-with-resources`, I hear? Okay, then what do I do with this code: atomicIncrement(inFlightRequests) result = doRequest() atomicDecrement(inFlightRequests) I should use try-finally. Thats true, but what if doRequest() doesn't throw? Then I don't need to. Why do those two look the same? Why is the language lying that they are the same? Now lets see whats available in Swift: * `try!` means - I don't expect this piece of code to throw, even though it "might". If it does, crash horribly. * `try` means - I know that this piece of code throws, but I can't handle that exception here, so propagate it. * no keyword - This code should not throw. If it returns a Result that may be an error, I want to handle it right here. * `try?` similar as above, but discards the specific error and returns an Optional. Thats it. You manage the risk, the language only makes that explicit. It also makes it really clear whats going on during code review. Here is how the problematic piece of code looks when `doRequest` might throw: atomicIncrement(inFlightRequests) result = try doRequest() atomicDecrement(inFlightRequests) The possible bug is now very easy to spot. We don't need to remember whether `doRequest` throws or to look that up. Its clear that a finally or defer block is necessary here. Whats even better is how the code looks like when `doRequest` doesn't throw! atomicIncrement(inFlightRequests) result = doRequest() atomicDecrement(inFlightRequests) Since we know the compiler would force a `try` keyword there if there was the possibility of exception, we know this code is correct just by glancing it. No need for `defer` or `finally`. So are these two features not worth the cost? Maybe, if you are arguing that we don't need types as a tool to prevent errors. In that case, it would indeed be the programmer's fault for not writing a test for it. But if you are going to have types, you might as well have types that tell the truth. Otherwise they're useless. Finally, IMO this is not a good reason to hate on types. Stop complaining that the compiler wont accept your buggy code and take the time to do the changes. I'm happy when the compiler points out all the places that will be affected by the change - thats something that unit tests with mocks will never catch! Imagine that a change in doRequest caused it to throw, where previously it did not. The compiler will now tell you about ALL places that may be affected. What would happen if it didn't? The `inFlightRequests` counter will suddenly start behaving strange and increasing indefinitely. What would happen if above a certain value new requests don't get queued? A bug that leads to denial of service. Is it worth the hassle of changing all affected code? I don't know, what do you think? There are much better complaints against type systems, like the fact that most are not able to understand certain advanced styles of metaprogramming yet. Or that it doesn't do control flow analysis, e.g. if I type if (x != null) { doCode(x); } then `x` should have its nullability removed in the block or if (x == null) { x = nonNullValue; } then below this line, x should not be considered nulable. TypeScript for example does both of the above.