6 ms·

The fact that Python is slow isn't its only problem. What I care more about nowadays is wasting my time hunting bugs that could have been avoided by a static ty

by dom96 9y ago

The fact that Python is slow isn't its only problem. What I care more about nowadays is wasting my time hunting bugs that could have been avoided by a static type system.

blumomo 9y ago
Please tell me, do you write tests? I've learned that it's necessary. I do 100% code coverage and I'm enjoying TDD a lot.
- virmundi 9y ago
  The whole point of static type systems is that they give you the "type tests". 100% code coverage just to get what the compiler would give you is a waste of time. If python is supposed make developers more productive, this is just dragging them down.
- adwn 9y ago
  Time/cost to locate and fix a bug: at compile time < in unit test < in integration test < in production. Besides, tests cannot find all the bugs a good static type system can find, and vice versa. Tests, even at 100% coverage, are not a complete substitute.
- vog 9y ago
  If you haven't already tried, I recommend to port [1] a Python program to OCaml, F# or Haskell. There are so many tests you can throw away, so many corner cases you don't need to test anymore. (I'm talking about tests that you can't even write down without getting a compiler error.) Not to mention the assertions within your functions that aren't needed anymore. And those little type annotations are so much simpler and easier to write down than corresponding tests. If you really head for 100% testing, not just code coverage, but also all corner cases, you will love the modern static type systems. (However, you should really use an ML type system, because doing the same with Java or C++ is cumbersome and not much fun.) [1] I did so for my mathematics diploma thesis, starting implementing my formalism in Python, having lots of trouble when refactoring, then moving all code to OCaml and then being able to refactor the code alongside the developing formalism in the thesis.
- VT_Drew 9y ago
  >I'm enjoying TDD a lot Meh, test are great in so far as they can help you not break things if you upgrade the framework, or langauge in the future. But other than that I have to agree with Kent Beck's sentiment of "I get paid for code that works, not for tests". https://istacee.wordpress.com/2013/09/18/kent-beck-i-get-paid-for-code-that-works-not-for-tests/ https://istacee.wordpress.com/2013/09/18/kent-beck-i-get-pai... If I had started out trying to learn Django using TDD as in this tutorial: http://chimera.labs.oreilly.com/books/1234000000754/index.html http://chimera.labs.oreilly.com/books/1234000000754/index.ht... I never would have learned Django. I would have given up with the ridiculous slow progress of results and functionality.
- jsd1982 9y ago
  100% coverage is meaningless with respect to correctness and the ability to prove facts about your code. 100% coverage only means that you've been able to exercise all relevant lines of code at least once, not that you've tested all possible execution paths through the code. The idea that you could somehow replace the utility of static type checking with a suite of human written tests is laughable. Static typing systems allow you to prove facts about your code which make reasoning about correctness much easier than with dynamic type systems which practically disallow this (unless gradual typing is allowed in your language).
- flogic 9y ago
  Static types and tests check different things. Tests are for making things function as desired. Types are for making sure they actually connect sensibly where desired. Sometimes you can use them for the other purposes though that often produces ugly tests/types.
gipp 9y ago
Mypy / the PyCharm typechecker have come a long way. It's not the same, of course, but you can get a good bit of mileage out of the new gradual typing system these days.
nerdwaller 9y ago
Have you checked out MyPy and the optional type hinting? I've been going that route in my python recently and been liking the both/and of having an optional type checker.
- dom96 9y ago
  I have, and I do like that Python is heading in this direction. But this feature does still feel half-baked to me, the error messages are often confusing and I do recall running into a few bugs along the way. I'm also not certain that the type hinting will ever be 100%, the dynamic nature of Python seems to prevent that. This could end up giving some people a false sense of correctness.
  - nerdwaller 9y ago
    The _getting started_ part could certainly be a bit smoother, I have had a few issues as well. Generally speaking I think for in-house things you can probably alleviate a bit of the possible dynamic issues that you mention - just by discouraging too much meta-ness.
ageofwant 9y ago
You only pay the the price while debugging. In your static language you pay the price continually. You are wasting orders of magnitude more time fighting your language's type system every day, you have to read reams of boilerplate code that are unrelated to the problem at hand, each extra line increases the attack surface and complexity of your code.
- christophilus 9y ago
  Having worked in both dynamic and statically typed systems, I don't think this is true. A good statically typed language doesn't get in your way very often, and it doesn't produce any more boilerplate than doc comments do in dynamic languages. You do write doc comments, right? In fact, I'd say that most of the time, a good statically typed language does the opposite of get in your way. It helps you w/ refactoring tools, better intellisense and editor help, self-documentation, etc. The older a codebase gets, the more helpful a static type system is. That said, there are statically typed languages which produce a ton of boilerplate (Java used to, at least), and there are those that don't (F# comes to mind). The only dynamically typed language I've used where I didn't regularly miss static typing is Clojure. (Elixir may fit the bill, but the jury's still out on that.)
- asah 9y ago
  This. Let me add a radical point: static typing is often thrown out anyway, when you encode your data as strings, serialize/marshal it, create polymorphic lists and hierarchical data structures, etc etc Meanwhile, disciplined use of python is perfectly easy to get right: - use pylint - be ruthlessly consistent in naming classes, methods and variables (which pylint helps btw,) - for large projects, consider mypy etc
  - contravariant 9y ago
    If you're working in a statically typed language and cast everything to string / object you deserve all the bugs you get.
- carlmr 9y ago
  >You are wasting orders of magnitude more time fighting your language's type system every day I program C# a lot, I never fight the type system. It helps me. It tells me while coding that something is wrong. In Python I'll catch this as well...eventually, when I run the code. A type error in C# would be a runtime error in Python. That's very inconvenient. The static types are also documentation showing intent. It's hard to read old dynamically typed code bases, because I have to do the type processing in my head to understand what is what. Much simpler if I have nicely defined interfaces.
autokad 9y ago
I see this stated as if its a universal fact, but it really true that static types reduce over-all bug density? I myself have found this to not be true, and I have found the same in reading / talking to others. However, if you have found a resource that has data on the contrary, I would find it very interesting to read. https://medium.com/javascript-scene/the-shocking-secret-about-static-types-514d39bf30a3 https://medium.com/javascript-scene/the-shocking-secret-abou...
- contravariant 9y ago
  A static type system doesn't protect you from choosing the wrong abstraction, and all the bugs that result from it. But if used properly I think it does help.
  - traverseda 9y ago
    How? I don't think I've ever gotten a type I wasn't expecting in any of my python code. It's just not a problem for me. That is, past the first time I run it. I've been known to pass something an x when it wanted a y, but that's a "fail early, fail loud" bug and not an actual problem.
    - zwerdlds 9y ago
      So, I have a tiny bit of experience with Python. What I think you're describing is a good representation of my experience: If you're not branching too much, or you unit test enough, Python can be really, fabulously, great. If your logic is complex, the solace that having a really good type system (eg tagged unions paired with pattern matching) is that your edge cases are handled. Maybe not according to your business rules, but at least you'll find fewer runtime errors. Python is a tool. I think of it similar to Legos: it allows you to build small units of work. But when the work gets more complex, it's time to look at something that can handle that complexity. This was posted the other day, and probably communicates my point about runtime/compiler time identification of issues better: http://samwho.co.uk/blog/2017/09/06/move-your-bugs-to-the-left/ http://samwho.co.uk/blog/2017/09/06/move-your-bugs-to-the-le...

by dom96 9y ago

The fact that Python is slow isn't its only problem. What I care more about nowadays is wasting my time hunting bugs that could have been avoided by a static type system.

blumomo 9y ago

Please tell me, do you write tests? I've learned that it's necessary. I do 100% code coverage and I'm enjoying TDD a lot.

virmundi 9y ago
The whole point of static type systems is that they give you the "type tests". 100% code coverage just to get what the compiler would give you is a waste of time. If python is supposed make developers more productive, this is just dragging them down.
adwn 9y ago
Time/cost to locate and fix a bug: at compile time < in unit test < in integration test < in production. Besides, tests cannot find all the bugs a good static type system can find, and vice versa. Tests, even at 100% coverage, are not a complete substitute.
vog 9y ago
If you haven't already tried, I recommend to port [1] a Python program to OCaml, F# or Haskell. There are so many tests you can throw away, so many corner cases you don't need to test anymore. (I'm talking about tests that you can't even write down without getting a compiler error.) Not to mention the assertions within your functions that aren't needed anymore. And those little type annotations are so much simpler and easier to write down than corresponding tests. If you really head for 100% testing, not just code coverage, but also all corner cases, you will love the modern static type systems. (However, you should really use an ML type system, because doing the same with Java or C++ is cumbersome and not much fun.) [1] I did so for my mathematics diploma thesis, starting implementing my formalism in Python, having lots of trouble when refactoring, then moving all code to OCaml and then being able to refactor the code alongside the developing formalism in the thesis.
VT_Drew 9y ago
>I'm enjoying TDD a lot Meh, test are great in so far as they can help you not break things if you upgrade the framework, or langauge in the future. But other than that I have to agree with Kent Beck's sentiment of "I get paid for code that works, not for tests". https://istacee.wordpress.com/2013/09/18/kent-beck-i-get-paid-for-code-that-works-not-for-tests/ https://istacee.wordpress.com/2013/09/18/kent-beck-i-get-pai... If I had started out trying to learn Django using TDD as in this tutorial: http://chimera.labs.oreilly.com/books/1234000000754/index.html http://chimera.labs.oreilly.com/books/1234000000754/index.ht... I never would have learned Django. I would have given up with the ridiculous slow progress of results and functionality.
jsd1982 9y ago
100% coverage is meaningless with respect to correctness and the ability to prove facts about your code. 100% coverage only means that you've been able to exercise all relevant lines of code at least once, not that you've tested all possible execution paths through the code. The idea that you could somehow replace the utility of static type checking with a suite of human written tests is laughable. Static typing systems allow you to prove facts about your code which make reasoning about correctness much easier than with dynamic type systems which practically disallow this (unless gradual typing is allowed in your language).
flogic 9y ago
Static types and tests check different things. Tests are for making things function as desired. Types are for making sure they actually connect sensibly where desired. Sometimes you can use them for the other purposes though that often produces ugly tests/types.

gipp 9y ago

Mypy / the PyCharm typechecker have come a long way. It's not the same, of course, but you can get a good bit of mileage out of the new gradual typing system these days.

nerdwaller 9y ago

Have you checked out MyPy and the optional type hinting? I've been going that route in my python recently and been liking the both/and of having an optional type checker.

dom96 9y ago
I have, and I do like that Python is heading in this direction. But this feature does still feel half-baked to me, the error messages are often confusing and I do recall running into a few bugs along the way. I'm also not certain that the type hinting will ever be 100%, the dynamic nature of Python seems to prevent that. This could end up giving some people a false sense of correctness.
- nerdwaller 9y ago
  The _getting started_ part could certainly be a bit smoother, I have had a few issues as well. Generally speaking I think for in-house things you can probably alleviate a bit of the possible dynamic issues that you mention - just by discouraging too much meta-ness.

ageofwant 9y ago

You only pay the the price while debugging. In your static language you pay the price continually. You are wasting orders of magnitude more time fighting your language's type system every day, you have to read reams of boilerplate code that are unrelated to the problem at hand, each extra line increases the attack surface and complexity of your code.

christophilus 9y ago
Having worked in both dynamic and statically typed systems, I don't think this is true. A good statically typed language doesn't get in your way very often, and it doesn't produce any more boilerplate than doc comments do in dynamic languages. You do write doc comments, right? In fact, I'd say that most of the time, a good statically typed language does the opposite of get in your way. It helps you w/ refactoring tools, better intellisense and editor help, self-documentation, etc. The older a codebase gets, the more helpful a static type system is. That said, there are statically typed languages which produce a ton of boilerplate (Java used to, at least), and there are those that don't (F# comes to mind). The only dynamically typed language I've used where I didn't regularly miss static typing is Clojure. (Elixir may fit the bill, but the jury's still out on that.)
asah 9y ago
This. Let me add a radical point: static typing is often thrown out anyway, when you encode your data as strings, serialize/marshal it, create polymorphic lists and hierarchical data structures, etc etc Meanwhile, disciplined use of python is perfectly easy to get right: - use pylint - be ruthlessly consistent in naming classes, methods and variables (which pylint helps btw,) - for large projects, consider mypy etc
- contravariant 9y ago
  If you're working in a statically typed language and cast everything to string / object you deserve all the bugs you get.
carlmr 9y ago
>You are wasting orders of magnitude more time fighting your language's type system every day I program C# a lot, I never fight the type system. It helps me. It tells me while coding that something is wrong. In Python I'll catch this as well...eventually, when I run the code. A type error in C# would be a runtime error in Python. That's very inconvenient. The static types are also documentation showing intent. It's hard to read old dynamically typed code bases, because I have to do the type processing in my head to understand what is what. Much simpler if I have nicely defined interfaces.

autokad 9y ago

I see this stated as if its a universal fact, but it really true that static types reduce over-all bug density? I myself have found this to not be true, and I have found the same in reading / talking to others. However, if you have found a resource that has data on the contrary, I would find it very interesting to read. https://medium.com/javascript-scene/the-shocking-secret-about-static-types-514d39bf30a3 https://medium.com/javascript-scene/the-shocking-secret-abou...

contravariant 9y ago
A static type system doesn't protect you from choosing the wrong abstraction, and all the bugs that result from it. But if used properly I think it does help.
- traverseda 9y ago
  How? I don't think I've ever gotten a type I wasn't expecting in any of my python code. It's just not a problem for me. That is, past the first time I run it. I've been known to pass something an x when it wanted a y, but that's a "fail early, fail loud" bug and not an actual problem.
  - zwerdlds 9y ago
    So, I have a tiny bit of experience with Python. What I think you're describing is a good representation of my experience: If you're not branching too much, or you unit test enough, Python can be really, fabulously, great. If your logic is complex, the solace that having a really good type system (eg tagged unions paired with pattern matching) is that your edge cases are handled. Maybe not according to your business rules, but at least you'll find fewer runtime errors. Python is a tool. I think of it similar to Legos: it allows you to build small units of work. But when the work gets more complex, it's time to look at something that can handle that complexity. This was posted the other day, and probably communicates my point about runtime/compiler time identification of issues better: http://samwho.co.uk/blog/2017/09/06/move-your-bugs-to-the-left/ http://samwho.co.uk/blog/2017/09/06/move-your-bugs-to-the-le...