16 ms·

Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. I've just gotten back to a complex, experimental codebase after only a coup

Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. I've just gotten back to a complex, experimental codebase after only a couple months of absence, and am refactoring it to accommodate for the implementation of a number of previously unplanned features. Even with type hinting and heuristic linting it's such a huge pain! After making a large number of changes I end up having to rerun the code repeatedly to find and squash bugs, and that says nothing of the code paths I don't end up taking. Is there a better way to utilize the convenience of python for experimental code without running into the scalability issues of large python codebases? Contrast this to my experience with C# in visual studio (not visual studio code, which is inferior). The state of flow I can get into when performing large scale refactoring, with immediate, accurate feedback in the form of a clickable, line by line list of what's broken is unmatched. I would love such a pleasant experience in a python IDE but I'm not sure it's possible because of the nature of duck typing. It's like a real time, automatically generated checklist of exactly where to propagate changes, takes a massive load off my working memory and provides an uninterrupted flow of dopamine. A true state of zen if I've ever experienced one.

ehsankia 7y ago
I disagree. On the other side is static languages adding things such as "auto" and other ways to automate trivial tasks. Type annotations in dynamic languages is the flip-side, trying to reach this "ideal middle" from the other end. What's great with Python type annotation is that you can give it as much as you want, and it'll do the best it can with that. If you don't want to give it any type, then don't. If you want to give type for one function, then only do for that. You get to choose how deep down the well you wan to go.
- pkolaczk 7y ago
  I disagree. Type inference in statically typed languages (e.g. auto or var) does not weaken static typechecking guarantees. So it is not the same as writing type annotations only as much as you want. It is stronger.
  - ehsankia 7y ago
    I never said it weakens it, but it's trying to keep the strong type checking while making the language easier. Python is the opposite, it's easy to write but doesn't have the strong guarantees, and it's trying to become safer while keeping the ease of writing. This ideal "middle state" I speak of is a language that is both as easy as a dynamic language but as safe as a static language. Static languages are making it easier to write and dynamic languages are making it safer.
    - humanrebar 7y ago
      If you limit the model to source code diagnostics, you have a good point. But statically typed languages also use the types for other things: optimization, generic code instantiation, symbol resolution, and even certain kinds of metaprogramming. Optional type checking is a nice way to reduce certain kinds of errors and tighten up some underspecified designs, but it falls very much short of actual static type checking. That being said, most of the unique things static typing provides aren't needed for typical CRUD apps and glue logic, so type annotated python isn't a bad choice.
      - ehsankia 7y ago
        The way Python's type checking is implemented (it's literally just annotations) it can be used for all of the above. There are already libraries that existed before where you can accelerate specific functions, like Numba. Those libraries could switch to using type annotations instead. Similarly, IDEs are already use type annotations to enhance their symbol resolution and so on. Lastly, I'm not sure what you mean by "falls very much short of actual static type checking". With mypy and PyType, you can already catch many bugs that you wouldn't have otherwise, without running the code. That's the definition of static type checking. It may not do as much as c++, but it still catches significantly more than most other pure dynamic languages with no type.
        humanrebar 7y ago
        There are other implications, but binary linking is fraught. Minor errors corrupt entire executables, systems, and data stores, including entirely unrelated subsystems. Often statically typed systems use (at the binary level!) type information to trade off defensiveness (checking that array indexing is in bounds, maybe) for other things (latency, throughput, etc.). Point being, the type information is used for more things than just telling coders about API mismatches. Maybe some libraries could use type information to dynamically use better code paths, but statically typed languages use this information far before that... when the equivalent of a wheel is created, for instance.
- rebeccaskinner 7y ago
  This isn't the first time I've seen this argument- clearly there are people that it rings true for, but I just have a hard time seeing it. The benefits you get from types start to very quickly diminish as soon as you start adding untyped values into the mix. Sure, there are the odd cases where you might happen to have a typed function and a typed value and the checker can find a bug for you, but my experience writing python tells me these cases in reality are vanishingly rare. And the cost of this flexibility seems to be quite high. The overall quality of type checking that mypy gives you seems to be pretty abysmal, and it requires quite a bit more handholding and awkward workarounds to deal with the proliferation of any's and uninferable types than a language that just starts with types in the first place.
  - ehsankia 7y ago
    Not really. Enable something like MyPy or PyType on a big enough codebase with zero explicit annotation and it'll already find plenty of bugs and unhandled cases from the inferred types alone. Some of these are stuff a strong IDE may catch too (using the wrong function name, passing wrong number of args, etc), but some other ones are actually deeper in the code. So already, with zero annotations, you already get value out, let alone once you type a few tricky variables that are harder for Python to track.
    - rebeccaskinner 7y ago
      I've never seen that personally, but most of my python work has been on small to medium sized projects (generally in the 5-10 KLOC range).
      - nl 7y ago
        What kind of "untyped data" are you dealing with?
  - Too 7y ago
    That's like saying the value of unit tests quickly diminish once you add non tested code to the mix. For fearless refactoring, sure, you need quite high type-coverage to not get a false sense of security, but still, the more you have the safer it is. For just finding bugs and code navigation, any addition of types is going to be a win.
- umanwizard 7y ago
  `auto`, type inference, etc. has nothing to do with dynamic types. The types are still 100% static, not some sort of compromise or middle ground between static and dynamic.
  - ehsankia 7y ago
    You missed my point. Static languages try to make it easier to use, while dynamic languages try to become safer. They each try to keep what makes their side good, while getting some hints of what the other side has.
    - umanwizard 7y ago
      I guess I didn’t understand your point because I think statically typed languages are more easy to use than Python, not less, as soon as the program is longer than 500 lines or so.
      - ehsankia 7y ago
        Right, but in python, especially with use of the third party libraries, I can glue together something using less than 20 lines of code. I once had a script that recorded the microphone, converted the waveform, passed it to TTS to turn it to text, translated it, then converted it back into audio and played it. All with under 30 lines of Python. Meanwhile with C++, just trying to manipulate some strings will take that many lines. I guess we have different definition of "easy to use".
        umanwizard 7y ago
        C++ is not the only statically-typed language. Can you elaborate on why Python's lack of type checking specifically (as opposed to any of the other ease-of-use features of Python) made it easier for you to write this program? Were there cases where you were using something that a statically typed language would have considered the "wrong" type for a function argument (or anything else), but it worked fine in Python?
- zozbot234 7y ago
  > What's great with Python type annotation is that you can give it as much as you want That's also what's Not Great about this whole 'annotation' or 'gradual typing' approach. Having only some arbitrary part of the program be statically typed introduces a huge amount of 'interaction' or 'blame' points between the static and the dynamic portion of your program, that significantly impact the usual benefits of static typing. Dynamic types should be used only when strictly necessary, everything else should be made static as soon as feasible.
  - joshuamorton 7y ago
    Hard disagree: interfaces should be statically typed, but implementations need not be unless it adds value. Languages like python with a typechecker are much closer to rust (safe, but with the ability to have unsafe areas) than true statically typed langs, because static languages don't give easy escape hatches (you end up having to cast everything, no static Lang I'm aware of ships with an Any type like python's).
    - _jsdw 7y ago
      Rust is a true statically typed language; `unsafe` in rust relates to code that violates memory safety guarantees, but you can't escape the static type checking using it. Rust actually has an `Any` trait which allows for a certain amount of runtime type asserting if you don't know what concrete type you'll be working with.
      - joshuamorton 7y ago
        This is a mostly semantic distinction. Rust provides safety guarantees, but you're able to opt out of some of those in certain well defined ways.
    - colatkinson 7y ago
      Never actually seen it used, but: https://en.cppreference.com/w/cpp/utility/any https://en.cppreference.com/w/cpp/utility/any
    - 7y ago
- ibejoeb 7y ago
  Sounds like a good rationalization. I really think it's hinting for an IDE so that it knows where and when UDTs are used. The cool thing about python is that the stdlib is readily memorizable. It's a little idiomatic, but it's relatively small. The onus is on the IDE--if you want one--to be smart. Type hinting probably doesn't catch many bugs, especially with metaprogramming, but it might add some value to large codebases or team development.
threatofrain 7y ago
It’s the same feeling I have watching people work in Java; their IDE and whole language development experience is a tier above Python. I am surprised that the world decided on Python as the informal lingua franca of science.
- mafribe 7y ago
  It's a short-term vs long-term trade-off. Learning a dynamically typed language like Python is moderately easier than learning a statically typed language, especially old-fashioned languages like Java without type inference. OTOH using a statically typed language, once you've learned how to handle it, makes for a more powerful software engineering experience, but you'll have to climb the hill of typing first.
  - ken 7y ago
    How tall is that hill? I wrote C# professionally every day for 3.5 years. The last day was as painful as the first. I can appreciate making an investment in tools that make me more productive in the long run, but at some point it has to start paying off. Programming languages have finite useful lifetimes. Common Lisp took me years to learn well, too, but it started paying big dividends after only a couple weeks.
    - threatofrain 7y ago
      Are you still a Common Lisper, and what do you use it for?
  - friedman23 7y ago
    People don't use dynamic typing over static typing because dynamic typing is "easier" to learn. Types are not a difficult concept.
- inertiatic 7y ago
  It was very surreal moving from a team using idea for java development to a team using vim for python development. A lot of the smartest people are super productive and super considerate of all the possible codepaths when making changes. Bugs still occur due to statically discoverable coding errors. Just how much harder it is to navigate a new codebase is staggering.
stevesimmons 7y ago
I'm a Python programmer who has to interoperate with a messy legacy C# codebase. Consequently I don't have a good impression of C#. Can you suggest some modern, best practice C# projects for me to see how the language should be used?
- bathtub365 7y ago
  What are the issues you’re running into?
- avtar 7y ago
  > Can you suggest some modern, best practice C# projects for me to see how the language should be used? I'm curious about this as well.
- dvdgsng 7y ago
  Jellyfin is modern and written in C#, but not sure if it follows any best practices. https://github.com/jellyfin/jellyfin https://github.com/jellyfin/jellyfin
  - commotionfever 7y ago
    It mightn't be the best example in the world. It's a fork of Emby, and one of the Jellyfin devs on the Jellyfin reddit says it's been a lot of work cleaning up
ken 7y ago
Counterpoint: I feel like static typing is a bandaid for the fundamental problem that the language isn't powerful enough to allow one's code to truly be OAOO. Back when I wrote C#, static typing was indeed helpful for making large-scale changes and providing an automated check that this wasn't destroying everything. But then, it's only in languages like C# that I have to make these kinds of large-scale changes. There's no macros or syntactic abstraction. When you want to construct the same category of operation in different contexts, and functions and classes don't operate on the right axis, you're SOL. You just end up repeating yourself. When I write Lisp, I never have to make changes that cover more than their one area of responsibility. At worst, I'll rename a function, and a simple textual find-and-replace is more than sufficient -- better than most refactoring browsers, even, since it will hit my comments and documentation. Do fancy refactoring tools update your README and docs/ folder yet? (In one case, I knew an experienced C# programmer to build and compile an expression at run-time using LambdaExpression [1]. It takes about 10 times as many lines to achieve the same thing, and you have to write in a style that looks nothing at all like a normal function, so in practice nobody ever does this. In contrast, the way to accomplish this in Lisp takes one character, and the code looks identical to a normal function, so it's not unheard of.) I feel that Python, in many ways, combines the worst aspects of both worlds. It's not strict enough to be a good static language, and not powerful enough to be a good dynamic language. Sadly, the most popular dynamic languages today are Python and JS and PHP, so a lot of failures and limitations of these languages get blamed on dynamic languages in general. [1]: https://docs.microsoft.com/en-us/dotnet/api/system.linq.expressions.lambdaexpression?view=netframework-4.8 https://docs.microsoft.com/en-us/dotnet/api/system.linq.expr...
- whb07 7y ago
  There is a better approach by using statically typed functional languages like F#, Ocaml, or Haskell. They don’t need to have the type declaration as most things can be inferred. So you get the feelings of writing Python with the full suite of benefits of a strongly typed functional language when using say F#
  - mixmastamyk 7y ago
    I’d be very interested if any of these decided on non-atrocious syntax. Any progress on that front?
jonahbenton 7y ago
Agree about the dopamine flow from working with a good compiler/IDE. Golang's compiler is super fast and super fun, with very helpful messaging. Enjoyable refactoring process, to be sure. In terms of dynamic languages, the Clojure ecosystem has a lot of techniques that help with dynamic language codebase scalability. Very few are exhibited in Python codebases I see, I imagine for various reasons. The most important is short, single purpose functions arranged in layers- essential for working at the REPL, where you may want to inspect or interject at any point in the data flow. Related is a clear segregation between code and data, and explicit data flow/state control. I see this violated all the time in Python classes, which are very clever but should almost never be used. Clojure Spec is also a revelation, much more expressive than a static type system and also (IMO) much more comprehensible. Nothing equivalent in the Python world. Python works hard to be readable in the small, but it is a struggle in the large, both because the language does not help you and because the ecosystem is still pretty immature.
- j88439h84 7y ago
  I would love to see clojure spec in python.
- edflsafoiewq 7y ago
  > Related is a clear segregation between code and data, and explicit data flow/state control. I see this violated all the time in Python classes, which are very clever but should almost never be used. Could you expand on this one?
  - jonahbenton 7y ago
    Appreciate the question. The crux of this observation is that the easy opportunity Python affords to create class-based encapsulations introduces a tension between a world where data flow and state is hidden, implicit, and private- in service of attempting to define precise type-like abstractions- and a world where data flow and state is explicit and public. Although Python does a lovely job in defining type-like protocols that classes can participate in- e.g. all the special dunder methods- most Python code in the wild that I see is not beautiful-in-the-large, well-developed, reusable, leverageable abstractions. It's just business logic, conditionals, utility code, scripts-turned-into-apps, etc. Plenty of reusable functions, very few reusable types. Putting business logic mush into classes with some state and calling it a type is a straight line to unmaintainability. It becomes very difficult to extend such a system with new "types" or to modify behavior of existing types, because there is a lack of clarity about the semantics of the existing types. Rather than trying to define types with state, much better to treat state as data- the difference is that the term state implies some special smart type-like semantics, while data is just dumb keys and values. The comparison I drew was with Clojure, which doesn't make it so easy to make classes that hide state or create fake types. Instead it has graduated options for lightly packaging data elements together- start with maps/dicts and move onto records- and optimizes for just chaining functions that operate on those data blobs. "Simple", as Rich Hickey famously says. There is no spaghetti-inducing tension between keeping logic "encapsulated" vs coding it "in the open." For regular line of business apps, the latter is so much better when it comes to maintainability, but the presence of classes provides a constant temptation to find reusability/encapsulation/abstraction where it doesn't actually exist. In Python, IMO, classes should only rarely be used. Hope that helps?
Galanwe 7y ago
> Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. What...? Everything you describe here is the symptom of a bad codebase, not the symptoms of a dynamic language... > I end up having to rerun the code repeatedly to find and squash bugs, and that says nothing of the code paths I don't end up taking
- hombre_fatal 7y ago
  I prefer tools that don't optimize for the unicorn "good" code base but rather help me with real world code. Also, many problems are irreducibly complex. Just because there are a lot of code paths doesn't mean it's bad code. Not every software project is a toy. Everything in their post describes real world code and all projects beyond toy-size.
mixmastamyk 7y ago
Pyflakes should get you ~90+% of the way. Pycharm etc can get farther. Eventually you fall back on your automated testing and manual spot checks on approach of the quality target. It’s not quite the burden hn-ers make it sound.
Iv 7y ago
I keep jumping between C++, Python and Java for various projects, and really, they are different worlds. In C++ you think hard about your problem, create strong abstractions that translate into fast execution. That's a highly satisfying engineering job. In Python you do happy exploratory programming. Changing data structures on the fly, hacking your way through unexpected problem by doing crass things like adding members to a class at runtime. You can do exploratory programming in C++ but that requires a lot of additional work to experience a fraction of that freedom. And you can do solid engineering in python but that requires as well a lot of additional work. Different tools for different things. Python is the off-road bike you take for a bit of fun in the forest. C++ is the family car you take to do the week's grocery.
- echelon 7y ago
  > In C++ you think hard about your problem, create strong abstractions that translate into fast execution. That's a highly satisfying engineering job. C++ is an awful language that requires too much thinking. Make a mistake and you'll pay for it with hidden memory leaks and segfaults. It offers no help. No other mainstream language is quite so unforgiving. Rust, Go, and Swift will hopefully relegate it to the dustbin. (Of the three, only Rust is suitably non-GC'd for bare metal requirements.) > C++ is the family car you take to do the week's grocery. C++ is the Soyuz space capsule. You'd better pack a gun, because there might be bears where you land.
  - pjmlp 7y ago
    Specially when one insists in coding C++ like C.
  - otabdeveloper2 7y ago
    > It offers no help. Try using a compiler that wasn't made in the last millennium.
kortex 7y ago
> Is there a better way to utilize the convenience of python for experimental code without running into the scalability issues of large python codebases? Type hints and doctests. If that doesn't get you to pretty big static typing wins, then you are probably just dealing with a toxic codebase or something.
BurningFrog 7y ago
I think what replaces static typing as a safety rail in dynamic languages is testing, and the rise of dynamic languages the last 1-2 decades has been made possible by the parallel rise of automated test suites. Personally, I would not attempt large refactorings without a decent test suite. I consider enabling fast and safe refactoring to be the main advantage of test suites!
- pjmlp 7y ago
  Except that as anyone on large enterprise projects knows, unit testing is the bullet point that comes just after having documentation ready.
  - mxz3000 7y ago
    I have to disagree. Unit testing comes as soon as you write _any_ code. You write a new function/class? Your pull request better have unit tests for it or I'm rejecting that shit.
    - pjmlp 7y ago
      Is the customer paying for the work hours doing code reviews? If they do, usually code reviews come after unit tests.
  - BurningFrog 7y ago
    I don't doubt that true in many places. Writing tests is also an absolute must in many organizations. My advice to you is to look for work in organizations that are not garbage. We're out here!
isatty 7y ago
Sure, but would you rather have a bandaid or nothing at all? I don't think many people disagree that strong typing is a requirement for huge projects.
kerkeslager 7y ago
Let me preface this by saying, this post isn't saying that Python is better than C#. I worked extensively in C# before I worked in Python, and if it weren't for licensing, I would still be working in C#. Both languages have tradeoffs and they are different enough that they're just hard to compare. Looking strictly at the development experience of using the language, I'd be hard-pressed to say one is better than the other. > Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. I've just gotten back to a complex, experimental codebase after only a couple months of absence, and am refactoring it to accommodate for the implementation of a number of previously unplanned features. Even with type hinting and heuristic linting it's such a huge pain! After making a large number of changes I end up having to rerun the code repeatedly to find and squash bugs, and that says nothing of the code paths I don't end up taking. Is there a better way to utilize the convenience of python for experimental code without running into the scalability issues of large python codebases? What's your unit test coverage like? I'm not asking for coverage percentages (those are effectively useless) I'm asking: have you built out test coverage to the point that you trust your test suite? C# is probably the best mainstream example of a strongly-typed, statically-typed language[1]. People coming from strong, statically-typed languages tend to rest heavily on the type system. So when they come to a dynamically-typed language, they feel like a vital tool for preventing bugs and structuring code has been taken away from them. In his essay, Yes, We Have Noticed the Skulls[2], Scott Alexander notes that outsiders often look at a group and see a pile of "skulls" in the group's wake--problems the group has had in the past--without realizing that those problems are just as obvious and concerning to the insiders of the group, and that they are already addressed. The Python community is no exception: we know that a lack of static types makes our code prone to bugs. And the solution the Python community has come up with is automated testing. It's no mistake that test coverage, test-driven development, behavior-driven development, etc., were popularized in communities built around dynamic languages, and there's extensive, effective tooling for these in Python. And here's the thing: automated testing gets you more than static typing does. Haskell programmers in particular like to make the claim that "if it compiles, it's correct", but even with Haskell's type system which is stronger than C#'s, that's just not true. Types aren't a silver bullet: there's no shortcut to having to run your code to test, and if you have to run your code to test it, then it makes sense to automate the running of your code to test it. And unit tests, the static type system gets in your way. Yes, you absolutely can (and should!) write unit tests in C#. But compilation slows down your unit tests. Having a static type system makes it harder to tell where you should write a test and where you should rely on the type system. You might want to pass a null value to a function in a test because the parameter doesn't matter for the test, but if that parameter can't be null in production, you don't want to have that parameter be nullable, because you want your type system to catch that, meaning you have to hydrate a potentially complex object just to satisfy the type system. Mocking is much more simple in dynamic languages--in C# you end up creating a lot of interfaces which are only implemented once, just so you can mock them. The list goes on. The result is a sort of Pareto 80/20[3] situation: for equivalent C#/Python codebases with high reliability, C#'s type system gets you 80% of the reliability with 20% of the work, but unit testing to cover the last 20% ends up filling up the other 80%. There is probably an upper bound to this. If you really need 100% reliability, it will probably take an extra 200% effort in C#, but it might not be possible with current tooling in Python. But very few projects fall into this category. My impression of type hinting in Python is that the endgame for type hinting actually has little to do with finding bugs: it's more about optimization. The fact that type hints can be used for code verification is just a happy side effect. As a final note, I'll add that assertions are an extremely useful tool which I feel are underused in both the C# and Python communities. In C#, assertions can fill out a lot of the gaps where the type system can't catch errors. In Python, assertions can be used to verify constraints without the boilerplate of a unit test (the tradeoff being that an assertion doesn't inherently run when you run your test suite). In both languages, assertions increase the value of your unit tests (because they test the constraints whenever the code is run in the unit test, even if the constraint isn't what the unit test is intended to test). And in both languages, assertions make debugging easier because they move the error detection closer to where the bug originates. [1] There are better examples of strongly-typed, statically-typed languages, but they aren't mainstream. I'm not criticizing anyone's language, so please don't bite my head off about this. [2] https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-the-skulls/ https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-th... [3] https://en.wikipedia.org/wiki/Pareto_principle https://en.wikipedia.org/wiki/Pareto_principle

ehsankia 7y ago

I disagree. On the other side is static languages adding things such as "auto" and other ways to automate trivial tasks. Type annotations in dynamic languages is the flip-side, trying to reach this "ideal middle" from the other end. What's great with Python type annotation is that you can give it as much as you want, and it'll do the best it can with that. If you don't want to give it any type, then don't. If you want to give type for one function, then only do for that. You get to choose how deep down the well you wan to go.

pkolaczk 7y ago
I disagree. Type inference in statically typed languages (e.g. auto or var) does not weaken static typechecking guarantees. So it is not the same as writing type annotations only as much as you want. It is stronger.
- ehsankia 7y ago
  I never said it weakens it, but it's trying to keep the strong type checking while making the language easier. Python is the opposite, it's easy to write but doesn't have the strong guarantees, and it's trying to become safer while keeping the ease of writing. This ideal "middle state" I speak of is a language that is both as easy as a dynamic language but as safe as a static language. Static languages are making it easier to write and dynamic languages are making it safer.
  - humanrebar 7y ago
    If you limit the model to source code diagnostics, you have a good point. But statically typed languages also use the types for other things: optimization, generic code instantiation, symbol resolution, and even certain kinds of metaprogramming. Optional type checking is a nice way to reduce certain kinds of errors and tighten up some underspecified designs, but it falls very much short of actual static type checking. That being said, most of the unique things static typing provides aren't needed for typical CRUD apps and glue logic, so type annotated python isn't a bad choice.
    - ehsankia 7y ago
      The way Python's type checking is implemented (it's literally just annotations) it can be used for all of the above. There are already libraries that existed before where you can accelerate specific functions, like Numba. Those libraries could switch to using type annotations instead. Similarly, IDEs are already use type annotations to enhance their symbol resolution and so on. Lastly, I'm not sure what you mean by "falls very much short of actual static type checking". With mypy and PyType, you can already catch many bugs that you wouldn't have otherwise, without running the code. That's the definition of static type checking. It may not do as much as c++, but it still catches significantly more than most other pure dynamic languages with no type.
      - humanrebar 7y ago
        There are other implications, but binary linking is fraught. Minor errors corrupt entire executables, systems, and data stores, including entirely unrelated subsystems. Often statically typed systems use (at the binary level!) type information to trade off defensiveness (checking that array indexing is in bounds, maybe) for other things (latency, throughput, etc.). Point being, the type information is used for more things than just telling coders about API mismatches. Maybe some libraries could use type information to dynamically use better code paths, but statically typed languages use this information far before that... when the equivalent of a wheel is created, for instance.
rebeccaskinner 7y ago
This isn't the first time I've seen this argument- clearly there are people that it rings true for, but I just have a hard time seeing it. The benefits you get from types start to very quickly diminish as soon as you start adding untyped values into the mix. Sure, there are the odd cases where you might happen to have a typed function and a typed value and the checker can find a bug for you, but my experience writing python tells me these cases in reality are vanishingly rare. And the cost of this flexibility seems to be quite high. The overall quality of type checking that mypy gives you seems to be pretty abysmal, and it requires quite a bit more handholding and awkward workarounds to deal with the proliferation of any's and uninferable types than a language that just starts with types in the first place.
- ehsankia 7y ago
  Not really. Enable something like MyPy or PyType on a big enough codebase with zero explicit annotation and it'll already find plenty of bugs and unhandled cases from the inferred types alone. Some of these are stuff a strong IDE may catch too (using the wrong function name, passing wrong number of args, etc), but some other ones are actually deeper in the code. So already, with zero annotations, you already get value out, let alone once you type a few tricky variables that are harder for Python to track.
  - rebeccaskinner 7y ago
    I've never seen that personally, but most of my python work has been on small to medium sized projects (generally in the 5-10 KLOC range).
    - nl 7y ago
      What kind of "untyped data" are you dealing with?
- Too 7y ago
  That's like saying the value of unit tests quickly diminish once you add non tested code to the mix. For fearless refactoring, sure, you need quite high type-coverage to not get a false sense of security, but still, the more you have the safer it is. For just finding bugs and code navigation, any addition of types is going to be a win.
umanwizard 7y ago
`auto`, type inference, etc. has nothing to do with dynamic types. The types are still 100% static, not some sort of compromise or middle ground between static and dynamic.
- ehsankia 7y ago
  You missed my point. Static languages try to make it easier to use, while dynamic languages try to become safer. They each try to keep what makes their side good, while getting some hints of what the other side has.
  - umanwizard 7y ago
    I guess I didn’t understand your point because I think statically typed languages are more easy to use than Python, not less, as soon as the program is longer than 500 lines or so.
    - ehsankia 7y ago
      Right, but in python, especially with use of the third party libraries, I can glue together something using less than 20 lines of code. I once had a script that recorded the microphone, converted the waveform, passed it to TTS to turn it to text, translated it, then converted it back into audio and played it. All with under 30 lines of Python. Meanwhile with C++, just trying to manipulate some strings will take that many lines. I guess we have different definition of "easy to use".
      - umanwizard 7y ago
        C++ is not the only statically-typed language. Can you elaborate on why Python's lack of type checking specifically (as opposed to any of the other ease-of-use features of Python) made it easier for you to write this program? Were there cases where you were using something that a statically typed language would have considered the "wrong" type for a function argument (or anything else), but it worked fine in Python?
zozbot234 7y ago
> What's great with Python type annotation is that you can give it as much as you want That's also what's Not Great about this whole 'annotation' or 'gradual typing' approach. Having only some arbitrary part of the program be statically typed introduces a huge amount of 'interaction' or 'blame' points between the static and the dynamic portion of your program, that significantly impact the usual benefits of static typing. Dynamic types should be used only when strictly necessary, everything else should be made static as soon as feasible.
- joshuamorton 7y ago
  Hard disagree: interfaces should be statically typed, but implementations need not be unless it adds value. Languages like python with a typechecker are much closer to rust (safe, but with the ability to have unsafe areas) than true statically typed langs, because static languages don't give easy escape hatches (you end up having to cast everything, no static Lang I'm aware of ships with an Any type like python's).
  - _jsdw 7y ago
    Rust is a true statically typed language; `unsafe` in rust relates to code that violates memory safety guarantees, but you can't escape the static type checking using it. Rust actually has an `Any` trait which allows for a certain amount of runtime type asserting if you don't know what concrete type you'll be working with.
    - joshuamorton 7y ago
      This is a mostly semantic distinction. Rust provides safety guarantees, but you're able to opt out of some of those in certain well defined ways.
  - colatkinson 7y ago
    Never actually seen it used, but: https://en.cppreference.com/w/cpp/utility/any https://en.cppreference.com/w/cpp/utility/any
  - 7y ago
ibejoeb 7y ago
Sounds like a good rationalization. I really think it's hinting for an IDE so that it knows where and when UDTs are used. The cool thing about python is that the stdlib is readily memorizable. It's a little idiomatic, but it's relatively small. The onus is on the IDE--if you want one--to be smart. Type hinting probably doesn't catch many bugs, especially with metaprogramming, but it might add some value to large codebases or team development.

threatofrain 7y ago

It’s the same feeling I have watching people work in Java; their IDE and whole language development experience is a tier above Python. I am surprised that the world decided on Python as the informal lingua franca of science.

mafribe 7y ago
It's a short-term vs long-term trade-off. Learning a dynamically typed language like Python is moderately easier than learning a statically typed language, especially old-fashioned languages like Java without type inference. OTOH using a statically typed language, once you've learned how to handle it, makes for a more powerful software engineering experience, but you'll have to climb the hill of typing first.
- ken 7y ago
  How tall is that hill? I wrote C# professionally every day for 3.5 years. The last day was as painful as the first. I can appreciate making an investment in tools that make me more productive in the long run, but at some point it has to start paying off. Programming languages have finite useful lifetimes. Common Lisp took me years to learn well, too, but it started paying big dividends after only a couple weeks.
  - threatofrain 7y ago
    Are you still a Common Lisper, and what do you use it for?
- friedman23 7y ago
  People don't use dynamic typing over static typing because dynamic typing is "easier" to learn. Types are not a difficult concept.
inertiatic 7y ago
It was very surreal moving from a team using idea for java development to a team using vim for python development. A lot of the smartest people are super productive and super considerate of all the possible codepaths when making changes. Bugs still occur due to statically discoverable coding errors. Just how much harder it is to navigate a new codebase is staggering.

stevesimmons 7y ago

I'm a Python programmer who has to interoperate with a messy legacy C# codebase. Consequently I don't have a good impression of C#. Can you suggest some modern, best practice C# projects for me to see how the language should be used?

bathtub365 7y ago
What are the issues you’re running into?
avtar 7y ago
> Can you suggest some modern, best practice C# projects for me to see how the language should be used? I'm curious about this as well.
dvdgsng 7y ago
Jellyfin is modern and written in C#, but not sure if it follows any best practices. https://github.com/jellyfin/jellyfin https://github.com/jellyfin/jellyfin
- commotionfever 7y ago
  It mightn't be the best example in the world. It's a fork of Emby, and one of the Jellyfin devs on the Jellyfin reddit says it's been a lot of work cleaning up

ken 7y ago

Counterpoint: I feel like static typing is a bandaid for the fundamental problem that the language isn't powerful enough to allow one's code to truly be OAOO. Back when I wrote C#, static typing was indeed helpful for making large-scale changes and providing an automated check that this wasn't destroying everything. But then, it's only in languages like C# that I have to make these kinds of large-scale changes. There's no macros or syntactic abstraction. When you want to construct the same category of operation in different contexts, and functions and classes don't operate on the right axis, you're SOL. You just end up repeating yourself. When I write Lisp, I never have to make changes that cover more than their one area of responsibility. At worst, I'll rename a function, and a simple textual find-and-replace is more than sufficient -- better than most refactoring browsers, even, since it will hit my comments and documentation. Do fancy refactoring tools update your README and docs/ folder yet? (In one case, I knew an experienced C# programmer to build and compile an expression at run-time using LambdaExpression [1]. It takes about 10 times as many lines to achieve the same thing, and you have to write in a style that looks nothing at all like a normal function, so in practice nobody ever does this. In contrast, the way to accomplish this in Lisp takes one character, and the code looks identical to a normal function, so it's not unheard of.) I feel that Python, in many ways, combines the worst aspects of both worlds. It's not strict enough to be a good static language, and not powerful enough to be a good dynamic language. Sadly, the most popular dynamic languages today are Python and JS and PHP, so a lot of failures and limitations of these languages get blamed on dynamic languages in general. [1]: https://docs.microsoft.com/en-us/dotnet/api/system.linq.expressions.lambdaexpression?view=netframework-4.8 https://docs.microsoft.com/en-us/dotnet/api/system.linq.expr...

whb07 7y ago
There is a better approach by using statically typed functional languages like F#, Ocaml, or Haskell. They don’t need to have the type declaration as most things can be inferred. So you get the feelings of writing Python with the full suite of benefits of a strongly typed functional language when using say F#
- mixmastamyk 7y ago
  I’d be very interested if any of these decided on non-atrocious syntax. Any progress on that front?

jonahbenton 7y ago

Agree about the dopamine flow from working with a good compiler/IDE. Golang's compiler is super fast and super fun, with very helpful messaging. Enjoyable refactoring process, to be sure. In terms of dynamic languages, the Clojure ecosystem has a lot of techniques that help with dynamic language codebase scalability. Very few are exhibited in Python codebases I see, I imagine for various reasons. The most important is short, single purpose functions arranged in layers- essential for working at the REPL, where you may want to inspect or interject at any point in the data flow. Related is a clear segregation between code and data, and explicit data flow/state control. I see this violated all the time in Python classes, which are very clever but should almost never be used. Clojure Spec is also a revelation, much more expressive than a static type system and also (IMO) much more comprehensible. Nothing equivalent in the Python world. Python works hard to be readable in the small, but it is a struggle in the large, both because the language does not help you and because the ecosystem is still pretty immature.

j88439h84 7y ago
I would love to see clojure spec in python.
edflsafoiewq 7y ago
> Related is a clear segregation between code and data, and explicit data flow/state control. I see this violated all the time in Python classes, which are very clever but should almost never be used. Could you expand on this one?
- jonahbenton 7y ago
  Appreciate the question. The crux of this observation is that the easy opportunity Python affords to create class-based encapsulations introduces a tension between a world where data flow and state is hidden, implicit, and private- in service of attempting to define precise type-like abstractions- and a world where data flow and state is explicit and public. Although Python does a lovely job in defining type-like protocols that classes can participate in- e.g. all the special dunder methods- most Python code in the wild that I see is not beautiful-in-the-large, well-developed, reusable, leverageable abstractions. It's just business logic, conditionals, utility code, scripts-turned-into-apps, etc. Plenty of reusable functions, very few reusable types. Putting business logic mush into classes with some state and calling it a type is a straight line to unmaintainability. It becomes very difficult to extend such a system with new "types" or to modify behavior of existing types, because there is a lack of clarity about the semantics of the existing types. Rather than trying to define types with state, much better to treat state as data- the difference is that the term state implies some special smart type-like semantics, while data is just dumb keys and values. The comparison I drew was with Clojure, which doesn't make it so easy to make classes that hide state or create fake types. Instead it has graduated options for lightly packaging data elements together- start with maps/dicts and move onto records- and optimizes for just chaining functions that operate on those data blobs. "Simple", as Rich Hickey famously says. There is no spaghetti-inducing tension between keeping logic "encapsulated" vs coding it "in the open." For regular line of business apps, the latter is so much better when it comes to maintainability, but the presence of classes provides a constant temptation to find reusability/encapsulation/abstraction where it doesn't actually exist. In Python, IMO, classes should only rarely be used. Hope that helps?

Galanwe 7y ago

> Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. What...? Everything you describe here is the symptom of a bad codebase, not the symptoms of a dynamic language... > I end up having to rerun the code repeatedly to find and squash bugs, and that says nothing of the code paths I don't end up taking

hombre_fatal 7y ago
I prefer tools that don't optimize for the unicorn "good" code base but rather help me with real world code. Also, many problems are irreducibly complex. Just because there are a lot of code paths doesn't mean it's bad code. Not every software project is a toy. Everything in their post describes real world code and all projects beyond toy-size.

mixmastamyk 7y ago

Pyflakes should get you ~90+% of the way. Pycharm etc can get farther. Eventually you fall back on your automated testing and manual spot checks on approach of the quality target. It’s not quite the burden hn-ers make it sound.

Iv 7y ago

I keep jumping between C++, Python and Java for various projects, and really, they are different worlds. In C++ you think hard about your problem, create strong abstractions that translate into fast execution. That's a highly satisfying engineering job. In Python you do happy exploratory programming. Changing data structures on the fly, hacking your way through unexpected problem by doing crass things like adding members to a class at runtime. You can do exploratory programming in C++ but that requires a lot of additional work to experience a fraction of that freedom. And you can do solid engineering in python but that requires as well a lot of additional work. Different tools for different things. Python is the off-road bike you take for a bit of fun in the forest. C++ is the family car you take to do the week's grocery.

echelon 7y ago
> In C++ you think hard about your problem, create strong abstractions that translate into fast execution. That's a highly satisfying engineering job. C++ is an awful language that requires too much thinking. Make a mistake and you'll pay for it with hidden memory leaks and segfaults. It offers no help. No other mainstream language is quite so unforgiving. Rust, Go, and Swift will hopefully relegate it to the dustbin. (Of the three, only Rust is suitably non-GC'd for bare metal requirements.) > C++ is the family car you take to do the week's grocery. C++ is the Soyuz space capsule. You'd better pack a gun, because there might be bears where you land.
- pjmlp 7y ago
  Specially when one insists in coding C++ like C.
- otabdeveloper2 7y ago
  > It offers no help. Try using a compiler that wasn't made in the last millennium.

kortex 7y ago

> Is there a better way to utilize the convenience of python for experimental code without running into the scalability issues of large python codebases? Type hints and doctests. If that doesn't get you to pretty big static typing wins, then you are probably just dealing with a toxic codebase or something.

BurningFrog 7y ago

I think what replaces static typing as a safety rail in dynamic languages is testing, and the rise of dynamic languages the last 1-2 decades has been made possible by the parallel rise of automated test suites. Personally, I would not attempt large refactorings without a decent test suite. I consider enabling fast and safe refactoring to be the main advantage of test suites!

pjmlp 7y ago
Except that as anyone on large enterprise projects knows, unit testing is the bullet point that comes just after having documentation ready.
- mxz3000 7y ago
  I have to disagree. Unit testing comes as soon as you write _any_ code. You write a new function/class? Your pull request better have unit tests for it or I'm rejecting that shit.
  - pjmlp 7y ago
    Is the customer paying for the work hours doing code reviews? If they do, usually code reviews come after unit tests.
- BurningFrog 7y ago
  I don't doubt that true in many places. Writing tests is also an absolute must in many organizations. My advice to you is to look for work in organizations that are not garbage. We're out here!

isatty 7y ago

Sure, but would you rather have a bandaid or nothing at all? I don't think many people disagree that strong typing is a requirement for huge projects.

kerkeslager 7y ago

Let me preface this by saying, this post isn't saying that Python is better than C#. I worked extensively in C# before I worked in Python, and if it weren't for licensing, I would still be working in C#. Both languages have tradeoffs and they are different enough that they're just hard to compare. Looking strictly at the development experience of using the language, I'd be hard-pressed to say one is better than the other. > Type hinting feels like a bandaid for a fundamental limitation of dynamic languages. I've just gotten back to a complex, experimental codebase after only a couple months of absence, and am refactoring it to accommodate for the implementation of a number of previously unplanned features. Even with type hinting and heuristic linting it's such a huge pain! After making a large number of changes I end up having to rerun the code repeatedly to find and squash bugs, and that says nothing of the code paths I don't end up taking. Is there a better way to utilize the convenience of python for experimental code without running into the scalability issues of large python codebases? What's your unit test coverage like? I'm not asking for coverage percentages (those are effectively useless) I'm asking: have you built out test coverage to the point that you trust your test suite? C# is probably the best mainstream example of a strongly-typed, statically-typed language[1]. People coming from strong, statically-typed languages tend to rest heavily on the type system. So when they come to a dynamically-typed language, they feel like a vital tool for preventing bugs and structuring code has been taken away from them. In his essay, Yes, We Have Noticed the Skulls[2], Scott Alexander notes that outsiders often look at a group and see a pile of "skulls" in the group's wake--problems the group has had in the past--without realizing that those problems are just as obvious and concerning to the insiders of the group, and that they are already addressed. The Python community is no exception: we know that a lack of static types makes our code prone to bugs. And the solution the Python community has come up with is automated testing. It's no mistake that test coverage, test-driven development, behavior-driven development, etc., were popularized in communities built around dynamic languages, and there's extensive, effective tooling for these in Python. And here's the thing: automated testing gets you more than static typing does. Haskell programmers in particular like to make the claim that "if it compiles, it's correct", but even with Haskell's type system which is stronger than C#'s, that's just not true. Types aren't a silver bullet: there's no shortcut to having to run your code to test, and if you have to run your code to test it, then it makes sense to automate the running of your code to test it. And unit tests, the static type system gets in your way. Yes, you absolutely can (and should!) write unit tests in C#. But compilation slows down your unit tests. Having a static type system makes it harder to tell where you should write a test and where you should rely on the type system. You might want to pass a null value to a function in a test because the parameter doesn't matter for the test, but if that parameter can't be null in production, you don't want to have that parameter be nullable, because you want your type system to catch that, meaning you have to hydrate a potentially complex object just to satisfy the type system. Mocking is much more simple in dynamic languages--in C# you end up creating a lot of interfaces which are only implemented once, just so you can mock them. The list goes on. The result is a sort of Pareto 80/20[3] situation: for equivalent C#/Python codebases with high reliability, C#'s type system gets you 80% of the reliability with 20% of the work, but unit testing to cover the last 20% ends up filling up the other 80%. There is probably an upper bound to this. If you really need 100% reliability, it will probably take an extra 200% effort in C#, but it might not be possible with current tooling in Python. But very few projects fall into this category. My impression of type hinting in Python is that the endgame for type hinting actually has little to do with finding bugs: it's more about optimization. The fact that type hints can be used for code verification is just a happy side effect. As a final note, I'll add that assertions are an extremely useful tool which I feel are underused in both the C# and Python communities. In C#, assertions can fill out a lot of the gaps where the type system can't catch errors. In Python, assertions can be used to verify constraints without the boilerplate of a unit test (the tradeoff being that an assertion doesn't inherently run when you run your test suite). In both languages, assertions increase the value of your unit tests (because they test the constraints whenever the code is run in the unit test, even if the constraint isn't what the unit test is intended to test). And in both languages, assertions make debugging easier because they move the error detection closer to where the bug originates. [1] There are better examples of strongly-typed, statically-typed languages, but they aren't mainstream. I'm not criticizing anyone's language, so please don't bite my head off about this. [2] https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-the-skulls/ https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-th... [3] https://en.wikipedia.org/wiki/Pareto_principle https://en.wikipedia.org/wiki/Pareto_principle