7 ms·

If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well?

• ocamoss 4mo ago
  Running more type checkers isn't really about strictness. The main benefit to library maintainers is to make sure that their APIs are compatible with whatever tools their users run. This wouldn't really be an issue for most other languages, but Python's typing ecosystem is uniquely fragmented, with only partial standardization between several popular tools.
  • deleted 4mo ago
    [deleted]
  • fg137 4mo ago
    Hmm... that doesn't answer the question?
    • locknitpicker 4mo ago
      > Hmm... that doesn't answer the question? GP's point is obvious: performance is immaterial to the discussion. Static code analysis is about preventing bugs. Therefore OP fails to make any sort of point, as it's a straw man argument.
• deleted 4mo ago
  [deleted]
• pmontra 4mo ago
  Hallelujah, that's always been my position. To the static typing folks: leave my dynamically typed languages alone and go coding with something that really suit your needs. If the answer is that Python, Ruby, JS, whatever are really much more pleasant to code with, my reply is that they are so precisely because we don't have to type type definitions. Tradeoffs.
  • Tade0 4mo ago
    Personally I like having my TypeScript cake and eating it. I also truly believe those who design type systems would benefit from taking a look what kind of code people programming in dynamically-typed languages produce.
    • anamexis 4mo ago
      I do too, but I feel like TypeScript stands alone as an unusually effective and pleasant to use bolted-on type system. I've not seen any other approach come close. (My sample size is Python, Ruby and Elixir)
      • dnautics 4mo ago
        you don't think the elixir type system is effective? I've never seen a bolted-on type system get so much acceptance from the hardcore "you can add types into my dead hands" crowd
      • jamesfinlayson 4mo ago
        I really like PHP's type hints (I think they were the first I used) though it's somewhat limited (can't type hint complex/nested structures last time I checked). Flow for Javascript was okay but Typescript I've found to be much nicer (last used flow years ago but occasionally I'd encounter bugs in Flow). Python's is okay but it feels clunky.
    • junon 4mo ago
      I hate TS's tooling with a burning, deep passion. But its type system is actually pretty incredible for what it is. There are times that I yearn for TS's ability to do duck type reasoning in e.g. Rust (despite that not being feasible) when working with very large data types.
    • joha4270 4mo ago
      Could you point me towards the kind of code people programming in dynamically-typed languages produce? I have lived in statically typed languages almost all of my life, and even when I don't, I pretend I do, just without having a typechecker. So I'm very curious about what I'm missing.
• SatvikBeri 4mo ago
  What statically typed language would you suggest for machine learning and large data pipelines? I don't love Python, but it has by far the largest ecosystem.
  • srean 4mo ago
    You could try Cython and Lush. An ML dialect for ML would have been nice, but doesn't exist.
    • pdpi 4mo ago
      As funny as it would be, ML isn't really a great fit for ML, I don't think.
      • srean 4mo ago
        That's true for current ML offerings. However, I think an ML designed for machine learning would be nice, especially if the type system is extended to multidimensional arrays shapes. Pattern matching on array shapes would be rather nice. Ocaml style interactive mode for exploration and compiling for performance would be nice too.
    • thr1owaway9621 4mo ago
      Cython is a niche language for writing perf-critical bits inside your Python codebase. It's like C for people who don't want to learn C. At least that's how I treated it, when I had to write some stuff to make some numpy ops faster. Cython is not in any real sense a replacement for a modern data/ml stack.
      • srean 4mo ago
        True but it's really nice way to get the benefit of type checking in Python. Just like you, I had started using Cython for performance but then realized that I can discard a bulk of type errors if I used for type checking. The other benefit is that the Python library ecosystem stays available.
  • 4mo ago
• Qem 4mo ago
  > If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well? You can use type-checking to get better performance already, without leaving Python. See https://blog.glyph.im/2022/04/you-should-compile-your-python-and-heres-why.html https://blog.glyph.im/2022/04/you-should-compile-your-python...
• ethagnawl 4mo ago
  Yeah, I can't say I really get the appeal of gradual typing. It's commented/documented code at best and outright lies at worst. Sure, you can build tooling around it and improve your DX a bit but isn't it always a house of cards?
  • ocamoss 4mo ago
    > commented/documented code at best Machine-checked documentation is always valuable, IMO
  • falcojr 4mo ago
    If you add one of these type checkers into your CI or a pre-commit hook, it provides the same guarantees you get from a compiler along with the same tooling benefits. It gives you the option of using the structure when you need it, but not being forced to use it when you want to take advantage of some of the more dynamic features of the language.
    • chuckadams 4mo ago
      But none of the runtime benefits of having static types in the compiler, since the runtime still can't trust the types. Still, half a loaf is better than none.
• hedora 4mo ago
  [dead]
• MeetingsBrowser 4mo ago
  strict type checking is an incredibly useful tool for cases when you really want to make sure your code is correct and behaving as expected (one of many tools). There are lots of people who like python and want to use it for things that where incorrect code has serious consequences. Type checking is helpful in these contexts. Type checking remains optional for the masses and is not practical in many cases. Still, pushing away people who want to use all available tools for writing correct python only hurts the community.
• locknitpicker 4mo ago
  > If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well? I don't understand your question. The whole point of static code analysis is preventing bugs. Don't you like Python code to not have bugs that are easily caught with static code analysis, or is preventing code a foreign idea that is better left to other languages?
  • necovek 4mo ago
    I don't understand your question. Are you saying static code analysis is impossible without type declarations? None of it?
• IshKebab 4mo ago
  Yes. If you have a choice. For people who don't have a choice, type checked Python is better than nothing.
• vadansky 4mo ago
  Personally because I'm making a blender add on that only uses python, and it's at the complexity where having types catches a ton of bugs easily.
• Hizonner 4mo ago
  Yes, but unfortunately Python has invaded everything, and one must adapt. Python is going to be preinstalled on almost any machine I use, with a reasonable assortment of libraries. And even if they're not preinstalled, the libraries I want are likely to exist. They'll have unstable APIs and weird quirks, and I'll have to take my choice of bad packaging systems to install them, and everything will just generally be a pain, but they'll exist and largely work. That's not true for any language I actually want to code in. I mean, I'm not going to deny that Python is better than shell scripts or (usually) C. It's not like it's a pleasant language to code in, especially if you actually want to use the type support, which is weird and irregular and keeps changing and has to work around fundamental design problems at the core of the language.
• scuff3d 4mo ago
  Seriously, just switch to Go or something
• bborud 4mo ago
  Python has other, bigger problems that make it a constant headache. One of them being the dismissive attitude towards any and all of problems that come from versioning, dependencies and quirks that make it challenging to have robustness. Criticisms are typically dismissed by suggesting heaping yet another "solution" onto the growing pile of "solutions" that you have to drag around with you. That people have to learn. That you have to install tooling for. That has to be vetted. That has to become part of the toolbox to get even seemingly simple things done. This attitude is a big part of the reason that I strongly advise people against using Python in production. On top of all the problems presented in a real-world setting. Almost all of the time, people who are fond of Python are more interested in defending python, disparaging me, downvoting me etc that listen to why I make that recommendation. (I get it. People like Python. What I think of Python as a language is irrelevant. In fact I don't have that much against it. But I do have a lot against it in a setting where you need reliability and repeatability) I have spent the last month of my life building a system that can run Python tooling reliably in a business critical application. I knew this was going to be a pretty big job when I started, but for every problem I solve, a bunch of new problems arise. I am starting to see light at the end of the tunnel but it hasn't exactly been smooth sailing. I'm almost there for a first version, but there are a bunch of problems still to solve. Mostly because I care about developer ergonomics and that things should "just work". One important goal is that my solution shouldn't impose any significant cognitive burden on people who use it. That's really hard. (I don't think the solution will be open source since my contract wouldn't allow for it. But I'll make the case at some point for why it should be open sourced) And yes. There are statically typed languages available today that have decent tooling that provides superior developer ergonomics. I can understand that people don't want to learn new languages, but if you have the capacity to do so I would recommend trying to move on if the code you write has to run outside your own workstation. If an old fart like me can learn and adopt new languages, so can you.
  • chuckadams 4mo ago
    Python the language is pretty nice. It has its warts, but I make my living in PHP which is practically made of warts. But the python ecosystem still seems to be trying to figure out this whole package management and project setup thing. In most languages I can do some form of `$blub install` where $blub is the language's official package manager or some close equivalent. It's just python that always screams at me that I have to set up and "enter" a virtualenv. I get what venv is for, but it's still a weird hack of hardlinks and relative paths that no other language seems to need, and a clumsy two-step dance of a UX that hasn't improved in like 20 years.
• UltraSane 4mo ago
  That is why I'm using C# and Rust more now than Python. You get far better RoI on types. and they are so much faster and can use all cores so much more easily.
  • pjmlp 4mo ago
    You can even add F# into the loop, with REPL and type inference much better than those two.
    • UltraSane 4mo ago
      F# is a really nice language, I just wish I could get paid for writing it.
• thrance 4mo ago
  Often, when I code in Python, it's because there are some libraries that aren't available in whatever other language would have been my first pick. Then, typing and type-checking are useful tools to stave off the codebase turning into the unruly mess that all Python projects eventually become.
  • necovek 4mo ago
    Don't projects in all languages turn into unruly mess by default? It requires special care to not let long-running projects evolve into it. Python is only special in that it is extremely productive and allows lots of easy evolutions of a project with not a care in the world, so the timeline is probably shorter on getting to the "unruly mess" if no special care is put in to make it survive many evolutions. Perhaps a bit special too in that it looks welcoming to the masses who have no idea how software systems evolve and thus do not even know there are some special patterns to introduce for code to survive many changes in the future. I am undecided if this is a pro or a con of the language itself.
• marcosdumay 4mo ago
  The goal is to be strict, with explicit exceptions. You don't use a static language because you want the exceptions, but the type checking can still statically validate most of your code.
• tptacek 4mo ago
  No? One has nothing to do with the other. I think those of us who work in compiled languages are just snooty about them. I'm a compiled language snoot, and happen to be working over the past couple days in typed Python for the first time. It's kind of nice. I like it. It's a huge improvement for me over ordinary Python/Ruby/Javascript; it materially improves the experience of working in the language.
  • tasuki 4mo ago
    > happen to be working over the past couple days in typed Python for the first time. It's kind of nice. I like it. I like me a good type system and have always hated about everything about types in Python. What do you find nice and like about it? (My experience with Python: all the type checkers are broken, there are false positives and false negatives everywhere. The LSPs are likewise broken, I have not found one that knew the types at least somewhat reliably...)
    • tptacek 4mo ago
      Lack of typing is my biggest problem with Python, Ruby, and ES6 Javascript; I have to write everything twice, once to do the stuff I want, and once to double check that it's actually doing stuff, because a single typo blow the program up despite it parsing fine. Python typing is easy to dip in and out of. It handles None nicely; not as nicely as a true Optional, but enough for daily driving. The annotations are readable and simple. What more could I ask for, without asking for an entirely different language? Python typing catches a lot of bugs I'd otherwise have to tediously unit-test for. The only thing I don't like about it is that it feels like it relies a lot on importing stuff from the swamp of the Python stdlib.
      • necovek 4mo ago
        I believe you are right to point out how you feel about some of these features of a particular language, and to let that guide your decision in how and whether you use them. It does not, however, say anything about the actual productivity gain or loss of using types in a language like Python which does not require them — that should be the ultimate objective measure of whether they make sense or not. With most languages, I get annoyed if I need to create separate types for every variant of a basic type (eg. let's have a firstNameString, familyNameString, CountryCodeString, CountryNameString... when is it too much?) - I do not think there is any way someone can prove going this deep improves maintainability long term. Eg. imagine you introduced validation of CountryCodeString based on ISO-3166, and there has been a change to ISO-3166 which happen every couple of years — how do you start supporting new codes? How do you deprecate and remove old ones? All of those are not helped with a type being very strict, you still have the persistent data to worry about, code actually supporting any of those, etc — the basic type check is trivial with a couple of small unit tests in comparison. You also quickly venture into a territory of complex types with complex interaction rules (this subvalue can be one of A if another one is X; but B if another is Y). So for me covering the basic invariants with a unit test is not much more effort and — especially with Python — does not stop one from refactoring effectively and building stable, long-running systems. Really, complex relationships in data are complex, and encoding it in a declarative way using a complex schema does not guarantee correctness (see Pydantic); if you want just very basic data conformance (type) checking, it's mostly a question of ergonomics. Basically, you need to keep to some principles of code structure and architecture, but they are a simple set of principles — perhaps the fact that most Python projects do not abide by these should be a knock against Python? I only attribute it to the approachability of Python, but I am open to being wrong and this being the latent forced idiomatic use that projects always evolve into?
• globular-toast 4mo ago
  Show us the language and we'll all switch tomorrow.
• Animats 4mo ago
  The modern approach seems to be to require full typing on items seen from outside the function or object. Within functions, have the compiler infer as much as it can. Newer languages (Go, Rust) seem to be converging on this approach. Function parameters need type info as guidance for people and LLMs calling the function. Even though cross-function type inference is technically possible, it's too confusing. Long-distance inference failures tend to generate poor messages. Within a function, if you have typed parameters, the type inference engine has a local starting point and a good chance of success on most local variables. Unchecked advisory typing in Python was a terrible idea. All the work of writing type declarations with none of the benefits.
• deleted 4mo ago
  [deleted]
• throwaway2037 4mo ago
  Shhh! You're not supposed to say that part out loud.
• lijok 4mo ago
  ML Data tooling Talent pool Libraries for customers Brownfield codebases Academics I can keep going…
• chpatrick 4mo ago
  Nothing can beat the Python numpy/ML ecosystem. There's a lot of value in just being able to run a Python script as well without any compilation step. The typing isn't perfect right now but it's usable. For vectorizable problems there also won't be huge performance gains from switching to a compiled language because all the hard stuff is already done in highly optimized native code. The only time it really makes a difference is if you have to write a custom for loop or traversal.
• fastasucan 4mo ago
  That depends on your needs and goals. Super strict type checking might not be the most important feature. Its like saying "if you want a car that is a bit sporty, wouldn't it be best to buy a Lamborghini??
  • dsign 4mo ago
    Yes, but using five type checkers on Python is the equivalent of buying a Nissan Leaf and trying to turn it into a Lamborghini by adding an internal combustion engine which is only supposed to produce a nice roar but no thrust.
• pjmlp 4mo ago
  Definitely, and statically typed language have REPLs as well. I know Python since 1.6, and it has always been mostly for OS scripting. I do see a value on it being the new BASIC, and like BASIC, building full businesses applications with it, comes with gotchas. Additionally, it appears the C libraries as "Python" libraries culture will never go away.
• gbacon 4mo ago
  If they’re using Python, performance isn’t high on their list of requirements.