6 ms·

I can't articulate specifically why but using typing in Python just feels like so much pain compared to other languages that have "opt-in" nominal typing syntax

I can't articulate specifically why but using typing in Python just feels like so much pain compared to other languages that have "opt-in" nominal typing syntax (PHP et al.). The workflow feels frustrating, the ecosystem seems diverse and has no clear "blessed" path, and I'm still confused about what is bundled by Python and what I need to pull in from an external source. I REALLY want to use mypy but by the time I've figured out how to pull it all together I probably could have finished the program I'm working on. The relevant factor here might be the size of Python programs I typically work on, somewhere between a few hundred lines and a few thousand. I'm glad other people are having success because hopefully that'll smooth the pavement for the next time I circle around and try to add some meaningful types to my Python programs.

chc 4y ago
I think something that might help you understand a little better is this: Python includes everything you need to write type hints, but it doesn't include any type checking functionality. They're making a dynamic language where you can write types if you want, but they don't really do anything. To check your types, you need to use a third-party typechecker like Mypy.
- kungfufrog 4y ago
  I appreciate your response but I'm actually a pretty experienced Python programmer. I get it conceptually, I've even used it in practice just to get a feel for it, I just found the whole experience _painful_!
  - WesolyKubeczek 4y ago
    The gods intend that you use those annotations with a checker and some IDE that supports autocompletions and highlighting based on those. They basically lure you into annotating types all over the place so that their products can work better. When you work in an editor that doesn’t support those fancy things, you’ll feel exhausted pretty soon.
ReflectedImage 4y ago
Using type checking in Python is always the wrong thing to do. You need to learn to use Microservices where you split up a large code base into seperate 3k lines Microservice chunks of code that interact over well documented APIs. That's just how effective Python is done.
- codekansas 4y ago
  I don't think you're understanding their use case, they aren't building some Flask service, they're an ML drug discovery company supporting research scientists and as such they're more or less building a framework. There's a lot of code reuse and most large orgs building similar frameworks have adopted similar code quality standards - think PyTorch, Fairseq, Pytorch Lightning, Huggingface - which is why tools like MyPy / PyLint / Black exist. For people who have used those libraries there is a night-and-day difference between the anything-goes ones and the ones that do linting and unit testing.
  - ReflectedImage 4y ago
    In my personal experience codebases using MyPy tend to be considerably worse than codebases that don't. By using a static type checker on a dynamically typed language, you have admitted you don't know what are you doing right of the bat. This means the software engineers in the project are very bad and therefore the code quality overall will also be bad. Tools like MyPy exist to make Python appear to be more like Java to help Java developers, who aren't willing to learn how to code in a different programming paradigm. That code will always be far worse than Python code written using the Python development paradigms.
    - kortex 4y ago
      What are some examples of popular python libraries that use this ideal python development paradigm?
      - ReflectedImage 4y ago
        Most of the Python libraries that are not locked to specific versions of Python, which is often the case with a lot of the badly written ML libraries.
goodoldneon 4y ago
Personally, I like type hints at any program size. I do understand the benefit isn’t as obvious in small programs. But the value skyrockets when you have a large codebase that you can’t keep in your head. All that said, Mypy is far worse than TypeScript. I think Mypy would be so much better if it improved support around dict inference. It shouldn’t just say “I dunno, it’s a dict of strings?” but rather infer an anonymous TypedDict
whoopdeepoo 4y ago
It certainly still feels half baked. Ironically the best type checker imo, pyright, is written in typescript.
- kortex 4y ago
  I love me some python, it's my favorite and most-versed language by miles, but typescript definitely gets the W when it comes to its type system. Idk exactly why - both js and python "bolted on" their type systems (syntactic superset), both are highly dynamic, yet python really struggles with typing systems and JITing. I think python is simply "more dynamic" than js. Js has a much simpler data model.
  - RussianCow 4y ago
    > I think python is simply "more dynamic" than js. Js has a much simpler data model. I don't think it has much to do with that. There are some major differences between TypeScript and typed Python that explain why TS is so successful and typed Python is not: 1. TypeScript is its own programming language with its own syntax. Even though it's a superset of JavaScript, this still matters. The only special, type-related syntax Python supports is type annotations; defining and otherwise using types is done via Python's normal syntax, which is not very ergonomic or intuitive. 2. Microsoft invested a ton of engineering effort into TypeScript. The Python ecosystem simply has not received that large of an investment in static typing. 3. The Python programming language does not dictate how type checking should be done, so engineering effort is scattered across several projects like MyPy and Pyright. By contrast, TypeScript has essentially "won" that battle on the JavaScript side. (I know Flow exists, but I don't know anyone who uses it for new projects today—everything is done with TS.)
throwaway343244 4y ago
matsemann 4y ago
MyPy is good, and works as expected and similar to how typescript and flow works for javascript. The problem with Python is that most code is shit. Sorry, but "fighting the compiler" because you want to return different types based on some flag isn't the compiler being annoying, it's the compiler exposing bad code. It's not until before summer typing for django actually started to work somewhat. All stubs did was to make mypy not complain, but not until now it actually can catch some errors. Getting a python project typed is a world of pain, and in the end you can't really trust it anyways because of all the hacks making the type-checker happy.
- RussianCow 4y ago
  > MyPy is good, and works as expected and similar to how typescript and flow works for javascript. Last time I used MyPy (1.5-2 years ago?) it wasn't nearly as full-featured or ergonomic as TypeScript. Has that changed recently? I remember running into lots of cases that it didn't support that were easy in TS, although it was long enough ago that I don't have specific examples.
IshKebab 4y ago
There are a few reasons why it sucks so much: 1. Most of the Python community doesn't really get the importance of static typing. As a result tons of dependencies completely lack typing, or are written in a dynamic way that is incompatible with static typing. 2. You have to go to some significant effort to set up static type checking in CI, and because of the above issue getting 0 errors is much harder than in e.g. Typescript. 3. The semantics of type hints are not defined so there are multiple incompatible interpretations. Better hope your dependencies use the same type checker as you! 4. MyPy is way better than nothing but it's actually pretty buggy and absolutely full of unsoundness and legacy hacks (not that surprising given its age). I would strongly recommend using Pyright if you make the unfortunate mistake of starting a new project in Python. It is much better than MyPy and the author is some kind of bug fixing robot.
urthor 4y ago
> I can't articulate specifically why but using typing in Python just feels like so much pain compared to other languages that have "opt-in" nominal typing syntax (PHP et al.). Cognitive Load.

chc 4y ago

I think something that might help you understand a little better is this: Python includes everything you need to write type hints, but it doesn't include any type checking functionality. They're making a dynamic language where you can write types if you want, but they don't really do anything. To check your types, you need to use a third-party typechecker like Mypy.

kungfufrog 4y ago
I appreciate your response but I'm actually a pretty experienced Python programmer. I get it conceptually, I've even used it in practice just to get a feel for it, I just found the whole experience _painful_!
- WesolyKubeczek 4y ago
  The gods intend that you use those annotations with a checker and some IDE that supports autocompletions and highlighting based on those. They basically lure you into annotating types all over the place so that their products can work better. When you work in an editor that doesn’t support those fancy things, you’ll feel exhausted pretty soon.

ReflectedImage 4y ago

Using type checking in Python is always the wrong thing to do. You need to learn to use Microservices where you split up a large code base into seperate 3k lines Microservice chunks of code that interact over well documented APIs. That's just how effective Python is done.

codekansas 4y ago
I don't think you're understanding their use case, they aren't building some Flask service, they're an ML drug discovery company supporting research scientists and as such they're more or less building a framework. There's a lot of code reuse and most large orgs building similar frameworks have adopted similar code quality standards - think PyTorch, Fairseq, Pytorch Lightning, Huggingface - which is why tools like MyPy / PyLint / Black exist. For people who have used those libraries there is a night-and-day difference between the anything-goes ones and the ones that do linting and unit testing.
- ReflectedImage 4y ago
  In my personal experience codebases using MyPy tend to be considerably worse than codebases that don't. By using a static type checker on a dynamically typed language, you have admitted you don't know what are you doing right of the bat. This means the software engineers in the project are very bad and therefore the code quality overall will also be bad. Tools like MyPy exist to make Python appear to be more like Java to help Java developers, who aren't willing to learn how to code in a different programming paradigm. That code will always be far worse than Python code written using the Python development paradigms.
  - kortex 4y ago
    What are some examples of popular python libraries that use this ideal python development paradigm?
    - ReflectedImage 4y ago
      Most of the Python libraries that are not locked to specific versions of Python, which is often the case with a lot of the badly written ML libraries.

goodoldneon 4y ago

Personally, I like type hints at any program size. I do understand the benefit isn’t as obvious in small programs. But the value skyrockets when you have a large codebase that you can’t keep in your head. All that said, Mypy is far worse than TypeScript. I think Mypy would be so much better if it improved support around dict inference. It shouldn’t just say “I dunno, it’s a dict of strings?” but rather infer an anonymous TypedDict

whoopdeepoo 4y ago

It certainly still feels half baked. Ironically the best type checker imo, pyright, is written in typescript.

kortex 4y ago
I love me some python, it's my favorite and most-versed language by miles, but typescript definitely gets the W when it comes to its type system. Idk exactly why - both js and python "bolted on" their type systems (syntactic superset), both are highly dynamic, yet python really struggles with typing systems and JITing. I think python is simply "more dynamic" than js. Js has a much simpler data model.
- RussianCow 4y ago
  > I think python is simply "more dynamic" than js. Js has a much simpler data model. I don't think it has much to do with that. There are some major differences between TypeScript and typed Python that explain why TS is so successful and typed Python is not: 1. TypeScript is its own programming language with its own syntax. Even though it's a superset of JavaScript, this still matters. The only special, type-related syntax Python supports is type annotations; defining and otherwise using types is done via Python's normal syntax, which is not very ergonomic or intuitive. 2. Microsoft invested a ton of engineering effort into TypeScript. The Python ecosystem simply has not received that large of an investment in static typing. 3. The Python programming language does not dictate how type checking should be done, so engineering effort is scattered across several projects like MyPy and Pyright. By contrast, TypeScript has essentially "won" that battle on the JavaScript side. (I know Flow exists, but I don't know anyone who uses it for new projects today—everything is done with TS.)

throwaway343244 4y ago

matsemann 4y ago

MyPy is good, and works as expected and similar to how typescript and flow works for javascript. The problem with Python is that most code is shit. Sorry, but "fighting the compiler" because you want to return different types based on some flag isn't the compiler being annoying, it's the compiler exposing bad code. It's not until before summer typing for django actually started to work somewhat. All stubs did was to make mypy not complain, but not until now it actually can catch some errors. Getting a python project typed is a world of pain, and in the end you can't really trust it anyways because of all the hacks making the type-checker happy.

RussianCow 4y ago
> MyPy is good, and works as expected and similar to how typescript and flow works for javascript. Last time I used MyPy (1.5-2 years ago?) it wasn't nearly as full-featured or ergonomic as TypeScript. Has that changed recently? I remember running into lots of cases that it didn't support that were easy in TS, although it was long enough ago that I don't have specific examples.

IshKebab 4y ago

There are a few reasons why it sucks so much: 1. Most of the Python community doesn't really get the importance of static typing. As a result tons of dependencies completely lack typing, or are written in a dynamic way that is incompatible with static typing. 2. You have to go to some significant effort to set up static type checking in CI, and because of the above issue getting 0 errors is much harder than in e.g. Typescript. 3. The semantics of type hints are not defined so there are multiple incompatible interpretations. Better hope your dependencies use the same type checker as you! 4. MyPy is way better than nothing but it's actually pretty buggy and absolutely full of unsoundness and legacy hacks (not that surprising given its age). I would strongly recommend using Pyright if you make the unfortunate mistake of starting a new project in Python. It is much better than MyPy and the author is some kind of bug fixing robot.

urthor 4y ago

> I can't articulate specifically why but using typing in Python just feels like so much pain compared to other languages that have "opt-in" nominal typing syntax (PHP et al.). Cognitive Load.