4 ms·

I've worked with an O(100k) line code base in Python that was pure torture. Honestly, I was so desperate for static-typing by the end that I would have preferre

I've worked with an O(100k) line code base in Python that was pure torture. Honestly, I was so desperate for static-typing by the end that I would have preferred if it was all written in C++. Large codebases are really hard reason about without types. I'm glad we now have projects like Pyre that are trying to bring typing to Python.

ineedasername 6y ago
> desperate for static-typing Can you explain why? I honestly don't know, because my experience with C++ was during school ~20 years ago, and since then professionally I've used mostly python in relatively small codebases where it's all my own code (mostly for data processing/analysis). Thanks! (Although I did have to write some C code to glue together data in a very old legacy system that didn't support C++, much less python. It took a lot more effort to do something simple, but it was also strangely a really rewarding experience. Kind of similar to feeling when I work with assembly on hobby projects)
- johntb86 6y ago
  The main problem with duck-typing like python has is the lack of consistency between different objects that code has to work on. Different callers may pass objects with different sets of methods into a function and expect it to work. You run into the case where the object that was passed in is one with subtly-mismatched behavior from what your method expects, but you don't know who created it - it was probably stored as a member variable by something 10 callstack levels and 5 classes distant from what you're currently working on. Static typing prevents that by telling you early where the mismatch is happening - some method calls into another with a variable of the wrong type, and that's where the bug is. It also allows tooling to look up the types of variables and quickly get information about their properties.
  - ineedasername 6y ago
    Got it, that makes sense. It also makes sense why I've not much been bothered by it in python since my relative small code bases don't have that many layers of abstraction laid on top of each other. I'm generally not working with more than 2,000-3,000 lines, and I can just about keep the basic structure in my head. (Unless it's been a while since I've had to revisit it... then I often hate my past self for getting "clever" in some way)
    - valenterry 6y ago
      For these small code bases, static typing is still great (if you are used to it already) but the adverse effects of not having it usually show much stronger with a team (and not a single person). And yeah, if you keep the structure in your head, then you are good anyways.
  - d0mine 6y ago
    > it was probably stored as a member variable by something 10 callstack levels and 5 classes distant from what you're currently working on If you can define methods on an object dynamically in Python, it doesn't mean that you should. Monkeypatching is not encouraged in culturally in Python. Most often it is seen in tests otherwise, it is rare. Nobody forbids using ABCs to define your custom interfaces or using type hints for readability/IDE support/linting (my order of preference).
chii 6y ago
There's some confounding issues that are often confused together tho. large python code bases _could_ be written with well modularized, clean separation of concerns and composability. Or it could be written in spaghetti. Using types _could_ help a code base from becoming spaghetti, but it's not the only way. I think the understandability and maintainability of a code base has more to do with the person writing it than the availability of a type system tbh.
- valenterry 6y ago
  No they can't, at least not with the same amount of effort. Of course, you can make everything good by throwing enough time and money on it, but that's not the point. The issue is that to have a nice and well architected code base, you have to constantly refactor and improve - sometimes you need to re-arrange and refactor huge parts of the code. Without types _and_ tests, this is just not gonna happen. It will be unproductive and scary, so that people will start to stop touching existing code and work their way around it. > I think the understandability and maintainability of a code base has more to do with the person writing it than the availability of a type system tbh. That is the same thing. Because someone who wants great maintainability will also want a great type system (amongst other things).
  - ThaJay 6y ago
    A good carpenter never complains about his tools. He works around their limitations or uses something else. The quality of the product is down to the skill of the worker either way.
    - d0mine 6y ago
      We can assume buffer overflows are less common in Java than in C and I doubt that Java programmers are better craftsmen. The same with types: they make some kind of errors much less likely though there is no silver bullet in general e.g., I much prefer a general purpose language such as Python for expressing complex requirements in tests over any type system (even if your type system is turing complete and you can express any requirement in it; it doesn't mean it is a good idea)
xapata 6y ago
Funny, I'm working on a project about the same size, and the overly aggressive type and value restrictions are the main problem that I struggle with daily.
PyDude 6y ago
I've worked with Python for more than 15 years, usually with code bases 50-100k lines per app. The only time I have had real issues with types was a codebase I inherited where previous developers were using None, (), "", [], and {} all to mean roughly the same thing and sometimes checking for "", sometimes () etc. I couldn't handle it, so I put asserts everywhere and slowly found out where those things were coming from and sanitized it to be consistent.

ineedasername 6y ago

> desperate for static-typing Can you explain why? I honestly don't know, because my experience with C++ was during school ~20 years ago, and since then professionally I've used mostly python in relatively small codebases where it's all my own code (mostly for data processing/analysis). Thanks! (Although I did have to write some C code to glue together data in a very old legacy system that didn't support C++, much less python. It took a lot more effort to do something simple, but it was also strangely a really rewarding experience. Kind of similar to feeling when I work with assembly on hobby projects)

johntb86 6y ago
The main problem with duck-typing like python has is the lack of consistency between different objects that code has to work on. Different callers may pass objects with different sets of methods into a function and expect it to work. You run into the case where the object that was passed in is one with subtly-mismatched behavior from what your method expects, but you don't know who created it - it was probably stored as a member variable by something 10 callstack levels and 5 classes distant from what you're currently working on. Static typing prevents that by telling you early where the mismatch is happening - some method calls into another with a variable of the wrong type, and that's where the bug is. It also allows tooling to look up the types of variables and quickly get information about their properties.
- ineedasername 6y ago
  Got it, that makes sense. It also makes sense why I've not much been bothered by it in python since my relative small code bases don't have that many layers of abstraction laid on top of each other. I'm generally not working with more than 2,000-3,000 lines, and I can just about keep the basic structure in my head. (Unless it's been a while since I've had to revisit it... then I often hate my past self for getting "clever" in some way)
  - valenterry 6y ago
    For these small code bases, static typing is still great (if you are used to it already) but the adverse effects of not having it usually show much stronger with a team (and not a single person). And yeah, if you keep the structure in your head, then you are good anyways.
- d0mine 6y ago
  > it was probably stored as a member variable by something 10 callstack levels and 5 classes distant from what you're currently working on If you can define methods on an object dynamically in Python, it doesn't mean that you should. Monkeypatching is not encouraged in culturally in Python. Most often it is seen in tests otherwise, it is rare. Nobody forbids using ABCs to define your custom interfaces or using type hints for readability/IDE support/linting (my order of preference).

chii 6y ago

There's some confounding issues that are often confused together tho. large python code bases _could_ be written with well modularized, clean separation of concerns and composability. Or it could be written in spaghetti. Using types _could_ help a code base from becoming spaghetti, but it's not the only way. I think the understandability and maintainability of a code base has more to do with the person writing it than the availability of a type system tbh.

valenterry 6y ago
No they can't, at least not with the same amount of effort. Of course, you can make everything good by throwing enough time and money on it, but that's not the point. The issue is that to have a nice and well architected code base, you have to constantly refactor and improve - sometimes you need to re-arrange and refactor huge parts of the code. Without types _and_ tests, this is just not gonna happen. It will be unproductive and scary, so that people will start to stop touching existing code and work their way around it. > I think the understandability and maintainability of a code base has more to do with the person writing it than the availability of a type system tbh. That is the same thing. Because someone who wants great maintainability will also want a great type system (amongst other things).
- ThaJay 6y ago
  A good carpenter never complains about his tools. He works around their limitations or uses something else. The quality of the product is down to the skill of the worker either way.
  - d0mine 6y ago
    We can assume buffer overflows are less common in Java than in C and I doubt that Java programmers are better craftsmen. The same with types: they make some kind of errors much less likely though there is no silver bullet in general e.g., I much prefer a general purpose language such as Python for expressing complex requirements in tests over any type system (even if your type system is turing complete and you can express any requirement in it; it doesn't mean it is a good idea)

xapata 6y ago

Funny, I'm working on a project about the same size, and the overly aggressive type and value restrictions are the main problem that I struggle with daily.

PyDude 6y ago

I've worked with Python for more than 15 years, usually with code bases 50-100k lines per app. The only time I have had real issues with types was a codebase I inherited where previous developers were using None, (), "", [], and {} all to mean roughly the same thing and sometimes checking for "", sometimes () etc. I couldn't handle it, so I put asserts everywhere and slowly found out where those things were coming from and sanitized it to be consistent.