6 ms·

I’m in the midst of a python project right now - I’m fairly familiar with python but I haven’t used in a while, preferring TS. I’m currently majorly kicking mys

by jakear 6y ago

I’m in the midst of a python project right now - I’m fairly familiar with python but I haven’t used in a while, preferring TS. I’m currently majorly kicking myself for not starting out with mypy. Unfortunately even with mypy, soundness guarantees are nowhere near as powerful as TS, and the typing ecosystem is lacking (no definitely-typed equivalent i know of), so working with libraries is pretty much just guesswork. I really have no clue why people love this language so much.

pansa2 6y ago
In my experience, the people who love Python are those who are happy with dynamic typing - usually because they only use the language for small projects. When starting a large project, it would be better to skip Python altogether and use a proper statically-typed language. Type hints only make sense when you’re working on an existing large Python project - just as Guido was doing at Dropbox when he added them to the language. This is because type annotations are the worst-of-both-worlds - they require the verbosity of static typing and provide few of its benefits. > This is not a course for software engineers on how to write or maintain a one-million line Python application. I don't write programs like that, nor do most companies who use Python, and neither should you.
- ramraj07 6y ago
  If you already know your project is going to be as big as Dropbox's entire codebase, salut don Corleone! But otherwise, I feel like trying to choose the language that scales to a million lines sounds like dooming in the beginning itself by over-engineering. I attempted that myself but kept coming back to python because it's fast to code, it's forgiving as hell and its customizable. Once you're used to it's quirks you can breeze through it fast. I'm sure ruby and js have the same as well, and with ts you get better typing, but with the typing you can do with 3.8, I'll argue that python gives the _best_ of both worlds if it's especially mixed with a good IDE like pycharm. If you really want, you can also incorporate pytype or something from the get-go. If my Greenfield project's biggest problem is that python doesn't scale, then I'm a very happy man.
  - flyinglizard 6y ago
    Python doesn’t scale well beyond one file, one developer, one time write. Once a program needs maintaining, updating, expanding, debugging - the experience quickly deteriorates.
    - ramraj07 6y ago
      The codebase I work on now is python and I prefer it to both the java and scala codebases I've worked on before that were much smaller. Python gives a better debugging experience than java and a better updating experience than scala (especially someone else's scala) in my opinion.
- at_a_remove 6y ago
  I guess I should ask what you consider a small project. I have some five-digit LOC projects written entirely in Python. I have never noticed this typing issue everyone is very concerned about and am often baffled by the emphasis placed upon it. What am I missing? I am willing to entertain the idea that I have been very lucky or that I have some programming mannerism which has caused me to skate by this kind of thing, but I just don't get it.
  - ed25519FUUU 6y ago
    Are you the only one who works on the project? You might not notice it in that case, but the engineer who comes after you will.
    - at_a_remove 6y ago
      The only times I have heard from people who have worked on projects after me is the odd phone call to thank me for the documentation and the clarity of code, just as an intro to ask me a "what should we do about this?" or some kind of obscure historical follow-up. Case in point, I had written an apartment search website in Perl. Now, at this point I had been pretty disenchanted with Perl as a whole due to the "there's more than one way to do it" culture combining with the "executable line noise" syntax to give rise to a lot of very impenetrable write-once, read-never-again code from others. So, when I did this, I used real, appropriately-named variables, eschewing the convenience of $_, and I made sure each line of code did one thing and only one thing if at all possible. I did not pack a lot into a line. Each line should be obvious. Where I felt that it might be subject to interpretation, I added comments as to what I was doing and why. Each function had is own comment section which discussed what it was for and why, possible room for improvement, and so on. Eventually the duty passed to another set of hands, a bunch of students who had never seen Perl (the rise of PHP was strong at that point) and in the follow-up, they mentioned that they found it very straightforward to simply re-write each line. I heard back in the past few months about one of those very large projects I did in a previous job, just keeping touch with people. I asked about one of my personal pet projects. "No, it just keeps running." "It's very obvious where to make any changes." All Python. No type stuff. I just don't see what I am missing.
rxhernandez 6y ago
> I really have no clue why people love this language so much. You get used to it somehow. I did C and Java for about 10 years before Python. My first exposure to Python was a Computational Physics class and it was maddening not knowing what types went where. Eventually you realize it doesn't really matter as long as you're disciplined and rely on others who are disciplined enough to document their code well. I haven't had the chance to use mypy in the 7 years I've been using Python (across 3 industries; medical devices, scientific devices, and web) but I would be surprised if it significantly reduced the number of bugs I've seen in practice.
- jakear 6y ago
  It’s certainly possible to write out types in English and cross your fingers and hope they’re correct and stay correct as the file gets edited by more and more people, but why do that when you can do it in a language that an intelligent type checker can understand and validate, and further use to provide smart editor tab-completions/etc? This is especially big for any refactoring work - just yesterday I had to change the format of a config file that gets read by both TS and Python scripts. For ts, I updated the typedef and I instantly got every error in the codebase annotated. For python, I had to manually go through each line, using my own brain as the type checker. My brain is much faster and better at being a “squiggly red line spotter” than a type checker, so I finished my TS work in a few minutes and took probably an hour going through the python, despite the projects being similar in size.
  - the_af 6y ago
    This. My experience with Python goes like this: "I have to write a really small script... ok, not so small that I can do it with bash. I know, I'll use Python! This is going to be a small, throwaway utility anyway" (Some weeks later, it's turned into a large monstrosity and I need to refactor something, and everything breaks because refactoring anything nontrivial written in Python is a dangerous proposition) "Ouch! Python: never again!" (Repeat a few weeks later)
    - disgruntledphd2 6y ago
sooheon 6y ago
> why people love this language so much Because no matter what problem you have, after 12 minutes of googling, you can pip install foo. The language almost doesn't matter because most programming is via library api.
- jakear 6y ago
  I can do the same with npm, but if there’s a @types available (or better yet built in), I barely have to read long-form documentation and I get smart completions with documentation, types, etc right in my editor, specific to the exact expression I’m editing.
  - dragonwriter 6y ago
    And...exactly the same with mypy, either via in-library typings or the typeshed. Projects in the npm ecosystem may be somewhat more likely to.have typings if the project exists, but I find that, for anything other than web frontend, where JS is obviously king, the tool I'm looking for is more likely to actually exist in the python ecosystem.
    - jakear 6y ago
      Are there mypy typings available for things like sci-py and simpletransformers, and if so how do I acquire them?
      - deleted 6y ago
        [deleted]
      - ghostwriter 6y ago
        You can gradually annotate these packages locally for your project with [1], and then contribute it to [2]. Automatic stub generation may also be helpful [3] [1] https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules#examples https://github.com/python/mypy/wiki/Creating-Stubs-For-Pytho... [2] https://github.com/python/typeshed https://github.com/python/typeshed [3] https://mypy.readthedocs.io/en/stable/stubgen.html https://mypy.readthedocs.io/en/stable/stubgen.html
carapace 6y ago
Back in the day, coming from C, Python was magic. (And it wasn't C++ or Java.)
yodelshady 6y ago
It's unquestionably more for short scripts - what I mostly do - instead of big applications. My memory is of learning Java, where you're all but forced to create custom classes for everything, just in case you need extensibility later. With Python? It can .quack(), and that's all I should care about. I understand generics have helped here, but they still don't seem quite there. And ironically, I'm finding ML tasks in python to be something that could really benefit from type hints. Side note, and this bugs me: if people love it - empirically, they do - and you don't understand why - surely that should bring about some introspection? It seems all-too-often to bring about the opposite reflex. Badly indented python code doesn't run, which it shouldn't - my understanding and the machine's are different. Well-indented python code has a lot less visual noise than other languages. That alone should give pause for thought. Why would you not want that feature?
- jakear 6y ago
  Your comparison seems entirely about python v Java. I encourage you to check out some modern gradually types languages to help understand what Python lacks. TypeScript is particularly good for this. > Why would you not want that feature? Easy. Braces/etc. become basically invisible after using the language for a bit so they don’t bother me whatsoever. However moving code around in Python is always a pain because you must make sure everything is placed at the correct indentation level rather than simply letting the formatter take care of it. I’ve definitely experienced bugs from moved code having its first/last line at the wrong level, and it can be particularly confusing when the last line is at a different indentation than the previous. It’s so much easier to just grab a brace-enclosed block and smack it down somewhere (which also provides a good sanity check that you’ve yanked the entire block and aren’t missing any lines). > Badly indented python code doesn't run I wish this were the case. What’s actually the case is that badly indented python code will give you potentially different results than what you expect, which ranges from syntax errors to failed tests to very hard to diagnose bugs. > Side note, and this bugs me: if people love it - empirically, they do - and you don't understand why - surely that should bring about some introspection? Side note, and this bugs me: if people hate it - empirically, they do - and you don't understand why - surely that should bring about some introspection?
leu-mas 6y ago
I've been really enjoying pydantic lately. Run-time validation of types without forcing you to learn something other than normal type annotations. https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/

by jakear 6y ago

pansa2 6y ago

In my experience, the people who love Python are those who are happy with dynamic typing - usually because they only use the language for small projects. When starting a large project, it would be better to skip Python altogether and use a proper statically-typed language. Type hints only make sense when you’re working on an existing large Python project - just as Guido was doing at Dropbox when he added them to the language. This is because type annotations are the worst-of-both-worlds - they require the verbosity of static typing and provide few of its benefits. > This is not a course for software engineers on how to write or maintain a one-million line Python application. I don't write programs like that, nor do most companies who use Python, and neither should you.

ramraj07 6y ago
If you already know your project is going to be as big as Dropbox's entire codebase, salut don Corleone! But otherwise, I feel like trying to choose the language that scales to a million lines sounds like dooming in the beginning itself by over-engineering. I attempted that myself but kept coming back to python because it's fast to code, it's forgiving as hell and its customizable. Once you're used to it's quirks you can breeze through it fast. I'm sure ruby and js have the same as well, and with ts you get better typing, but with the typing you can do with 3.8, I'll argue that python gives the _best_ of both worlds if it's especially mixed with a good IDE like pycharm. If you really want, you can also incorporate pytype or something from the get-go. If my Greenfield project's biggest problem is that python doesn't scale, then I'm a very happy man.
- flyinglizard 6y ago
  Python doesn’t scale well beyond one file, one developer, one time write. Once a program needs maintaining, updating, expanding, debugging - the experience quickly deteriorates.
  - ramraj07 6y ago
    The codebase I work on now is python and I prefer it to both the java and scala codebases I've worked on before that were much smaller. Python gives a better debugging experience than java and a better updating experience than scala (especially someone else's scala) in my opinion.
at_a_remove 6y ago
I guess I should ask what you consider a small project. I have some five-digit LOC projects written entirely in Python. I have never noticed this typing issue everyone is very concerned about and am often baffled by the emphasis placed upon it. What am I missing? I am willing to entertain the idea that I have been very lucky or that I have some programming mannerism which has caused me to skate by this kind of thing, but I just don't get it.
- ed25519FUUU 6y ago
  Are you the only one who works on the project? You might not notice it in that case, but the engineer who comes after you will.
  - at_a_remove 6y ago
    The only times I have heard from people who have worked on projects after me is the odd phone call to thank me for the documentation and the clarity of code, just as an intro to ask me a "what should we do about this?" or some kind of obscure historical follow-up. Case in point, I had written an apartment search website in Perl. Now, at this point I had been pretty disenchanted with Perl as a whole due to the "there's more than one way to do it" culture combining with the "executable line noise" syntax to give rise to a lot of very impenetrable write-once, read-never-again code from others. So, when I did this, I used real, appropriately-named variables, eschewing the convenience of $_, and I made sure each line of code did one thing and only one thing if at all possible. I did not pack a lot into a line. Each line should be obvious. Where I felt that it might be subject to interpretation, I added comments as to what I was doing and why. Each function had is own comment section which discussed what it was for and why, possible room for improvement, and so on. Eventually the duty passed to another set of hands, a bunch of students who had never seen Perl (the rise of PHP was strong at that point) and in the follow-up, they mentioned that they found it very straightforward to simply re-write each line. I heard back in the past few months about one of those very large projects I did in a previous job, just keeping touch with people. I asked about one of my personal pet projects. "No, it just keeps running." "It's very obvious where to make any changes." All Python. No type stuff. I just don't see what I am missing.

rxhernandez 6y ago

> I really have no clue why people love this language so much. You get used to it somehow. I did C and Java for about 10 years before Python. My first exposure to Python was a Computational Physics class and it was maddening not knowing what types went where. Eventually you realize it doesn't really matter as long as you're disciplined and rely on others who are disciplined enough to document their code well. I haven't had the chance to use mypy in the 7 years I've been using Python (across 3 industries; medical devices, scientific devices, and web) but I would be surprised if it significantly reduced the number of bugs I've seen in practice.

jakear 6y ago
It’s certainly possible to write out types in English and cross your fingers and hope they’re correct and stay correct as the file gets edited by more and more people, but why do that when you can do it in a language that an intelligent type checker can understand and validate, and further use to provide smart editor tab-completions/etc? This is especially big for any refactoring work - just yesterday I had to change the format of a config file that gets read by both TS and Python scripts. For ts, I updated the typedef and I instantly got every error in the codebase annotated. For python, I had to manually go through each line, using my own brain as the type checker. My brain is much faster and better at being a “squiggly red line spotter” than a type checker, so I finished my TS work in a few minutes and took probably an hour going through the python, despite the projects being similar in size.
- the_af 6y ago
  This. My experience with Python goes like this: "I have to write a really small script... ok, not so small that I can do it with bash. I know, I'll use Python! This is going to be a small, throwaway utility anyway" (Some weeks later, it's turned into a large monstrosity and I need to refactor something, and everything breaks because refactoring anything nontrivial written in Python is a dangerous proposition) "Ouch! Python: never again!" (Repeat a few weeks later)
  - disgruntledphd2 6y ago

sooheon 6y ago

> why people love this language so much Because no matter what problem you have, after 12 minutes of googling, you can pip install foo. The language almost doesn't matter because most programming is via library api.

jakear 6y ago
I can do the same with npm, but if there’s a @types available (or better yet built in), I barely have to read long-form documentation and I get smart completions with documentation, types, etc right in my editor, specific to the exact expression I’m editing.
- dragonwriter 6y ago
  And...exactly the same with mypy, either via in-library typings or the typeshed. Projects in the npm ecosystem may be somewhat more likely to.have typings if the project exists, but I find that, for anything other than web frontend, where JS is obviously king, the tool I'm looking for is more likely to actually exist in the python ecosystem.
  - jakear 6y ago
    Are there mypy typings available for things like sci-py and simpletransformers, and if so how do I acquire them?
    - deleted 6y ago
      [deleted]
    - ghostwriter 6y ago
      You can gradually annotate these packages locally for your project with [1], and then contribute it to [2]. Automatic stub generation may also be helpful [3] [1] https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules#examples https://github.com/python/mypy/wiki/Creating-Stubs-For-Pytho... [2] https://github.com/python/typeshed https://github.com/python/typeshed [3] https://mypy.readthedocs.io/en/stable/stubgen.html https://mypy.readthedocs.io/en/stable/stubgen.html

carapace 6y ago

Back in the day, coming from C, Python was magic. (And it wasn't C++ or Java.)

yodelshady 6y ago

It's unquestionably more for short scripts - what I mostly do - instead of big applications. My memory is of learning Java, where you're all but forced to create custom classes for everything, just in case you need extensibility later. With Python? It can .quack(), and that's all I should care about. I understand generics have helped here, but they still don't seem quite there. And ironically, I'm finding ML tasks in python to be something that could really benefit from type hints. Side note, and this bugs me: if people love it - empirically, they do - and you don't understand why - surely that should bring about some introspection? It seems all-too-often to bring about the opposite reflex. Badly indented python code doesn't run, which it shouldn't - my understanding and the machine's are different. Well-indented python code has a lot less visual noise than other languages. That alone should give pause for thought. Why would you not want that feature?

jakear 6y ago
Your comparison seems entirely about python v Java. I encourage you to check out some modern gradually types languages to help understand what Python lacks. TypeScript is particularly good for this. > Why would you not want that feature? Easy. Braces/etc. become basically invisible after using the language for a bit so they don’t bother me whatsoever. However moving code around in Python is always a pain because you must make sure everything is placed at the correct indentation level rather than simply letting the formatter take care of it. I’ve definitely experienced bugs from moved code having its first/last line at the wrong level, and it can be particularly confusing when the last line is at a different indentation than the previous. It’s so much easier to just grab a brace-enclosed block and smack it down somewhere (which also provides a good sanity check that you’ve yanked the entire block and aren’t missing any lines). > Badly indented python code doesn't run I wish this were the case. What’s actually the case is that badly indented python code will give you potentially different results than what you expect, which ranges from syntax errors to failed tests to very hard to diagnose bugs. > Side note, and this bugs me: if people love it - empirically, they do - and you don't understand why - surely that should bring about some introspection? Side note, and this bugs me: if people hate it - empirically, they do - and you don't understand why - surely that should bring about some introspection?

leu-mas 6y ago

I've been really enjoying pydantic lately. Run-time validation of types without forcing you to learn something other than normal type annotations. https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/