6 ms·

To be fair, a division between hell and heaven will happen with any language. The question is: is this particular hell worth the result? There is no generic a

To be fair, a division between hell and heaven will happen with any language. The question is: is this particular hell worth the result? There is no generic answer to that of course, it just happens is has been the case for me during those 20 years. First, you have to get to the industrialization phase. And of course, you have to get there, with the constraint of time, budgets, and talent. Second, Python does have less benefits for that phase than rust or haskel, but it's not impotent. Industrialization is just hard, no matter the language. Because we are an industry where a lot of self-taught people get a career, we tend to forget this last point. Creating a serious computing system is engineering, and this includes the project management part, and being rigorous about a lot of thing. Granted, the rigor needs to be higher with Python once your reach that scale, and at a certain point (which I wish everybody would reach), you may want to ditch it. It's a good problem to have though. Yet we have to remember not all industrialization attempts are equal. Most are really tame. I'd argue you could replace half the website code base out there with a bunch of bash scripts on a VPS and they would still make money. So even in this context, Python can deliver a lot.

mumblemumble 3y ago
This is roughly my experience, too. Static type safety has interesting YAGNI characteristics. For the bits of the code that must work, and where defects and regressions due to type errors may be subtle and difficult to detect, it's indispensable. But it can also be an impediment to iteration. Sometimes the code you're working on is still so experimental that you don't really know what the best structure and flow of data will be yet, and it's easier to just bodge it together with maps and heterogeneous lists for the first few iterations so you can let the code tell you how it wants to be structured. Having to start with static types subtly anchors you to your first idea. And the first is typically the worst. Something I really like about Python for this sort of thing is that I have a lot more ability to delay this industrialization stuff to the last responsible moment, and be selective in where I spend my industrialization tokens. Here's a list of the languages where I've found selectively rewriting the important bits in Rust to work well: C, C++, Python. I know which of those I'd rather use for prototyping, scripting, and high level control. Relevant Fred Brooks quote: "Plan to throw one away. You will, anyway."
- giantrobot 3y ago
  > Something I really like about Python for this sort of thing is that I have a lot more ability to delay this industrialization stuff to the last responsible moment This is one of my favorite aspects of Python. I can start with every module in prototype form and industrialize each module as its design firms up. I can spend my early development getting an idea fleshed out with minimal overhead.
  - Yoric 3y ago
    > I can start with every module in prototype form and industrialize each module as its design firms up. That is definitely the theory. And this flexibility is indeed very precious for some types of work. However... does it actually happen as you describe? I can count on half of the fingers of one hand the number of Python codebases that I've seen that actually feel like they've properly been reworked into something of industrial quality. All the other codebases I've seen are of the "I guess it works, maybe?" persuasion, quite possibly because there is always something higher priority than quality.
    - duggan 3y ago
      Something that just about seems to work is something that has often met the threshold for "solves a real problem" while also meeting the other requirements that a product needs to be successful. It's easy to make something that works, is well designed, and either doesn't solve a problem someone has or nobody knows about it.
      - Yoric 3y ago
        I realize that this is the common wisdom these days: write code that works just well enough that we have a chance to fix before it does too much damage whenever it breaks. I suspect that this approach is strongly fueled by the unlimited VC money available in tech, since it means that any company can employ an unlimited number of full-time developers (and PR) just to handle catastrophes. We'll see how that wisdom holds if/when/as VC money dries up and/or moves to other sectors.
havercosine 3y ago
Wholeheartedly agree. I'm a self taught programmer who programs to get things done rather than just doing programming for fun (I started in ML/data science field). That way, my code often resembles the caricature of these hacky codebases alluded above. And I'm well aware of it. I often have arguments about utility of Python with my more technically solid engineer friends who keep telling me "Python doesn't scale" etc. And I often come back to the same point you highlighted: is the particular hell worth the result? For 85-90% of the cases the answer is resounding yes. For nimble teams, startup projects, internal tools : most of the code is often thrown away eventually. Python's beauty is that the language quickly gets out of the way. Everyone then is forced to focus on complexity of the business domain, "does the code solve the business problem". The architecture/scale/speed problems will eventually arrive with Python. But most purist engineers overestimate how soon it will come. At most of the failed startup attempts I was part of, the PMF problem was more pressing to solve than software constraints. Early days of YouTube, Dropbox, Instagram (and maybe OpenAI too) are big testament to this. I've made my peace these days by not fighting to prove Python is the best language etc. If someone tells me "Rust beats Python" kind of argument: I say I agree, wish them luck and focus on shipping things. tl:dr; Python is still the best choice to "quickly deliver value and test your business hypothesis".
- ReflectedImage 3y ago
  Python scales fine as long as you know microservices. The issue is people assuming that they don't need to learn anything new to use Python. You got people stating that using static typing in a dynamically typed language like Python is a good or reasonable idea. It's not. But people don't want to put in the effort to learn things like dynamic typing and microservices.
  - Yoric 3y ago
    How do microservices factor in this conversation? Do you find that they actually reduce complexity?

mumblemumble 3y ago

This is roughly my experience, too. Static type safety has interesting YAGNI characteristics. For the bits of the code that must work, and where defects and regressions due to type errors may be subtle and difficult to detect, it's indispensable. But it can also be an impediment to iteration. Sometimes the code you're working on is still so experimental that you don't really know what the best structure and flow of data will be yet, and it's easier to just bodge it together with maps and heterogeneous lists for the first few iterations so you can let the code tell you how it wants to be structured. Having to start with static types subtly anchors you to your first idea. And the first is typically the worst. Something I really like about Python for this sort of thing is that I have a lot more ability to delay this industrialization stuff to the last responsible moment, and be selective in where I spend my industrialization tokens. Here's a list of the languages where I've found selectively rewriting the important bits in Rust to work well: C, C++, Python. I know which of those I'd rather use for prototyping, scripting, and high level control. Relevant Fred Brooks quote: "Plan to throw one away. You will, anyway."

giantrobot 3y ago
> Something I really like about Python for this sort of thing is that I have a lot more ability to delay this industrialization stuff to the last responsible moment This is one of my favorite aspects of Python. I can start with every module in prototype form and industrialize each module as its design firms up. I can spend my early development getting an idea fleshed out with minimal overhead.
- Yoric 3y ago
  > I can start with every module in prototype form and industrialize each module as its design firms up. That is definitely the theory. And this flexibility is indeed very precious for some types of work. However... does it actually happen as you describe? I can count on half of the fingers of one hand the number of Python codebases that I've seen that actually feel like they've properly been reworked into something of industrial quality. All the other codebases I've seen are of the "I guess it works, maybe?" persuasion, quite possibly because there is always something higher priority than quality.
  - duggan 3y ago
    Something that just about seems to work is something that has often met the threshold for "solves a real problem" while also meeting the other requirements that a product needs to be successful. It's easy to make something that works, is well designed, and either doesn't solve a problem someone has or nobody knows about it.
    - Yoric 3y ago
      I realize that this is the common wisdom these days: write code that works just well enough that we have a chance to fix before it does too much damage whenever it breaks. I suspect that this approach is strongly fueled by the unlimited VC money available in tech, since it means that any company can employ an unlimited number of full-time developers (and PR) just to handle catastrophes. We'll see how that wisdom holds if/when/as VC money dries up and/or moves to other sectors.

havercosine 3y ago

Wholeheartedly agree. I'm a self taught programmer who programs to get things done rather than just doing programming for fun (I started in ML/data science field). That way, my code often resembles the caricature of these hacky codebases alluded above. And I'm well aware of it. I often have arguments about utility of Python with my more technically solid engineer friends who keep telling me "Python doesn't scale" etc. And I often come back to the same point you highlighted: is the particular hell worth the result? For 85-90% of the cases the answer is resounding yes. For nimble teams, startup projects, internal tools : most of the code is often thrown away eventually. Python's beauty is that the language quickly gets out of the way. Everyone then is forced to focus on complexity of the business domain, "does the code solve the business problem". The architecture/scale/speed problems will eventually arrive with Python. But most purist engineers overestimate how soon it will come. At most of the failed startup attempts I was part of, the PMF problem was more pressing to solve than software constraints. Early days of YouTube, Dropbox, Instagram (and maybe OpenAI too) are big testament to this. I've made my peace these days by not fighting to prove Python is the best language etc. If someone tells me "Rust beats Python" kind of argument: I say I agree, wish them luck and focus on shipping things. tl:dr; Python is still the best choice to "quickly deliver value and test your business hypothesis".

ReflectedImage 3y ago
Python scales fine as long as you know microservices. The issue is people assuming that they don't need to learn anything new to use Python. You got people stating that using static typing in a dynamically typed language like Python is a good or reasonable idea. It's not. But people don't want to put in the effort to learn things like dynamic typing and microservices.
- Yoric 3y ago
  How do microservices factor in this conversation? Do you find that they actually reduce complexity?