2 ms·
As someone with nearly two decades of experience with Python and who has deployed it in production in high impact scenarios, I’d like to nuance the above for p
by wenc 5y ago
As someone with nearly two decades of experience with Python and who has deployed it in production in high impact scenarios, I’d like to nuance the above for people who think Python is only good for prototyping and that you have to rewrite in a “proper” programming language. This is the kind of blanket thinking to avoid.
I think the rewriting part is only necessary if there’s some characteristic in your use case that you need that Python doesn’t provide — like raw computational speed, low latencies etc. If you’re a senior engineer familiar with Python, these trade offs are always front of mind.
For the majority of software I’ve developed, these characteristics were not essential. Python was a good choice for 80-90% of all of the important software products I’ve ever written.
For many data science projects, maintaining large code bases in Python is often the optimal decision (rather than reinventing the wheel and writing your own data frame and machine learning libraries, or using immature poorly maintained ones in other languages — for data manipulation and scientific algorithms, these libraries are highly optimized in Python anyway since the underlying code is in C or Fortran). Python is also a good choice for data engineering pipelines.
Where I might hesitate to recommend Python is when you have to write web services or desktop apps or any kind of application which requires speed or scale that cannot be handed down to a lower level library in Python.
Also for very large code bases, static typing truly helps to keep things sane, especially when you have different teams working on different parts of the codebase — with statically typed languages, no type checks in unit tests are needed, and refactoring is much more solid and error free. Python’s type annotations are an attempt to move in this direction, but static languages truly excel at type integrity (which are sometimes the cause of subtle errors in Python).
Otherwise my experience has often been that Python is a good first or second choice, depending on what you’re doing. Making that choice correctly is what sets senior/principal engineers apart from junior folks.
- takeda 5y ago> Python’s type annotations are an attempt to move in this direction, but static languages truly excel at type integrity (which are sometimes the cause of subtle errors in Python). I wouldn't call it an attempt. I mean, sure it has no chance against for example Rust, but Python's type system is actually quite decent and I think it is more powerful than the one from Go. Also you have freedom of using both nominal and structural typing if you chose. The only problem I have with it when I have to use a package that don't have types, but fortunately that is happening less and less. Some package authors refuse to add types, but others provide stubs to solve that problem. For example boto3-stubs provides types for boto3. I really like that thanks to types I can also easily refactor code without worrying about breaking something in the process. I suspect if types existed and were popular with Python 2 then the whole Python 2 -> Python 3 migration would be a non issue.
- disgruntledphd2 5y ago> For many data science projects, maintaining large code bases in Python is often the optimal decision (rather than reinventing the wheel and writing your own data frame and machine learning libraries, or using immature poorly maintained ones in other languages — for data manipulation and scientific algorithms, these libraries are highly optimized in Python anyway since the underlying code is in C or Fortran) Or you could use R, instead of the half-baked immature clones in Python :) I'm only slightly joking here, even if Python is much much better for string processing, it's much less useful for DS tasks (unless it's NLP or DL, to be fair). That being said, Python is the second best DS language, as well as second best for everything else, so it can be a good choice for these projects.