3 ms·

I mean this seriously, dispassionately, and without insult to Python, but this isn't used more broadly because if you're in a "counting microseconds", you shoul

by jerf 5y ago

I mean this seriously, dispassionately, and without insult to Python, but this isn't used more broadly because if you're in a "counting microseconds", you shouldn't have chosen Python in the first place, and you ought to be seriously considering getting whatever you're having this sort of performance issue with out of Python entirely. Python qua Python is so slow already that while a quick profile and some low-fruit plucking can be a quick win if you've never done it before, if you need something to go fast you need to get out of Python. The time you save with this technique is almost instantly eaten by all the other slow thing in Python. Precisely because that's such a great need, there's a lot of ways to do it, beyond the obvious "use another language directly". There's Ye Olde write a C extension. There's Cython, and a family of techniques there. There's trying to use PyPy. There's a whole array of more exotic things, if you do some searching around on the web, which may work for you. If you are in a "counting microseconds" situation, all of these are going to pay off much better than trying to squeeze Python. I include the first few words of my post here because I do not mean this as an "insult" or a slag on Python. It is simply something that people need to understand when they choose solutions to their problem. Everything has advantages and disadvantages. Python has many advantages, but one of its most noticeable disadvantages is that if you just naively and simply program Python, you end up working in the slowest class of languages. You can also slow down your code much more than you may realize if you pile the abstractions on too heavily. This isn't something you should process through a "rah rah! / booo!" "my team" filter, this is engineering facts on the ground.

amelius 5y ago
Like you said, Python has many advantages and disadvantages. Choosing one of the disadvantages and disqualifying Python based on it doesn't seem fair. It is also not very useful. Python has the largest ecosystem on the planet. Lack of X, Y of Z could disqualify a platform, yet Python has them all. If, by increasing performance even 10% you can put the Python ecosystem to better use, that's already a huge win.
- xg15 5y ago
  Other languages have large ecosystems as well, though, and are often far easier to optimize for speed. For python, speed was pretty much a non-goal. You could try to switch to Go or Nodejs - or do what Numpy/Scipy and TensorFlow did: write the performance-critical parts in C and only use python for the high-level control logic.
- jerf 5y ago
  You fundamentally misunderstand my point. My point is that if a particular project has a particular rock-solid need for something that Python can't do, you shouldn't choose Python. You can stick any language in there for that. You name the language, I'll name a disqualifying need for it. (A really simple one is "I want to work on $PLATFORM". There's hardly a language that's a great choice for iPhone, Android, desktop app, and server. There's some languages that can at least do all four but they aren't generally a great choice for all four.) It is about matching the needs of a project to the implementation language, and neither blindly picking languages because they are "good", so they must be good at everything, nor blindly disqualifying languages because they are "bad", so they must be bad at everything. If you need to go fast, you shouldn't have chosen Python. You don't always need to go fast. In fact, many people greatly overestimate their need to go fast! But if you did have that need to go fast, you shouldn't have chosen Python. I observe that this bothers people, because they treat it as an emotional decision about who or what they are endorsing, or what team they are or are not part of. This is the wrong view. It is an engineering question, just like I would ask whether I use steel or plywood. There is no "team steel" and "team plywood", no "plywood is better at everything!" "no way dude, plywood rules, steel droolz!". Just tools that need to be analyzed.

by jerf 5y ago

amelius 5y ago

Like you said, Python has many advantages and disadvantages. Choosing one of the disadvantages and disqualifying Python based on it doesn't seem fair. It is also not very useful. Python has the largest ecosystem on the planet. Lack of X, Y of Z could disqualify a platform, yet Python has them all. If, by increasing performance even 10% you can put the Python ecosystem to better use, that's already a huge win.

xg15 5y ago
Other languages have large ecosystems as well, though, and are often far easier to optimize for speed. For python, speed was pretty much a non-goal. You could try to switch to Go or Nodejs - or do what Numpy/Scipy and TensorFlow did: write the performance-critical parts in C and only use python for the high-level control logic.
jerf 5y ago
You fundamentally misunderstand my point. My point is that if a particular project has a particular rock-solid need for something that Python can't do, you shouldn't choose Python. You can stick any language in there for that. You name the language, I'll name a disqualifying need for it. (A really simple one is "I want to work on $PLATFORM". There's hardly a language that's a great choice for iPhone, Android, desktop app, and server. There's some languages that can at least do all four but they aren't generally a great choice for all four.) It is about matching the needs of a project to the implementation language, and neither blindly picking languages because they are "good", so they must be good at everything, nor blindly disqualifying languages because they are "bad", so they must be bad at everything. If you need to go fast, you shouldn't have chosen Python. You don't always need to go fast. In fact, many people greatly overestimate their need to go fast! But if you did have that need to go fast, you shouldn't have chosen Python. I observe that this bothers people, because they treat it as an emotional decision about who or what they are endorsing, or what team they are or are not part of. This is the wrong view. It is an engineering question, just like I would ask whether I use steel or plywood. There is no "team steel" and "team plywood", no "plywood is better at everything!" "no way dude, plywood rules, steel droolz!". Just tools that need to be analyzed.