4 ms·

First let me state that a language is just a brush with which to paint your logic. I really don't subscribe to the X is better than Y. I might concede that x is

First let me state that a language is just a brush with which to paint your logic. I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation. The first point about inconsistent spelling mistakes, I 100% sympathise. I suffer from this terribly. Pylint and syntastic are a silver bullet for me. I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change. Parallelism: I do look over the wall at Go and wish that python 3.x had spent more time on removing the GIL, but that is old news now. However I enjoy threading with python(not its limitations), because its so trivially easy. The Queues primitive saves 90% of the headache. as for ctrl-c not doing anything set all your threads to daemon (<threadObj>.setDaemon(True)) When the main thread exits, so does everything else. Go shines by having built in true parallelism If I need horsepower, then I have to run multiple processes (not multiprocessing, that has drawbacks.) which has the annoyance of serializing and message passing. (but that does mean scaling horizontally less painful further on). Again, Go really shines here. Statically compiling thing: YES. This means that I can 90% of the faff with docker (xyz version not working any more, the build fails) and use a real scheduling system that runs on real steel(or AWS) and not the omnifaff that is kubernetes.

earenndil 8y ago
I think the pypy people were able to get a trivial program running with the gil unlocked. I'd watch them for further updates.
JanisL 8y ago
I think when you really need performance it's a great example of a time to start looking at alternative Python implementations, CPython just isn't the best choice for a lot of tasks that actually have processing speed requirements.
- BerislavLopac 8y ago
  For starters, just adding Cython and/or Numba to the code should introduce great improvements. https://rushter.com/blog/numba-cython-python-optimization/ https://rushter.com/blog/numba-cython-python-optimization/
  - yen223 8y ago
    The word "just" is doing a lot of work. Cython might as well be a different language.
    - LyndsySimon 8y ago
      I generally don’t agree with the aversion to the word “just”, butbin this case you’re absolutely correct. Cython is a different language.
  - JanisL 8y ago
    After working in the Python speed optimization space as a consultant for a while I have come to see Cython as not really being the right answer for a lot of situations. The main reason is that CPython is slow but Cython really is quite heavily constraining you to the CPython runtime. Cython is significantly different enough to not be Python but it isn't the language I'd want to choose (C/C++/Rust) for going the whole way with foreign function interfacing (I might blog about this again sometime). Sure it's easy to get started with it but I feel like the maintenance costs are higher than people think. Numba on the other hand has a few nice use cases (where "nice" means close to Pareto-efficient improvement), but I tend to just be using PyPy if I'm going with JIT approach. If for some reason I had to use CPython but had a few bottleneck functions that could be sped up with Numba I'd strongly consider it.
codemusings 8y ago
> I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change. I'm happy to report that JetBrains' VIM plugin is almost flawless (Block-based editing, window splitting, save/close, it's all where it's supposed to be). I used to be in the same camp. But if you add a linter and parser and full blown backend to make lookups for code completion to your VIM configuration you might as well use an IDE. I've switched to IntelliJ for Java.
- abhishekjha 8y ago
  And they provide a free all pack subscription for open source contributors. Its nice to see all the features at one place though their pipenv integration is still a bit shaky[0]. I haven't got a response yet. [0] : https://youtrack.jetbrains.com/issue/PY-30855 https://youtrack.jetbrains.com/issue/PY-30855
- KaiserPro 8y ago
  I did see that it had vim support, but its always difficult to know which _bits_ of vim it supports. Thanks for this, I'll investigate.
fifnir 8y ago
Regarding multiprocessing and serializing between processes in python, and linux specifically, have you tried SharredArray? It dumps numpy arrays in /dev/shm which can loaded by other processes by the filepath, muuuuuch nicer than passing things around
- KaiserPro 8y ago
  Ooo, I have not. This sounds interesting.
kqr 8y ago
> I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation. Not even if X is better than Y for every specific task?
- KaiserPro 8y ago
  _if_ it was, then I'd be using that brush instead of the other. but to be honest, most languages are pretty much the same

earenndil 8y ago

I think the pypy people were able to get a trivial program running with the gil unlocked. I'd watch them for further updates.

JanisL 8y ago

I think when you really need performance it's a great example of a time to start looking at alternative Python implementations, CPython just isn't the best choice for a lot of tasks that actually have processing speed requirements.

BerislavLopac 8y ago
For starters, just adding Cython and/or Numba to the code should introduce great improvements. https://rushter.com/blog/numba-cython-python-optimization/ https://rushter.com/blog/numba-cython-python-optimization/
- yen223 8y ago
  The word "just" is doing a lot of work. Cython might as well be a different language.
  - LyndsySimon 8y ago
    I generally don’t agree with the aversion to the word “just”, butbin this case you’re absolutely correct. Cython is a different language.
- JanisL 8y ago
  After working in the Python speed optimization space as a consultant for a while I have come to see Cython as not really being the right answer for a lot of situations. The main reason is that CPython is slow but Cython really is quite heavily constraining you to the CPython runtime. Cython is significantly different enough to not be Python but it isn't the language I'd want to choose (C/C++/Rust) for going the whole way with foreign function interfacing (I might blog about this again sometime). Sure it's easy to get started with it but I feel like the maintenance costs are higher than people think. Numba on the other hand has a few nice use cases (where "nice" means close to Pareto-efficient improvement), but I tend to just be using PyPy if I'm going with JIT approach. If for some reason I had to use CPython but had a few bottleneck functions that could be sped up with Numba I'd strongly consider it.

codemusings 8y ago

> I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change. I'm happy to report that JetBrains' VIM plugin is almost flawless (Block-based editing, window splitting, save/close, it's all where it's supposed to be). I used to be in the same camp. But if you add a linter and parser and full blown backend to make lookups for code completion to your VIM configuration you might as well use an IDE. I've switched to IntelliJ for Java.

abhishekjha 8y ago
And they provide a free all pack subscription for open source contributors. Its nice to see all the features at one place though their pipenv integration is still a bit shaky[0]. I haven't got a response yet. [0] : https://youtrack.jetbrains.com/issue/PY-30855 https://youtrack.jetbrains.com/issue/PY-30855
KaiserPro 8y ago
I did see that it had vim support, but its always difficult to know which _bits_ of vim it supports. Thanks for this, I'll investigate.

fifnir 8y ago

Regarding multiprocessing and serializing between processes in python, and linux specifically, have you tried SharredArray? It dumps numpy arrays in /dev/shm which can loaded by other processes by the filepath, muuuuuch nicer than passing things around

KaiserPro 8y ago
Ooo, I have not. This sounds interesting.

kqr 8y ago

> I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation. Not even if X is better than Y for every specific task?

KaiserPro 8y ago
_if_ it was, then I'd be using that brush instead of the other. but to be honest, most languages are pretty much the same