4 ms·
Here is the thing though, and I am gonna phrase this very carefully as I have stepped on some toes before and got downvoted. Imagine a naive young scientist wh
by zwaps 6y ago
Here is the thing though, and I am gonna phrase this very carefully as I have stepped on some toes before and got downvoted.
Imagine a naive young scientist who has been "trained" to code in Matlab. He is not a computer scientist, of course, but he profits if his program goes fast, even before actual computer scientists get involved to make the thing proper. Indeed that may make the difference between the project getting off the ground or not going forward. Such cases have been known to exist.
Now, he comes across some code that can not be written in pre-optimized matrix routines, but nevertheless populates deterministic addresses in a container without any further side effects. It could wait for some IO, or perhaps it just does a thing that takes a while on the CPU core (like iterations of something with results getting saved somehwere).
In any case, it's gotta go into a for loop and that's slow for most languages of that sort.
Given that he has a good number of cores, threads etc. available, he figures it'd be pretty cool if the thing run in parallel.
So he changes the "for" statement into "parfor". Matlab then starts a CPU pool and runs the loop to completion in parallel on both threads and cores without further issues. The whole thing just runs x times faster.
My point here is this: The whole thing in Python is complicated. It is so, because other languages, like Matlab, make it laughably easy. The Pythonic way would have been to offer such an easy option. However, if you are not well-versed in serious coding, then concurrent anything in Python is hella complicated. If you disagree, you likely know more about multi-x than the average person typing things into a computer.
I do not doubt that such a simple approach has its limitations. But it seems to me that in practice - and in particular when I look at most of the examples presented - such a simple way would have been completely sufficient. I do readily accept that I may be completely wrong.
edit: And now I realize that your verb was "debug" and not "write". Sorry.
- downerending 6y agoYeah, I support these people regularly. If you can replace "for" with "parfor" and it just goes faster, well, God bless you. In my experience, 90% of these novices haven't the faintest clue of what's going on. "It worked before, but now I'm running out of memory." "Why are you telling me that I'm crushing disk I/O?" "I went from 1 core to 40, and now it runs slower!" "I requested 32 nodes--why doesn't it run 32x faster?" "I requested one core, but MATLAB saw the awesome stuff in /proc/cpuinfo and ran 40 cores. Why is it even slower than ever?" Bless the clueless, for they shall inherit the earth. (But please, don't let them write coronavirus models in public...)