6 ms·
Two days ago I took a proof-of-concept project written in Python, using the GMPY2 library (Gnu Multiple Precision library), and ported it to C. My C code was v
by kale 11y ago
Two days ago I took a proof-of-concept project written in Python, using the GMPY2 library (Gnu Multiple Precision library), and ported it to C.
My C code was very complicated in order to be fast. Lots of pointer swapping, bit-shifting, etc.
The code was about 4% faster in C, which was more lines of code, much harder to read (and write), compiled with every imaginable flag that could offer a speedup. Switching operating systems, this code was about 8% faster in C on Linux than Python in Windows 10. C in Linux would take about 9.5 days to complete, Python in Windows 10 would be about 10.5 days.
In addition, Python has the "pickle" module so that I can easily make the algorithm save state to pause and restore it any time I want. What's more, I can probably use mmap to share memory easily without making copies and switch to multiprocessing for an even greater speedup (my first attempt using Pool was a little slower despite using 3 cores, but that's the slowest way to multithread).
- marmaduke 11y agoAll this because you didn't profile your code to see that 95% of the time was spent in GMP routines.
- kale 11y agoYeah. I wasn't sure how GMPY was compiled, so I used C with my own compilation of GMP. I bet the flag differences made up the few percentage points. Plus, with each variable being ~10Mb, I was concerned about memory copying (I studied mechanical engineering, I missed a lot of this stuff in school), and with C I could make memory copying vs passing pointers explicit. All of that was unnecessary. It was another reinforcement of the idea: "Build it first, then optimize".