6 ms·

Quite old, plain Python is still slow but PyPy is in the league of Node.js. I rewrote the same program (compare two images, generate a third that shows the dif

by epx 11y ago

Quite old, plain Python is still slow but PyPy is in the league of Node.js. I rewrote the same program (compare two images, generate a third that shows the diff) in a number of languages. Considering CPython as the reference implementation (1x), I got 100x in Rust, 60x in Go, 12x in Node.js and 10-11x in PyPy. Initially I got 4x with PyPy but I did a light refactor, removing some map()s and zip()s that were gratitious (3-element lists) and then PyPy went real fast.

frozenport 11y ago
As a C++ developer I find it hilarious that Node.js is a high performance league! ( your benchmark for example shows Rust as 6x faster)
- deleted 11y ago
  [deleted]
- chrisseaton 11y ago
  But it's high performance given the semantics of the language. The work that has gone into making V8 perform as it does is extraordinary and should be respected, not mocked as 'hilarious'.
  - yeukhon 11y ago
    I believe the high performance usually refers to NodeJS' non-blocking I/O.
  - frozenport 11y ago
    The benchmarks show that by using another language his code went 8x faster. Perhaps if he optimized his C++ it would go even faster. It's funny that people are saying 10x off the theoretical is high performance. I wonder if these people are living in a Javascript bubble.
sorenjan 11y ago
I've never used Rust, how would it compare to a C++ solution?
- hardwaresofton 11y ago
  By what measurement? I think the only thing I could say would be that it would be safer? And possible terser and possibly easy to understand.
- grayrest 11y ago
  The standard response from the Rust team is that Rust should match or beat non-SIMD C++ performance and if it doesn't, you should file a bug. Note: The first thing anybody will ask when you complain about Rust being slow is whether you compiled with optimizations turned on (`cargo build --release`) since it tends to make a 10-15x difference.
- im_down_w_otp 11y ago
  Theoretically the Rust borrow-checker also knows enough about your code's protection and dispatch semantics such that additional information could be used to create deeper optimizations than are available in either C or C++. Numerical analysis in Rust could compete with Fortran in performance, but I don't know if any of that has been actualized in Rust yet.
  - viperscape 11y ago
    I think some of that might start to come along with increase in compiler plugins, which is feature gated to nightly build right now.
joshvm 11y ago
You've picked a poor example, I think. And you're probably coding in an inefficient way. Here's the OpenCV way for a pair of 2048x2048 images: import cv2 import time t = time.clock() a = cv2.imread("./image_1.tiff", cv2.IMREAD_GRAYSCALE); b = cv2.imread("./image_0.tiff", cv2.IMREAD_GRAYSCALE); c = a-b cv2.imwrite("out.tiff", c); print time.clock()-t Takes about 0.2 CPU seconds on average for me (note using time.clock, not time.time on UNIX). #include <opencv2/opencv.hpp> #include <ctime> using namespace cv; using namespace std; int main(void){ clock_t begin = clock(); Mat a = imread("./image_1.tiff", IMREAD_GRAYSCALE); Mat b = imread("./image_0.tiff", IMREAD_GRAYSCALE); Mat c = a - b; imwrite("out.tiff", c); clock_t end = clock(); double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC; cout << elapsed_secs << endl; } Again, about 0.2 seconds. The difference is negligible if you use the right libraries. Python should not be your bottleneck for high performance code.
- coryrc 11y ago
  My conclusion is: as long as you don't do anything new, Python is fast enough.
  - ProblemFactory 11y ago
    That's true, but most of the time you can express the "new" stuff in terms of existing fast libraries. I've had a few cases where you can't, and have become a big fan of Cython for that. It lets you add C typedefs to Python code, and then compile the module at import time. Example here: http://pastebin.com/sF8KmyiU http://pastebin.com/sF8KmyiU All of pure Python is still allowed in these modules, but the typedeffed variables become pure C variables instead of objects, and loops become pure C loops. For this particular function, I got a 1000x speedup compared to the original Python code. In the end, this isn't Python any more - but it's close enough, and only needed for loops that run over millions of items.
- Someone 11y ago
  Now, try implement that matrix subtraction or something more complex such as blurring or edge detection in Python, and compare results. Also, are you sure that doesn't measure disk speed?

by epx 11y ago

frozenport 11y ago

As a C++ developer I find it hilarious that Node.js is a high performance league! ( your benchmark for example shows Rust as 6x faster)

deleted 11y ago
[deleted]
chrisseaton 11y ago
But it's high performance given the semantics of the language. The work that has gone into making V8 perform as it does is extraordinary and should be respected, not mocked as 'hilarious'.
- yeukhon 11y ago
  I believe the high performance usually refers to NodeJS' non-blocking I/O.
- frozenport 11y ago
  The benchmarks show that by using another language his code went 8x faster. Perhaps if he optimized his C++ it would go even faster. It's funny that people are saying 10x off the theoretical is high performance. I wonder if these people are living in a Javascript bubble.

sorenjan 11y ago

I've never used Rust, how would it compare to a C++ solution?

hardwaresofton 11y ago
By what measurement? I think the only thing I could say would be that it would be safer? And possible terser and possibly easy to understand.
grayrest 11y ago
The standard response from the Rust team is that Rust should match or beat non-SIMD C++ performance and if it doesn't, you should file a bug. Note: The first thing anybody will ask when you complain about Rust being slow is whether you compiled with optimizations turned on (`cargo build --release`) since it tends to make a 10-15x difference.
im_down_w_otp 11y ago
Theoretically the Rust borrow-checker also knows enough about your code's protection and dispatch semantics such that additional information could be used to create deeper optimizations than are available in either C or C++. Numerical analysis in Rust could compete with Fortran in performance, but I don't know if any of that has been actualized in Rust yet.
- viperscape 11y ago
  I think some of that might start to come along with increase in compiler plugins, which is feature gated to nightly build right now.

joshvm 11y ago

You've picked a poor example, I think. And you're probably coding in an inefficient way. Here's the OpenCV way for a pair of 2048x2048 images: import cv2 import time t = time.clock() a = cv2.imread("./image_1.tiff", cv2.IMREAD_GRAYSCALE); b = cv2.imread("./image_0.tiff", cv2.IMREAD_GRAYSCALE); c = a-b cv2.imwrite("out.tiff", c); print time.clock()-t Takes about 0.2 CPU seconds on average for me (note using time.clock, not time.time on UNIX). #include <opencv2/opencv.hpp> #include <ctime> using namespace cv; using namespace std; int main(void){ clock_t begin = clock(); Mat a = imread("./image_1.tiff", IMREAD_GRAYSCALE); Mat b = imread("./image_0.tiff", IMREAD_GRAYSCALE); Mat c = a - b; imwrite("out.tiff", c); clock_t end = clock(); double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC; cout << elapsed_secs << endl; } Again, about 0.2 seconds. The difference is negligible if you use the right libraries. Python should not be your bottleneck for high performance code.

coryrc 11y ago
My conclusion is: as long as you don't do anything new, Python is fast enough.
- ProblemFactory 11y ago
  That's true, but most of the time you can express the "new" stuff in terms of existing fast libraries. I've had a few cases where you can't, and have become a big fan of Cython for that. It lets you add C typedefs to Python code, and then compile the module at import time. Example here: http://pastebin.com/sF8KmyiU http://pastebin.com/sF8KmyiU All of pure Python is still allowed in these modules, but the typedeffed variables become pure C variables instead of objects, and loops become pure C loops. For this particular function, I got a 1000x speedup compared to the original Python code. In the end, this isn't Python any more - but it's close enough, and only needed for loops that run over millions of items.
Someone 11y ago
Now, try implement that matrix subtraction or something more complex such as blurring or edge detection in Python, and compare results. Also, are you sure that doesn't measure disk speed?