6 ms·

I wish Octave the best of luck, but it faces an uphill battle against MATLAB (established engineering projects) and Python (new starts). The biggest issue with

by ptero 6y ago

I wish Octave the best of luck, but it faces an uphill battle against MATLAB (established engineering projects) and Python (new starts). The biggest issue with Octave for me is the slowness of its plotting compared to MATLAB. A major use case for me is visualizing large data sets. I can generate data using any backend (Python, C++, MATLAB, etc.) and want the ability to plot it, zoom in/out to a rectangle, filter to a subset / go back to the full dataset. Simple 2D plotting, no 3D or hard rendering stuff. In such plotting I find Octave (and to a lesser degree Python's matplotlib) super slow compared to MATLAB. To see it, make a scatterplot of 1-5 million points (say, with x and y random, uniformly distributed in [0, 1]). Zoom to a portion a couple of times. MATLAB response is instant, Octave takes seconds. Do the same for 20 million points (which is still a very small dataset by today's standards) and forget about interactive plotting with Octave :(. As I said, I still wish Octave the best and hope it succeeds, but with my student days behind me I prefer to just buy an individual license of MATLAB every 5 years or so for my hobby projects.

dorfsmay 6y ago
Is Julia not the new player in that game?
- sgillen 6y ago
  Julia is a new player but probably still "too new" for most people (I say this as a Julia fan). Python is a pretty old language at this point, but the data sciencey eco system has only matured relatively recently (compared to the age of Matlab anyway).
- ptero 6y ago
  It probably is, but new vs old is not a big decision driver for me. The things that matter to me for scientific computing languages are visualization tools (most critical to me, by far), libraries for standard functionality like optimization and signal processing (not critical as I can use another language for back-end processing) and price. I will try Julia's visualization tools sometime, but if MATLAB is significantly faster will stick to it for now. Just a single user's data point, not claiming that this is a universal view.
  - enriquto 6y ago
    > Just a single user's data point, not claiming that this is a universal view. Definitely not for me. As a matter of principle, I try to keep my computation and visualization tools strictly separate. Even, "artificially" separate, if you may. I would actually like octave more if it didn't provide a plot function. It would be closer to the spirit of the unix philosophy.
    - ptero 6y ago
      Your point is perfectly valid. I do separate my computation and visualization (sometimes both are MATLAB; just separate). For me the MATLAB is primarily an interactive visualization tool, although the main use case for most engineers I am sure is to use MATLAB for its scientific computation functionality.
  - dfrankow 6y ago
    ggplot? If your benchmark is plotting 2 million points, it may also be slow. However, I find its ability to rapidly prototype tons of different visualizations useful. I try to avoid needing to plot 2 million points anyway.
    - ptero 6y ago
      I often need to visualize data and unfortunately, plotting and working with many millions of points is a key capability. For example, we might have an expensive piece of hardware that we are using and need to understand if an issue we are seeing is a problem in our user software, a firmware we wrote to interface with it, or comes from the hardware itself. Having technical representatives from each team (each sure that his team has nothing to do with it; can they please go back to their offices now) in a room looking at the same instrumentation data with ability to quickly dig into various features and subsets is a huge help for such problem isolation. At least in my experience it depends on fast interactive plotting and filtering of large datasets.
- IshKebab 6y ago
  Julia's plotting is also unusable slow unfortunately.
enriquto 6y ago
Curiously, I have the opposite experience as yours between matlab and octave. I have used both, and I find matlab clunky and slow to start, while octave is very lean and responsive. My way of using them is probably not very representative though. I never do plotting, it's mostly saving the computed solutions into files and looking at them with other programs. In the rare cases I need to plot something, I write the data into a file and call gnuplot separately. Also, I do not run the interactive interpreter, but I write scripts that launch a new interpreter for each run. Matlab seems to be extremely ill-suited for this usage. For example, it takes about 3 seconds just for launching (!). Also, there are bizarre problems (apart from ridiculous memory usage), when running several matlab instances in parallel.
- ptero 6y ago
  Agree completely -- MATLAB is slow to start and is a memory hog. The language itself is horrible (which is why I prefer to use another language for complex back-end computation). Libraries for reading and writing formatted data are a joke (try reading a large XML or JSON file; or even a large CSV file with a mix of numeric and string fields); etc., etc. But it is just so good for visualization that I am willing to suffer its other problems. For example, try the below and zoom in / out / resize in both MATLAB and Octave: n = 2000000; x = rand(1, n); y = rand(1, n); plot(x, y, '.')
  - AlotOfReading 6y ago
    If your primary concern is only the quality of visualizations and not language or libraries, I'm curious why you'd choose MATLAB over something like Mathematica. I've always found the latter to produce far prettier plots (with far less effort).
    - ptero 6y ago
      I gave a more involved example in a sibling comment, but I need fast, interactive plotting for large datasets; not necessarily the prettiest. And as another user mentioned, MATLAB currently has a lock on fast scientific plotting. It really is in a league of its own for now.
fnordfnordfnord 6y ago
Like you, I haven't used either for serious work in many years but the last time I was doing serious work I found Python to be much faster. I discovered this by accident after porting one of my Matlab programs to Python and noticed an immediate 4x speed improvement. I worked with both for many more years afterward and in my experience Python was usually faster with a few exceptions. That said, Matlab had great tech support and the sales rep was always super helpful. They were much nicer to work with than say National Instruments.
qppo 6y ago
For me the issue is less performance and more correctness/completeness. Mathworks has teams of engineers working on their toolboxes, which helps a lot for users that need features and needs them to work correctly. It's been some time since I had to use Octave because I didn't want to pay for a Matlab license, but I recall remez() didn't properly implement the Parks-McClellan algorithm and would return suboptimal results (which is a problem, when the algorithm is intended to find the optimal solution). Also, Simulink.
IshKebab 6y ago
> The biggest issue with Octave for me is the slowness of its plotting compared to MATLAB I was going to say exactly the same thing. The language implementation is mostly fine-ish. But plotting (which is like half the point of MATLAB) is unusable slow. It's not just Octave though. All of Julia's plotting libraries are similarly slow. MATLAB still has a monopoly on fast scientific plotting. Matlab's hobby license is sufficiently cheap that I really would recommend getting that unless you absolutely can't afford it.
- ptero 6y ago
  This (individual non-commercial license) is my current approach, too. I wonder though, why do we have such a lack of fast scientific plotting, as you wrote? Is it due to the fact that few people really care or is there some inherent set of hard problems that Mathworks figured out how to handle?
  - IshKebab 6y ago
    I have no idea to be honest. I don't think there is much that is really hard about it. It's probably a ton of work because to be really useful you have to support a ton of options (like rotated labels, log axes, dashed lines, stepped lines, gaps in lines, etc. etc.). It's a nearly endless list and I guess to do that and to make it not slow is just loooads of work. Probably in Octave's case it is just really ancient code that uses X11 or something. They probably need to rewrite it to use Skia or similar, or even OpenGL. Maybe something like MathGL: http://mathgl.sourceforge.net/doc_en/plot-sample.html#plot-sample http://mathgl.sourceforge.net/doc_en/plot-sample.html#plot-s... I've never tried that though.
tomsmeding 6y ago
How fast does it have to be? I was curious and implemented a trivial loop in C++ that draws "scatter plot" on your benchmark (scatter plot with lots of [0,1]^2 points). Its concept of "scatter plot" is: 1920x1080 image, draw a non-antialiased (i.e. ugly) filled blue circle of radius 4 at each point, where the unit square is stretched over the entire image. Doing this (C++, CPU) takes ~0.21 seconds for 2 million points, and ~2.06 seconds for 20 million points. The result is a solid blue rectangle, but that's beside the point. Doing the same on a trivial GPU implementation using CUDA, where I cheated by not even using atomic operations (so this only works because the dots are solid), takes ~0.23 seconds for 2 million points and ~2.3 seconds for 20 million points. Apparently the (my? (GTX 1050 mobile)) GPU doesn't help here. For comparison, with matplotlib (`plt.scatter(x, y); plt.show()`): for 2 million points, plt.scatter() takes ~10 seconds, and plt.show() takes about 3 seconds fill first image. Zooms/resizes etc. seem to be the same time, or if less points show, proportional to the number of visible points. So unsurprisingly my cheap C++ code is faster than matplotlib (it doesn't even anti-alias), but is it faster than Matlab? I have neither Octave nor Matlab, perhaps you can compare with matplotlib? EDIT: my code here https://gist.github.com/tomsmeding/1631090df10ab5ac98403304ec2b8ac9 https://gist.github.com/tomsmeding/1631090df10ab5ac98403304e...
- ptero 6y ago
  Thank you for the data point, but I do not think the issue is with rendering of the dots on screen (although I am not sure). In fact, I have done some real-time, frame-by-frame video processing using OpenCV and had no issue processing 1920x1080 images in real-time at 25fps (40ms per frame), including some simple edge detection, segmentation and rendering on screen using my office workstation. Modern hardware is certainly fast enough to render dots on the screen as fast as interactive processing needs. What I suspect is taking place with Octave/Python/Julia-based tools is the additional information that needs to be handled for each plot. You need to draw axes and ticks, keep tracks of which points are displayed and obscured at at the current axes limits and update all that information every time any of those things change (reset/move/zoom, etc.). It is certainly possible to make abstraction layers that are convenient for software engineers but would not scale well with respect to the number of points that need to be plotted. Or, instead of carefully tracking each state change one could regenerate a large portion of it to make it simpler (and slower). Just a guess though.

by ptero 6y ago

dorfsmay 6y ago

Is Julia not the new player in that game?

sgillen 6y ago
Julia is a new player but probably still "too new" for most people (I say this as a Julia fan). Python is a pretty old language at this point, but the data sciencey eco system has only matured relatively recently (compared to the age of Matlab anyway).
ptero 6y ago
It probably is, but new vs old is not a big decision driver for me. The things that matter to me for scientific computing languages are visualization tools (most critical to me, by far), libraries for standard functionality like optimization and signal processing (not critical as I can use another language for back-end processing) and price. I will try Julia's visualization tools sometime, but if MATLAB is significantly faster will stick to it for now. Just a single user's data point, not claiming that this is a universal view.
- enriquto 6y ago
  > Just a single user's data point, not claiming that this is a universal view. Definitely not for me. As a matter of principle, I try to keep my computation and visualization tools strictly separate. Even, "artificially" separate, if you may. I would actually like octave more if it didn't provide a plot function. It would be closer to the spirit of the unix philosophy.
  - ptero 6y ago
    Your point is perfectly valid. I do separate my computation and visualization (sometimes both are MATLAB; just separate). For me the MATLAB is primarily an interactive visualization tool, although the main use case for most engineers I am sure is to use MATLAB for its scientific computation functionality.
- dfrankow 6y ago
  ggplot? If your benchmark is plotting 2 million points, it may also be slow. However, I find its ability to rapidly prototype tons of different visualizations useful. I try to avoid needing to plot 2 million points anyway.
  - ptero 6y ago
    I often need to visualize data and unfortunately, plotting and working with many millions of points is a key capability. For example, we might have an expensive piece of hardware that we are using and need to understand if an issue we are seeing is a problem in our user software, a firmware we wrote to interface with it, or comes from the hardware itself. Having technical representatives from each team (each sure that his team has nothing to do with it; can they please go back to their offices now) in a room looking at the same instrumentation data with ability to quickly dig into various features and subsets is a huge help for such problem isolation. At least in my experience it depends on fast interactive plotting and filtering of large datasets.
IshKebab 6y ago
Julia's plotting is also unusable slow unfortunately.

enriquto 6y ago

Curiously, I have the opposite experience as yours between matlab and octave. I have used both, and I find matlab clunky and slow to start, while octave is very lean and responsive. My way of using them is probably not very representative though. I never do plotting, it's mostly saving the computed solutions into files and looking at them with other programs. In the rare cases I need to plot something, I write the data into a file and call gnuplot separately. Also, I do not run the interactive interpreter, but I write scripts that launch a new interpreter for each run. Matlab seems to be extremely ill-suited for this usage. For example, it takes about 3 seconds just for launching (!). Also, there are bizarre problems (apart from ridiculous memory usage), when running several matlab instances in parallel.

ptero 6y ago
Agree completely -- MATLAB is slow to start and is a memory hog. The language itself is horrible (which is why I prefer to use another language for complex back-end computation). Libraries for reading and writing formatted data are a joke (try reading a large XML or JSON file; or even a large CSV file with a mix of numeric and string fields); etc., etc. But it is just so good for visualization that I am willing to suffer its other problems. For example, try the below and zoom in / out / resize in both MATLAB and Octave: n = 2000000; x = rand(1, n); y = rand(1, n); plot(x, y, '.')
- AlotOfReading 6y ago
  If your primary concern is only the quality of visualizations and not language or libraries, I'm curious why you'd choose MATLAB over something like Mathematica. I've always found the latter to produce far prettier plots (with far less effort).
  - ptero 6y ago
    I gave a more involved example in a sibling comment, but I need fast, interactive plotting for large datasets; not necessarily the prettiest. And as another user mentioned, MATLAB currently has a lock on fast scientific plotting. It really is in a league of its own for now.

fnordfnordfnord 6y ago

Like you, I haven't used either for serious work in many years but the last time I was doing serious work I found Python to be much faster. I discovered this by accident after porting one of my Matlab programs to Python and noticed an immediate 4x speed improvement. I worked with both for many more years afterward and in my experience Python was usually faster with a few exceptions. That said, Matlab had great tech support and the sales rep was always super helpful. They were much nicer to work with than say National Instruments.

qppo 6y ago

For me the issue is less performance and more correctness/completeness. Mathworks has teams of engineers working on their toolboxes, which helps a lot for users that need features and needs them to work correctly. It's been some time since I had to use Octave because I didn't want to pay for a Matlab license, but I recall remez() didn't properly implement the Parks-McClellan algorithm and would return suboptimal results (which is a problem, when the algorithm is intended to find the optimal solution). Also, Simulink.

IshKebab 6y ago

> The biggest issue with Octave for me is the slowness of its plotting compared to MATLAB I was going to say exactly the same thing. The language implementation is mostly fine-ish. But plotting (which is like half the point of MATLAB) is unusable slow. It's not just Octave though. All of Julia's plotting libraries are similarly slow. MATLAB still has a monopoly on fast scientific plotting. Matlab's hobby license is sufficiently cheap that I really would recommend getting that unless you absolutely can't afford it.

ptero 6y ago
This (individual non-commercial license) is my current approach, too. I wonder though, why do we have such a lack of fast scientific plotting, as you wrote? Is it due to the fact that few people really care or is there some inherent set of hard problems that Mathworks figured out how to handle?
- IshKebab 6y ago
  I have no idea to be honest. I don't think there is much that is really hard about it. It's probably a ton of work because to be really useful you have to support a ton of options (like rotated labels, log axes, dashed lines, stepped lines, gaps in lines, etc. etc.). It's a nearly endless list and I guess to do that and to make it not slow is just loooads of work. Probably in Octave's case it is just really ancient code that uses X11 or something. They probably need to rewrite it to use Skia or similar, or even OpenGL. Maybe something like MathGL: http://mathgl.sourceforge.net/doc_en/plot-sample.html#plot-sample http://mathgl.sourceforge.net/doc_en/plot-sample.html#plot-s... I've never tried that though.

tomsmeding 6y ago

How fast does it have to be? I was curious and implemented a trivial loop in C++ that draws "scatter plot" on your benchmark (scatter plot with lots of [0,1]^2 points). Its concept of "scatter plot" is: 1920x1080 image, draw a non-antialiased (i.e. ugly) filled blue circle of radius 4 at each point, where the unit square is stretched over the entire image. Doing this (C++, CPU) takes ~0.21 seconds for 2 million points, and ~2.06 seconds for 20 million points. The result is a solid blue rectangle, but that's beside the point. Doing the same on a trivial GPU implementation using CUDA, where I cheated by not even using atomic operations (so this only works because the dots are solid), takes ~0.23 seconds for 2 million points and ~2.3 seconds for 20 million points. Apparently the (my? (GTX 1050 mobile)) GPU doesn't help here. For comparison, with matplotlib (`plt.scatter(x, y); plt.show()`): for 2 million points, plt.scatter() takes ~10 seconds, and plt.show() takes about 3 seconds fill first image. Zooms/resizes etc. seem to be the same time, or if less points show, proportional to the number of visible points. So unsurprisingly my cheap C++ code is faster than matplotlib (it doesn't even anti-alias), but is it faster than Matlab? I have neither Octave nor Matlab, perhaps you can compare with matplotlib? EDIT: my code here https://gist.github.com/tomsmeding/1631090df10ab5ac98403304ec2b8ac9 https://gist.github.com/tomsmeding/1631090df10ab5ac98403304e...

ptero 6y ago
Thank you for the data point, but I do not think the issue is with rendering of the dots on screen (although I am not sure). In fact, I have done some real-time, frame-by-frame video processing using OpenCV and had no issue processing 1920x1080 images in real-time at 25fps (40ms per frame), including some simple edge detection, segmentation and rendering on screen using my office workstation. Modern hardware is certainly fast enough to render dots on the screen as fast as interactive processing needs. What I suspect is taking place with Octave/Python/Julia-based tools is the additional information that needs to be handled for each plot. You need to draw axes and ticks, keep tracks of which points are displayed and obscured at at the current axes limits and update all that information every time any of those things change (reset/move/zoom, etc.). It is certainly possible to make abstraction layers that are convenient for software engineers but would not scale well with respect to the number of points that need to be plotted. Or, instead of carefully tracking each state change one could regenerate a large portion of it to make it simpler (and slower). Just a guess though.