5 ms·

Drop-in GPU Acceleration of GNU Octave

82 points by bprs 12y ago

osivertsson 12y ago
We choose GNU Octave, since it supports single precision which the GK104 we’ll be running performs best on. Is single-precision computations useful in the context where GNU Octave is generally used?
- valarauca1 12y ago
  Single vs Double precisions is fine. Most of Matlab calculations are homework or simulations. Generally speaking the real world will add more errors then double floating points will reduce. Honestly GNU Octave has been a surprisingly fast developing product. Matlab only got GPU acceleration 2 years ago (according to my buddy who uses Matlab) (which is very fast for a GNU project). A lot of academics are picking up on it, even professors recommending it. I think the most glowing recommendation I heard was, "Well its free, and free goes along way when you're living on a research stipend."
  - wtallis 12y ago
    Double-precision is a good default for a tool like Octave, but I agree that single-precision can be plenty accurate for a lot of uses. It's often just not worth the added time and code complexity to determine where single-precision should be used.
  - maaku 12y ago
    Depends very much on the algorithm. Some are sensitive to rounding precision, others are not.
- mjcohen 12y ago
  Note that GNU Octave, the real thing, is available for Android along with a number of its toolboxes: https://play.google.com/store/search?q=octave&c=apps https://play.google.com/store/search?q=octave&c=apps
  - bane 12y ago
    For even more math fun, Maxima is also on Android. https://play.google.com/store/apps/details?id=jp.yhonda&hl=en https://play.google.com/store/apps/details?id=jp.yhonda&hl=e...
- foxhill 12y ago
  it could very easily run double precision, but then they'd have to use their tesla cards to run it, and at several thousand dollars a piece, wont be quite as attractive to the user..
  - gh02t 12y ago
    Uh most recent Nvidia GPUs support double precision, albeit at a speed penalty. And they are using a Tesla card in the article. It says they are using a K10.
    - foxhill 12y ago
      of course they all can do double, but at 1/24th of the speed of single. (1/32th in maxwell) they use a GK104 based tesla, i.e a "dual GPU" GTX680, so it has no real double precision. unless you consider 2x 95GFlops noteworthy.
  - wtallis 12y ago
    Consumer GPUs can do double-precision. At the high end, consumer parts usually (though not always) get crippled performance relative to their pro counterparts, but for the low and mid-range chips even the workstation cards have lackluster double-precision performance.
    - foxhill 12y ago
      well, yes, technically correct, but a CPU has much better double precision performance than that. 1/24th (or 1/32th in maxwell) of single precision performance doesn't really sell the usefulness of CUDA.
- lightcatcher 12y ago
  Single precision is about 2.5x faster than double precision on current GPUs (for matrix multiplication, which is compute dominated). It really depends on the application, but in my experience more often than not the only reason you are using GPUs is because you want to squeeze out every last ounce of performance. In these cases, single precision makes a lot of sense (assuming you're algorithm doesn't depend heavily on 64 bits of precision). Within the neural nets community, single precision is almost always used (at least on GPUs).
jordigh 12y ago
Sigh, I wish people would address the problem here that nvidia's CUDA runtime is non-free. There is no OpenCL free implementation that I know of either, except Clover, but it currently merely translates the OpenCL to run on the CPU instead of the GPU. I think I've heard of some free CUDA implementations that were in similar embryonic stages. Octave is a GNU package. GNU's purpose is to ensure that you can use free software. Running to tie Octave to flashy features like GPU acceleration without first pausing to fix the initial problem of non-free GPU acceleration is putting the horse before the cart. This works against Octave's goal, to provide a free alternative to Matlab, one that lets you understand and control your computations down to the hardware level. If we don't emphasise software freedom, then there is no need for Octave, since we already have Matlab. Indeed, it is this very freedom that nvidia is abusing here to accelerate Octave's BLAS libraries, a task that would be much more difficult with Matlab, where they don't have the source code. I know that nobody wants to even think that this problem exists and even fewer people want to fix Clover because it's such a difficult task, but it's a task that we can't ignore. Note also that because the GPU libraries are not a system library as defined by the GPL (they are not shipped with the OS), you can't even distribute GPU-accelerated Octave object code. We consider the Octave C++ API to fall well under the domain of the GPL's copyleft. Personally, as a GNU Octave developer, I am very unhappy that nvidia is using Octave to advertise its hardware and non-free drivers. I am also unhappy that nvidia is luring users to use non-free software, acting against our goals. I reiterate Linus Torvalds's well-known sentiments against nvidia.
- seiji 12y ago
  All nvidia did was make an API compatible BLAS library with automatic GPU offloading. Other programs using BLAS functions then get automatically GPU accelerated. Nobody altered octave. They just LD_PRELOAD it in place. The same thing works with R too. The goal is "have software that works," not "have software that retains ideological purity at the cost of advancement."
  - aw3c2 12y ago
    That's your goal. In my opinion our goal as society should absolutely be software that everyone can "run, copy, distribute, study, change and improve".
zurn 12y ago
The entire GPU programming landscape is so fragmented and unapproachable now. If they'd only exposed the processors directly and worked to unify behind a common compiler frontend... Imagine a world where a good GPGPU language was a standard GCC/LLVM frontend and all the GPU vendors worked in harmony to improve their back-ends in the upstream compiler. This would enable development of new cross-vendor GPU languages to compete with the ailing OpenCL. (This is what Apple has done with Metal Shading Language btw, minus the open source part. And what Intel's Larrabee would have enabled.) Even if you disregard the licensing, the union of NV + AMD + Intel capabilities (software stacks included) is so weak that it almost never worth the effort. The comically bad software from AMD and lack of hardware oomph from Intel means the only option is to require NVidia and build on their software support. This is passable for a narrow sector of activities that can tolerate vendor lock-in (Cuda), like short lived HPC projects. All this has resulted in lack of open GPU programming languages. OpenCL is better than nothing, but even if the implementations were of usable quality, it's not a good compiler target for higher level languages and it's not a good language to write by hand. The situation pretty much guarantees GPU computing stays in the fringes for the foreseeable future.
- graphene 12y ago
  AMD and ARM are working on something that sounds like what you describe: http://en.wikipedia.org/wiki/Heterogeneous_System_Architecture http://en.wikipedia.org/wiki/Heterogeneous_System_Architectu... I believe nVidia has a similar effort going (can't remember the name), so it's still not a single agreed standard, but it's moving in that direction I feel.
  - lightcatcher 12y ago
    nVidia's PTX is the most similar thing I can think of (and I know nVidia's GPGPU products pretty well right now). PTX is an ISA for a virtual machine, which makes it similar to HSAIL. PTX doesn't come with a memory model specification though. However, I don't think either HSA or PTX is very important if nVidia, AMD, and Intel (wrt Xeon Phi's) don't start agreeing on standards. I haven't seen anything to convince me that nVidia wants to work within standards, and CUDA seems to have most of the market share right now, so I'm not too hopefully for portable heterogeneous computing in the near future. To my grandparent poster: I don't think working with GPUs is that bad right now if you bind yourself to a vendor. In my experience, drinking the CUDA kool-aid (CU-aid?) isn't all that bad.
cieyo9938y88 12y ago
I dont know much, but I REMEMBER the finger gesture of Linus or the LINUX founder about no no vidia and the big hassles on no no vidia drivers for BSD and FreeBSD. Python, MATLAB and closed source?? - Using Haskell on toy examples can get FUNNY SUPRISE. FUNNY SURPRISE. FUNNY SURPRISE. How about connecting the deep secret GPU to the no function Intel chip TSX transaction and one safe thread, while observing the side channels and effects for better GUESSING GAMES and understanding PSEUDO or fake random numbers?
- codygman 12y ago
  > Using Haskell on toy examples can get FUNNY SUPRISE. Funny surprises such as?

82 points by bprs 12y ago

We choose GNU Octave, since it supports single precision which the GK104 we’ll be running performs best on. Is single-precision computations useful in the context where GNU Octave is generally used?

valarauca1 12y ago
Single vs Double precisions is fine. Most of Matlab calculations are homework or simulations. Generally speaking the real world will add more errors then double floating points will reduce. Honestly GNU Octave has been a surprisingly fast developing product. Matlab only got GPU acceleration 2 years ago (according to my buddy who uses Matlab) (which is very fast for a GNU project). A lot of academics are picking up on it, even professors recommending it. I think the most glowing recommendation I heard was, "Well its free, and free goes along way when you're living on a research stipend."
- wtallis 12y ago
  Double-precision is a good default for a tool like Octave, but I agree that single-precision can be plenty accurate for a lot of uses. It's often just not worth the added time and code complexity to determine where single-precision should be used.
- maaku 12y ago
  Depends very much on the algorithm. Some are sensitive to rounding precision, others are not.
mjcohen 12y ago
Note that GNU Octave, the real thing, is available for Android along with a number of its toolboxes: https://play.google.com/store/search?q=octave&c=apps https://play.google.com/store/search?q=octave&c=apps
- bane 12y ago
  For even more math fun, Maxima is also on Android. https://play.google.com/store/apps/details?id=jp.yhonda&hl=en https://play.google.com/store/apps/details?id=jp.yhonda&hl=e...
foxhill 12y ago
it could very easily run double precision, but then they'd have to use their tesla cards to run it, and at several thousand dollars a piece, wont be quite as attractive to the user..
- gh02t 12y ago
  Uh most recent Nvidia GPUs support double precision, albeit at a speed penalty. And they are using a Tesla card in the article. It says they are using a K10.
  - foxhill 12y ago
    of course they all can do double, but at 1/24th of the speed of single. (1/32th in maxwell) they use a GK104 based tesla, i.e a "dual GPU" GTX680, so it has no real double precision. unless you consider 2x 95GFlops noteworthy.
- wtallis 12y ago
  Consumer GPUs can do double-precision. At the high end, consumer parts usually (though not always) get crippled performance relative to their pro counterparts, but for the low and mid-range chips even the workstation cards have lackluster double-precision performance.
  - foxhill 12y ago
    well, yes, technically correct, but a CPU has much better double precision performance than that. 1/24th (or 1/32th in maxwell) of single precision performance doesn't really sell the usefulness of CUDA.
lightcatcher 12y ago
Single precision is about 2.5x faster than double precision on current GPUs (for matrix multiplication, which is compute dominated). It really depends on the application, but in my experience more often than not the only reason you are using GPUs is because you want to squeeze out every last ounce of performance. In these cases, single precision makes a lot of sense (assuming you're algorithm doesn't depend heavily on 64 bits of precision). Within the neural nets community, single precision is almost always used (at least on GPUs).

jordigh 12y ago

Sigh, I wish people would address the problem here that nvidia's CUDA runtime is non-free. There is no OpenCL free implementation that I know of either, except Clover, but it currently merely translates the OpenCL to run on the CPU instead of the GPU. I think I've heard of some free CUDA implementations that were in similar embryonic stages. Octave is a GNU package. GNU's purpose is to ensure that you can use free software. Running to tie Octave to flashy features like GPU acceleration without first pausing to fix the initial problem of non-free GPU acceleration is putting the horse before the cart. This works against Octave's goal, to provide a free alternative to Matlab, one that lets you understand and control your computations down to the hardware level. If we don't emphasise software freedom, then there is no need for Octave, since we already have Matlab. Indeed, it is this very freedom that nvidia is abusing here to accelerate Octave's BLAS libraries, a task that would be much more difficult with Matlab, where they don't have the source code. I know that nobody wants to even think that this problem exists and even fewer people want to fix Clover because it's such a difficult task, but it's a task that we can't ignore. Note also that because the GPU libraries are not a system library as defined by the GPL (they are not shipped with the OS), you can't even distribute GPU-accelerated Octave object code. We consider the Octave C++ API to fall well under the domain of the GPL's copyleft. Personally, as a GNU Octave developer, I am very unhappy that nvidia is using Octave to advertise its hardware and non-free drivers. I am also unhappy that nvidia is luring users to use non-free software, acting against our goals. I reiterate Linus Torvalds's well-known sentiments against nvidia.

seiji 12y ago
All nvidia did was make an API compatible BLAS library with automatic GPU offloading. Other programs using BLAS functions then get automatically GPU accelerated. Nobody altered octave. They just LD_PRELOAD it in place. The same thing works with R too. The goal is "have software that works," not "have software that retains ideological purity at the cost of advancement."
- aw3c2 12y ago
  That's your goal. In my opinion our goal as society should absolutely be software that everyone can "run, copy, distribute, study, change and improve".

zurn 12y ago

The entire GPU programming landscape is so fragmented and unapproachable now. If they'd only exposed the processors directly and worked to unify behind a common compiler frontend... Imagine a world where a good GPGPU language was a standard GCC/LLVM frontend and all the GPU vendors worked in harmony to improve their back-ends in the upstream compiler. This would enable development of new cross-vendor GPU languages to compete with the ailing OpenCL. (This is what Apple has done with Metal Shading Language btw, minus the open source part. And what Intel's Larrabee would have enabled.) Even if you disregard the licensing, the union of NV + AMD + Intel capabilities (software stacks included) is so weak that it almost never worth the effort. The comically bad software from AMD and lack of hardware oomph from Intel means the only option is to require NVidia and build on their software support. This is passable for a narrow sector of activities that can tolerate vendor lock-in (Cuda), like short lived HPC projects. All this has resulted in lack of open GPU programming languages. OpenCL is better than nothing, but even if the implementations were of usable quality, it's not a good compiler target for higher level languages and it's not a good language to write by hand. The situation pretty much guarantees GPU computing stays in the fringes for the foreseeable future.

graphene 12y ago
AMD and ARM are working on something that sounds like what you describe: http://en.wikipedia.org/wiki/Heterogeneous_System_Architecture http://en.wikipedia.org/wiki/Heterogeneous_System_Architectu... I believe nVidia has a similar effort going (can't remember the name), so it's still not a single agreed standard, but it's moving in that direction I feel.
- lightcatcher 12y ago
  nVidia's PTX is the most similar thing I can think of (and I know nVidia's GPGPU products pretty well right now). PTX is an ISA for a virtual machine, which makes it similar to HSAIL. PTX doesn't come with a memory model specification though. However, I don't think either HSA or PTX is very important if nVidia, AMD, and Intel (wrt Xeon Phi's) don't start agreeing on standards. I haven't seen anything to convince me that nVidia wants to work within standards, and CUDA seems to have most of the market share right now, so I'm not too hopefully for portable heterogeneous computing in the near future. To my grandparent poster: I don't think working with GPUs is that bad right now if you bind yourself to a vendor. In my experience, drinking the CUDA kool-aid (CU-aid?) isn't all that bad.

cieyo9938y88 12y ago

I dont know much, but I REMEMBER the finger gesture of Linus or the LINUX founder about no no vidia and the big hassles on no no vidia drivers for BSD and FreeBSD. Python, MATLAB and closed source?? - Using Haskell on toy examples can get FUNNY SUPRISE. FUNNY SURPRISE. FUNNY SURPRISE. How about connecting the deep secret GPU to the no function Intel chip TSX transaction and one safe thread, while observing the side channels and effects for better GUESSING GAMES and understanding PSEUDO or fake random numbers?

codygman 12y ago
> Using Haskell on toy examples can get FUNNY SUPRISE. Funny surprises such as?