4 ms·

While the project is interesting, the main benchmark is very misleading. They claim that terracuda matrix multiplication is faster than the one written in cuda.

While the project is interesting, the main benchmark is very misleading. They claim that terracuda matrix multiplication is faster than the one written in cuda. This can be only true if it is compared to NAIVE matrix multiplication in cuda, which can be confirmed by the time that is reported in seconds, which is orders of magnitude slower than the "real" cuda matrix multiplication. Why is that benchmark completely meaningles? Because cuda matrix multiplication is so optimized that no one with a bit of sense would roll out his own, and even if he does, that implementation would have to use very specialized and ugly code - can this be done in terracuda, and is it simpler than in C? Addidionally, the main issue in gpu kernels is not the ugliness of c itself, but the need to optimize algorithms to the hardware of the gpu, thus making the code much more verbose and ugly than the naive c code. That's why terracuda and the likes look fantastic when you compare them with naive C implementations, but they try to solve the wrong problem IMO.

wcrichton 11y ago
Project author here. Your first claim is true, and you can verify it here: https://github.com/willcrichton/terracuda/blob/master/matrix/matrix.cu https://github.com/willcrichton/terracuda/blob/master/matrix... However, I would dispute that it makes the benchmark meaningless. I agree that no one will write the hand-optimized matrix multiply routine in Terra, but I think that misunderstands the use case of the language. Terracuda is intended to make everyday GPGPU computations super easy, so you can speed up simple scripting code that would otherwise be order of magnitudes slower. Look at the other examples in the repo: if you wanted to write a simple hash function, Mandelbrot generator, or renderer, Terracuda makes it simple to accelerate Lua code. There will always be room for super optimized kernels for common functions like matrix multiplication, but I believe there's a strong need for a language that handles everything else.
- dragandj 11y ago
  My point is that while it can sometimes accelerate lua code, it will also (i'd say) decelerate lua code, when the algorithm is not trivially parallelizable (and that is the majority of cases). So, to get the gpu boost, or evento understand how to implement the algorithm properly, I'd have to drop to the GPU specifics. The main question is then: do terracuda helps with that or will it be the obstacle? (it is not a rethorical question; while I suspect it will be the obstacle, I am interested to hear what is your experience with such cases)
  - wcrichto 11y ago
    I strongly doubt writing Terracuda GPU code would ever decelerate your original application, although it might not be as fast as a hand-written CUDA program (and that's an important distinction!). Terracuda doesn't actually lack that many GPU specifics--really it just doesn't provide access to shared memory constructs which are important for hand-tuned matrix mult kernels and blocked algorithms. And that's in the NVPTX API so it's not a true obstacle, just something I haven't gotten around to writing yet.
    - dragandj 11y ago
      But have you tried? I've seen a good share of naive kernels that are slower on a monster gpu than on the cpu. My guess is that most of the code written by users that do not have a clue about the specifics of how gpus work (which is the typical person that terracuda might appeal to) would fall for that trap.
    - jakub_h 11y ago
      > although it might not be as fast as a hand-written CUDA program If you're dealing with larger pieces of code at once (that is, if you have a bigger view of what the programmer wants to run, which you should have), one interesting thing you could do is "exploratory compiling". Try different compilation step sequences and transformations, if you have enough invariants in the code. Machine time is cheap and this could yield some interesting results. Of course, the more constrained the language model is, the more opportunities for this. (There's been some interesting results in this area, like superoptimizers for local optimization, but larger scale compilation would probably use something like genetic algorithms, as you're unlikely to have the opportunity for exhaustive searches.)
jakub_h 11y ago
> Because cuda matrix multiplication is so optimized that no one with a bit of sense would roll out his own Really? What if you have, for example, special matrices, not general ones? What's going to be faster, generating a matrix and then running the fastest general matrix multiplication code money can buy on it, or, even a somewhat slower code, but one that can instead use some extra invariants? > Addidionally, the main issue in gpu kernels is not the ugliness of c itself, but the need to optimize algorithms to the hardware of the gpu, thus making the code much more verbose and ugly than the naive c code. But that sounds like a compiler issue to be solved. And then you don't have to do it by hand, you let the machine find it out. (And I can't help but think that C is terrible for array based programming anyway. Granted, pure Lua might not be great either for the same reasons.)
- dragandj 11y ago
  While these things are desirable, please point to: 1. a solution such as you describe that might be used today 2. how would a typical user of lua (a GPU non-expert) program those special matrices 3. even if he/she can code the algorithm, how can it be done from lua instead of C

wcrichton 11y ago

Project author here. Your first claim is true, and you can verify it here: https://github.com/willcrichton/terracuda/blob/master/matrix/matrix.cu https://github.com/willcrichton/terracuda/blob/master/matrix... However, I would dispute that it makes the benchmark meaningless. I agree that no one will write the hand-optimized matrix multiply routine in Terra, but I think that misunderstands the use case of the language. Terracuda is intended to make everyday GPGPU computations super easy, so you can speed up simple scripting code that would otherwise be order of magnitudes slower. Look at the other examples in the repo: if you wanted to write a simple hash function, Mandelbrot generator, or renderer, Terracuda makes it simple to accelerate Lua code. There will always be room for super optimized kernels for common functions like matrix multiplication, but I believe there's a strong need for a language that handles everything else.

dragandj 11y ago
My point is that while it can sometimes accelerate lua code, it will also (i'd say) decelerate lua code, when the algorithm is not trivially parallelizable (and that is the majority of cases). So, to get the gpu boost, or evento understand how to implement the algorithm properly, I'd have to drop to the GPU specifics. The main question is then: do terracuda helps with that or will it be the obstacle? (it is not a rethorical question; while I suspect it will be the obstacle, I am interested to hear what is your experience with such cases)
- wcrichto 11y ago
  I strongly doubt writing Terracuda GPU code would ever decelerate your original application, although it might not be as fast as a hand-written CUDA program (and that's an important distinction!). Terracuda doesn't actually lack that many GPU specifics--really it just doesn't provide access to shared memory constructs which are important for hand-tuned matrix mult kernels and blocked algorithms. And that's in the NVPTX API so it's not a true obstacle, just something I haven't gotten around to writing yet.
  - dragandj 11y ago
    But have you tried? I've seen a good share of naive kernels that are slower on a monster gpu than on the cpu. My guess is that most of the code written by users that do not have a clue about the specifics of how gpus work (which is the typical person that terracuda might appeal to) would fall for that trap.
  - jakub_h 11y ago
    > although it might not be as fast as a hand-written CUDA program If you're dealing with larger pieces of code at once (that is, if you have a bigger view of what the programmer wants to run, which you should have), one interesting thing you could do is "exploratory compiling". Try different compilation step sequences and transformations, if you have enough invariants in the code. Machine time is cheap and this could yield some interesting results. Of course, the more constrained the language model is, the more opportunities for this. (There's been some interesting results in this area, like superoptimizers for local optimization, but larger scale compilation would probably use something like genetic algorithms, as you're unlikely to have the opportunity for exhaustive searches.)

jakub_h 11y ago

> Because cuda matrix multiplication is so optimized that no one with a bit of sense would roll out his own Really? What if you have, for example, special matrices, not general ones? What's going to be faster, generating a matrix and then running the fastest general matrix multiplication code money can buy on it, or, even a somewhat slower code, but one that can instead use some extra invariants? > Addidionally, the main issue in gpu kernels is not the ugliness of c itself, but the need to optimize algorithms to the hardware of the gpu, thus making the code much more verbose and ugly than the naive c code. But that sounds like a compiler issue to be solved. And then you don't have to do it by hand, you let the machine find it out. (And I can't help but think that C is terrible for array based programming anyway. Granted, pure Lua might not be great either for the same reasons.)

dragandj 11y ago
While these things are desirable, please point to: 1. a solution such as you describe that might be used today 2. how would a typical user of lua (a GPU non-expert) program those special matrices 3. even if he/she can code the algorithm, how can it be done from lua instead of C