5 ms·

Graphics Chips Help Process Big Data Sets in Milliseconds

41 points by robabbott 13y ago

digitailor 13y ago
It's cool that it's essentially a consumer-participatory GPU database, but every time I read about GPU crunching I'm underwhelmed. The power waste and cost is absurd. FPGAs have become as capable, or more, than GPUs at a ridiculously higher level of efficiency. I understand that this platform is about technology accessibility... but having casual computer users burning their graphics card 24/7 seems cute but irrelevant to current methods of supercomputation. I find a lot of people who care about efficient natural resource usage don't think twice about very wasteful computing. Those watts are a whole lot of coal.
- robabbott 13y ago
  I agree. It only makes sense for certain types of workloads and for systems that are constantly in use (like supercomputers). This has been borne out through the movement of Bitcoin miner rigs from GPUs to FPGAs to ASICs. The power consumption and comparative performance of the GPU cannot win out over these devices.
  - digitailor 13y ago
    You're right, I think BTC mining is our current best example of the trajectory of HPC. It proved from an economic point of view that GPUs are a kludge at best. BTC has really spread awareness about FPGA and ASIC technology. Highly capable FPGAs (not just the standard glue logic type) have mostly been the provence of HFT and the NSA until now. And finance has only really been hip for the past 3-4 years as they chase ultra-low trading latencies. And now a MicroZed costs $199 and Digilent is releasing the Zybo at $149! Technology sea change, anyone? It's going to make coding REAL interesting again...
    - modeless 13y ago
      GPUs are no more a kludge or waste of power than CPUs. They just provide access to a different point on the efficiency vs. ease of programming graph. FPGAs may be getting cheaper but they're still much more difficult to program than GPUs. The sea change will come when someone makes an FPGA that's an order of magnitude easier to program.
      - digitailor 13y ago
        Totally agree, that's why I mentioned this is about technology accessibility and really not much else. That's what I mean by kludge- it's far from ideal, but it "just works." And it's funny you mention CPUs- Von Neumann Turing machines are looking like a real kludge to me at this point as well... In my view SOCs like the Zynq are the real news, and those don't get talked about much on HN. Sequential PLUS parallel with high efficiency. So what if it's difficult to understand? Trying to really flexes the brain, and it's been worth it for me. But not easy. That's why I wanted to start this conversation, I think it's germane to GPU efforts and people may not be aware. I would be more excited about a project like this spreading FPGA tech, and more effort being spent there. My company is doing it and we want a bigger open-source community with us. (We're not in a position to release code yet, but we will.) Effort is energy. I'm hoping mentioning these technologies will give people pursuing projects like this some options on where to spend their efforts. There's a finite pool of people who are working in this space and I hope it grows.
        modeless 13y ago
        Agreed that accessible FPGA SoCs are super exciting and I want to learn more about them. In the past when I've looked at these I've always been disappointed in the raw FLOPS available vs. GPUs, though it's difficult to compare directly, and I'm totally unfamiliar with the FPGA world. If I buy e.g. a ZYBO, make something cool, and want to scale it up to beat e.g. an NVIDIA GTX Titan in raw performance (4.5 TFLOPS), what are my options?
        digitailor 13y ago
- mtdewcmu 13y ago
  I don't think power is relevant here, because the GPU is being used to reduce latency when running queries interactively, so it's only on for a small fraction of the time.
- deutronium 13y ago
  You can use OpenCL, which is normally used for programming GPUs, on FPGAs http://www.altera.com/products/software/opencl/opencl-index.html http://www.altera.com/products/software/opencl/opencl-index.... Also see http://www.anandtech.com/show/7334/a-look-at-alteras-opencl-sdk-for-fpgas http://www.anandtech.com/show/7334/a-look-at-alteras-opencl-... for more information
- yetanotherphd 13y ago
  Someone correct me if I'm out of line here, but I think it would be appropriate to mention your personal interest in FPGA's (which I discovered from your comments lower down). On the point itself, "accessibility" is probably understating the nature of the problem. Actually being able to program for a platform, and how easily you can do it, is a huge issue. Issues of "accessibility" apply as much to huge companies with giant server farms (note how little even GPU computing is used by them) as they do to tinkerers.
  - digitailor 13y ago
    You're not out of line, and I haven't tried to hide the fact that I'm now professionally interested, in fact that's all I'm blabbing about below. But that's no coincidence- this is why I got into FPGAs: the problem spaces that are like what this cool project addresses. But I rejected GPUs as a platform completely for the reasons I list. And that's how I came to be interested in FPGAs. I am not an EE and it was a hard decision to make the switch to pursuing this, and I'm sharing that experience in the hope of inspiring/meeting others. As well as trying to open myself up to input/advice from people more experienced than I.
    - yetanotherphd 13y ago
      Fair point, as often happens I forgot to consider the possibility that your preference for FPGA's caused you to get a job selling an FPGA platform and not vice versa. In any case, my own viewpoint is that CUDA and OpenCL are the best that has been produced by a huge effort by well financed and technically sophisticated groups. GPU computing offers advantages in flops/watt and flops/dollar, and yet is still not widely utilized because of the increased programming complexity. Given this, I think it will be very hard for a small group to compete using a completely new architecture. On the other hand, you are starting with a blank slate and you are controlling the entire stack, so I hope you can use that to your advantage.
- cmccabe 13y ago
  It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs." It depends on what you're trying to do! If you want to multiply big matrices, GPUs are great for that. A lot of scientific applications work well on GPUs. And obviously, GPUs are great for graphics. Also, FPGAs are no longer blank templates on which you can stamp any design. The new ones all come with built-in "IP blocks" which you can't change. So you get a bunch of gates you can modify, but also perhaps dozen CPUs and a bank of memory, or so. Maybe someday GPUs and FPGAs will converge-- the former are getting more flexible, and the latter are getting less so. The biggest problem with contemporary GPUs is that they're I/O-starved, which I don't see anywhere in your comments. PCI-e is just not enough bandwidth. That is why GPUs are a sideshow in big data. It doesn't matter how many cores you have if you're sipping your data through a straw. The power consumption argument seems like a strawman. Replace a few incandescent lightbulbs in your home with LED ones. Congratulations. You can now run your GPU 24/7 and come out ahead in power consumption.
  - digitailor 13y ago
    >> It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs." Umm, you took "at a ridiculously higher level of efficiency" out of my quote in order to call it silly? C'mon man. If you've ever diligently crunched numbers on FPGA versus GPU monthly electricity costs at scale, I don't think you'd be capable of saying power efficiency is a strawman. If the idealistic efficient-resource-usage aspect is not convincing enough for you, we are talking millions and millions of dollars here (actually more). Look at Amazon GPU numbers and get back to me. We have detailed breakdowns of price comparisons that have been worked out in depth using the same algorithm. This is nothing like swapping out lightbulbs, that's an actual absurd and uninformed statement that can be verified and quantified as such. And I do mention both IP blocks (even in quotes as you do) and how SoC's CPU-PL communication is much faster from being same-chip-integrated, hours before you did, so I'm not sure what you mean. I'm also not getting why you use hard IP blocks as an argument that FPGAs are getting less flexible. They are helping to advance FPGA accessibility and flexibility a ton. Why do you think they're there? The development time savings are huge and the testing and optimization is by nature better. The 7000 series has pretty heavy options, you can get a lot PL all your own on them. We've been specifically talking about an ARM+PL SoC, are you trying to say that an FPGA like a Virtex or Kintex is somehow not a "blank slate"? You want no standardization of any kind? The lack of that is a huge problem, and it's getting solutions.
justin66 13y ago
Has the guy released any code yet?
- robabbott 13y ago
  I haven't seen any releases yet. One article says he plans to open source it within the next year.
  - tmostak 13y ago
    Hey, MapD author here (Todd Mostak). I'm working on a release now - hopefully it will be out by the end of the year! Would really appreciate help from anyone interested in the project - todd@map-d.com
    - robabbott 13y ago
      Todd - Would love to help you out. I will send you my email.
    - justin66 13y ago
      That sounds really interesting. I will send you an email.
    - peatmoss 13y ago
      As a side note, I was excited to see that there was some geospatial support in there. That's certainly piqued my interest! How much of a focus was spatial support?
polskibus 13y ago
I wonder how it compares to: http://wiki.postgresql.org/wiki/PGStrom http://wiki.postgresql.org/wiki/PGStrom (in terms of architecture, not performance benchmarks)
robabbott 13y ago
In addition to GPU, Cray has recently added the Intel Phi coprocessor to its XC30 Cascade supercomputers (http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1860101&highlight= http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...). I think that this supports the argument that certain problems are better handled on traditional processors than on GPGPU platforms.
- pbsd 13y ago
  The Xeon Phi resembles a GPU much more than it does a CPU. The original project, Larrabee, was at one point meant to be a GPU that competed with NVIDIA and ATI.
  - zurn 13y ago
    It's a multicore x86. It resembles GPUs only in the sense that it sits on its own PCIe board and has a lot of cores, but the programming model is just x86. It would have been very different from the NVidia/AMD competition if it had ended up as a GPU. One of the biggest obstacles to GPGPU exploitation is being at the mercy of each vendors proprietary software stack and the resulting fragmentation & lack of openness. It's like the pre-PC era, without Unix... Larrabee might have helped. Now Xeon Phi is a niche product.
    - pbsd 13y ago
      It is an x86 with very weak in-order cores, which instead have very large (512-bit) vector units. Very much like an NVIDIA "symmetric multiprocessor". If you try to program the Xeon Phi anything like a general-purpose CPU, you will not get within 5% of its power --- the programming model is essentially that of a GPU. The one thing that makes the Xeon Phi more general-purposy than current GPUs is the cache-coherence across cores.
      - zurn 13y ago
        They are programmed the way shared memory parallel machines have been used since time immemorial in the HPC world. Hundreds of cores is normal there. The same Intel software stack (OpenMP, Threaded Building Blocks, parallel Fortran etc) is used, same one that Intel markets for regular HPC. Yeah, you have slower cores and the SIMD is twice as wide, but its main selling point is that you get to keep the regular programming model - unlike with NVidia or AMD. Along with benefits that come with it (mature software, openness, etc). See eg. Intel's Xeon Phi Programmin Guide for an intro: http://software.intel.com/sites/default/files/article/330164/an-overview-of-programming-for-intel-xeon-processors-and-intel-xeon-phi-coprocessors.pdf http://software.intel.com/sites/default/files/article/330164...

41 points by robabbott 13y ago

It's cool that it's essentially a consumer-participatory GPU database, but every time I read about GPU crunching I'm underwhelmed. The power waste and cost is absurd. FPGAs have become as capable, or more, than GPUs at a ridiculously higher level of efficiency. I understand that this platform is about technology accessibility... but having casual computer users burning their graphics card 24/7 seems cute but irrelevant to current methods of supercomputation. I find a lot of people who care about efficient natural resource usage don't think twice about very wasteful computing. Those watts are a whole lot of coal.

robabbott 13y ago
I agree. It only makes sense for certain types of workloads and for systems that are constantly in use (like supercomputers). This has been borne out through the movement of Bitcoin miner rigs from GPUs to FPGAs to ASICs. The power consumption and comparative performance of the GPU cannot win out over these devices.
- digitailor 13y ago
  You're right, I think BTC mining is our current best example of the trajectory of HPC. It proved from an economic point of view that GPUs are a kludge at best. BTC has really spread awareness about FPGA and ASIC technology. Highly capable FPGAs (not just the standard glue logic type) have mostly been the provence of HFT and the NSA until now. And finance has only really been hip for the past 3-4 years as they chase ultra-low trading latencies. And now a MicroZed costs $199 and Digilent is releasing the Zybo at $149! Technology sea change, anyone? It's going to make coding REAL interesting again...
  - modeless 13y ago
    GPUs are no more a kludge or waste of power than CPUs. They just provide access to a different point on the efficiency vs. ease of programming graph. FPGAs may be getting cheaper but they're still much more difficult to program than GPUs. The sea change will come when someone makes an FPGA that's an order of magnitude easier to program.
    - digitailor 13y ago
      Totally agree, that's why I mentioned this is about technology accessibility and really not much else. That's what I mean by kludge- it's far from ideal, but it "just works." And it's funny you mention CPUs- Von Neumann Turing machines are looking like a real kludge to me at this point as well... In my view SOCs like the Zynq are the real news, and those don't get talked about much on HN. Sequential PLUS parallel with high efficiency. So what if it's difficult to understand? Trying to really flexes the brain, and it's been worth it for me. But not easy. That's why I wanted to start this conversation, I think it's germane to GPU efforts and people may not be aware. I would be more excited about a project like this spreading FPGA tech, and more effort being spent there. My company is doing it and we want a bigger open-source community with us. (We're not in a position to release code yet, but we will.) Effort is energy. I'm hoping mentioning these technologies will give people pursuing projects like this some options on where to spend their efforts. There's a finite pool of people who are working in this space and I hope it grows.
      - modeless 13y ago
        Agreed that accessible FPGA SoCs are super exciting and I want to learn more about them. In the past when I've looked at these I've always been disappointed in the raw FLOPS available vs. GPUs, though it's difficult to compare directly, and I'm totally unfamiliar with the FPGA world. If I buy e.g. a ZYBO, make something cool, and want to scale it up to beat e.g. an NVIDIA GTX Titan in raw performance (4.5 TFLOPS), what are my options?
        digitailor 13y ago
mtdewcmu 13y ago
I don't think power is relevant here, because the GPU is being used to reduce latency when running queries interactively, so it's only on for a small fraction of the time.
deutronium 13y ago
You can use OpenCL, which is normally used for programming GPUs, on FPGAs http://www.altera.com/products/software/opencl/opencl-index.html http://www.altera.com/products/software/opencl/opencl-index.... Also see http://www.anandtech.com/show/7334/a-look-at-alteras-opencl-sdk-for-fpgas http://www.anandtech.com/show/7334/a-look-at-alteras-opencl-... for more information
yetanotherphd 13y ago
Someone correct me if I'm out of line here, but I think it would be appropriate to mention your personal interest in FPGA's (which I discovered from your comments lower down). On the point itself, "accessibility" is probably understating the nature of the problem. Actually being able to program for a platform, and how easily you can do it, is a huge issue. Issues of "accessibility" apply as much to huge companies with giant server farms (note how little even GPU computing is used by them) as they do to tinkerers.
- digitailor 13y ago
  You're not out of line, and I haven't tried to hide the fact that I'm now professionally interested, in fact that's all I'm blabbing about below. But that's no coincidence- this is why I got into FPGAs: the problem spaces that are like what this cool project addresses. But I rejected GPUs as a platform completely for the reasons I list. And that's how I came to be interested in FPGAs. I am not an EE and it was a hard decision to make the switch to pursuing this, and I'm sharing that experience in the hope of inspiring/meeting others. As well as trying to open myself up to input/advice from people more experienced than I.
  - yetanotherphd 13y ago
    Fair point, as often happens I forgot to consider the possibility that your preference for FPGA's caused you to get a job selling an FPGA platform and not vice versa. In any case, my own viewpoint is that CUDA and OpenCL are the best that has been produced by a huge effort by well financed and technically sophisticated groups. GPU computing offers advantages in flops/watt and flops/dollar, and yet is still not widely utilized because of the increased programming complexity. Given this, I think it will be very hard for a small group to compete using a completely new architecture. On the other hand, you are starting with a blank slate and you are controlling the entire stack, so I hope you can use that to your advantage.
cmccabe 13y ago
It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs." It depends on what you're trying to do! If you want to multiply big matrices, GPUs are great for that. A lot of scientific applications work well on GPUs. And obviously, GPUs are great for graphics. Also, FPGAs are no longer blank templates on which you can stamp any design. The new ones all come with built-in "IP blocks" which you can't change. So you get a bunch of gates you can modify, but also perhaps dozen CPUs and a bank of memory, or so. Maybe someday GPUs and FPGAs will converge-- the former are getting more flexible, and the latter are getting less so. The biggest problem with contemporary GPUs is that they're I/O-starved, which I don't see anywhere in your comments. PCI-e is just not enough bandwidth. That is why GPUs are a sideshow in big data. It doesn't matter how many cores you have if you're sipping your data through a straw. The power consumption argument seems like a strawman. Replace a few incandescent lightbulbs in your home with LED ones. Congratulations. You can now run your GPU 24/7 and come out ahead in power consumption.
- digitailor 13y ago
  >> It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs." Umm, you took "at a ridiculously higher level of efficiency" out of my quote in order to call it silly? C'mon man. If you've ever diligently crunched numbers on FPGA versus GPU monthly electricity costs at scale, I don't think you'd be capable of saying power efficiency is a strawman. If the idealistic efficient-resource-usage aspect is not convincing enough for you, we are talking millions and millions of dollars here (actually more). Look at Amazon GPU numbers and get back to me. We have detailed breakdowns of price comparisons that have been worked out in depth using the same algorithm. This is nothing like swapping out lightbulbs, that's an actual absurd and uninformed statement that can be verified and quantified as such. And I do mention both IP blocks (even in quotes as you do) and how SoC's CPU-PL communication is much faster from being same-chip-integrated, hours before you did, so I'm not sure what you mean. I'm also not getting why you use hard IP blocks as an argument that FPGAs are getting less flexible. They are helping to advance FPGA accessibility and flexibility a ton. Why do you think they're there? The development time savings are huge and the testing and optimization is by nature better. The 7000 series has pretty heavy options, you can get a lot PL all your own on them. We've been specifically talking about an ARM+PL SoC, are you trying to say that an FPGA like a Virtex or Kintex is somehow not a "blank slate"? You want no standardization of any kind? The lack of that is a huge problem, and it's getting solutions.

justin66 13y ago

Has the guy released any code yet?

robabbott 13y ago
I haven't seen any releases yet. One article says he plans to open source it within the next year.
- tmostak 13y ago
  Hey, MapD author here (Todd Mostak). I'm working on a release now - hopefully it will be out by the end of the year! Would really appreciate help from anyone interested in the project - todd@map-d.com
  - robabbott 13y ago
    Todd - Would love to help you out. I will send you my email.
  - justin66 13y ago
    That sounds really interesting. I will send you an email.
  - peatmoss 13y ago
    As a side note, I was excited to see that there was some geospatial support in there. That's certainly piqued my interest! How much of a focus was spatial support?

polskibus 13y ago

I wonder how it compares to: http://wiki.postgresql.org/wiki/PGStrom http://wiki.postgresql.org/wiki/PGStrom (in terms of architecture, not performance benchmarks)

robabbott 13y ago

In addition to GPU, Cray has recently added the Intel Phi coprocessor to its XC30 Cascade supercomputers (http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1860101&highlight= http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...). I think that this supports the argument that certain problems are better handled on traditional processors than on GPGPU platforms.

pbsd 13y ago
The Xeon Phi resembles a GPU much more than it does a CPU. The original project, Larrabee, was at one point meant to be a GPU that competed with NVIDIA and ATI.
- zurn 13y ago
  It's a multicore x86. It resembles GPUs only in the sense that it sits on its own PCIe board and has a lot of cores, but the programming model is just x86. It would have been very different from the NVidia/AMD competition if it had ended up as a GPU. One of the biggest obstacles to GPGPU exploitation is being at the mercy of each vendors proprietary software stack and the resulting fragmentation & lack of openness. It's like the pre-PC era, without Unix... Larrabee might have helped. Now Xeon Phi is a niche product.
  - pbsd 13y ago
    It is an x86 with very weak in-order cores, which instead have very large (512-bit) vector units. Very much like an NVIDIA "symmetric multiprocessor". If you try to program the Xeon Phi anything like a general-purpose CPU, you will not get within 5% of its power --- the programming model is essentially that of a GPU. The one thing that makes the Xeon Phi more general-purposy than current GPUs is the cache-coherence across cores.
    - zurn 13y ago
      They are programmed the way shared memory parallel machines have been used since time immemorial in the HPC world. Hundreds of cores is normal there. The same Intel software stack (OpenMP, Threaded Building Blocks, parallel Fortran etc) is used, same one that Intel markets for regular HPC. Yeah, you have slower cores and the SIMD is twice as wide, but its main selling point is that you get to keep the regular programming model - unlike with NVidia or AMD. Along with benefits that come with it (mature software, openness, etc). See eg. Intel's Xeon Phi Programmin Guide for an intro: http://software.intel.com/sites/default/files/article/330164/an-overview-of-programming-for-intel-xeon-processors-and-intel-xeon-phi-coprocessors.pdf http://software.intel.com/sites/default/files/article/330164...