7 ms·

It's irrelevant to researchers. Research operates on rapid cycles: prototype, publish, move on. It does impact businesses. It doesn't prevent adoption for e.g.

by wegs 6y ago

It's irrelevant to researchers. Research operates on rapid cycles: prototype, publish, move on. It does impact businesses. It doesn't prevent adoption for e.g. deep learning, but I haven't seen e.g. GPU-based databases reach broad adoption, or many other places where MIMD/SIMD would reduce costs or improve performance. Using classical hardware is clearly cheaper than the business risk and engineering time of relying on a proprietary, closed hardware solution. I'm at the edge, where my workloads don't require GPU, but could benefit from it. This sort of thing factors into decision-making. I dabble in GPU, but never beyond prototypes, for those reasons. I think this is one of the reasons why these devices haven't reached wide-spread marketshare. Most computers sold have an integrated chipset. People buying NVidia GPUs are researchers (who don't care), deep learning applications (who don't have a choice), and gamers. There have been predictions for two decades that GPU-style SIMD and MIMD architectures would displace the centrality of the GPU. Technically, it makes sense. If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. I think the reason that hasn't happened is because x86 and x64 are open and widely-supported. NVidia is a walled garden, and is only practical for markets NVidia explicitly targets. This story plays out over and over. Business people push for closed. Eventually, open comes along, and wipes it out. Sometimes, as with x86 or the iPhone, that leads to increased profits. Sometimes, as with Wikipedia, that kills businesses.

ethbro 6y ago
Hypothetically, from an ISA perspective, why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions? (as in, way more fully than MMX / SSE / AVX) Naive question, because I literally don't know the link between CPU instruction stream and GPGPU instruction stream. But it seems like there would be an opportunity to seize the higher (open) ground at the ISA level, and then force Nvidia to implement its own support for that standard. With the point of being able to run identical code across CPU / CPU-with-embedded-GPU / CPU + GPU. Understand we're talking about mind-boggling levels of complexity here, but it feels like the CPU shops ceded the role of graphical ISA to Nvidia & Microsoft (DirectX).
- deleted 6y ago
  [deleted]
- bitL 6y ago
  > why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions I think there is that latency vs bandwidth trade-off where CPU likes lower latency and GPU higher bandwidth, but you can't achieve the same with a single chip.
  - throwaway2048 6y ago
    GPUs are all a single chip atm.
  - ethbro 6y ago
    I guess this is fundamentally a homogeneous vs heterogeneous ISA question. I.e. is your ISA intended to operate one chip, or multiple cooperative chips / complexes?
    - enos_feedler 6y ago
      Does it make sense for a hardware ISA to express cooperation between chips? I would think HW ISA is meant to control it's local microarchitecture. I could see a virutal ISA or compiler IR built with a multi chip view.
- pjmlp 6y ago
  That is what Intel tried to do with Larabee and failed spectacularly.
  - johncalvinyoung 6y ago
    I wouldn't say they failed. I'd say they gave up on it before product maturity.
    - ethbro 6y ago
      I'd say they aimed for the wrong market (graphics processing, where they were competing against very specialized and experienced competitors) and failed to partner. Maybe Intel ~1998 could have solo-launched a new architecture, but the only way they'd get uptake now is something in cooperation with AMD. And maybe the AMD partnership bridge is burnt from previous shenanigans, but it seems like both AMD and Intel would have incentive in more tightly coupling graphics compatibility to CPU ISA, vs Nvidia designing their own. That said, in that hypothetical reality, Nvidia wouldn't have been able to innovate and execute nearly as fast as they have. As one of my Comp-E professors once quipped, "If a structural engineer ever tells you programming close to processors is easy, ask them how they'd like their job if the physical properties of lumber changed every 2 years."
aseipp 6y ago
GPU based databases haven't reached broad adoption because sending things over the PCIe link is a huge waste of time if you can avoid it. Working around this with custom design like NVLink/NVSwitch do is ridiculously expensive (and why a DGX costs a gajillion dollars), and there is simply not enough volume to subsidize it. They are largely analytics focused, because the parallel hardware can obviously map onto primitives like sequential scan and filters relatively easily. Futhermore, data sizes are not small. Thus the architectures tend to emphasize things like in-memory (VRAM) workloads that get scaled horizontally via RDMA (or RoCE, whatever people are doing these days), which is expensive and limited. Major businesses (i.e. people with money, who nvidia are targeting) already pay for proprietary databases, regularly, every day. That's not the barrier. All of the actual true secret sauce is in the hardware design, and you can't replicate that. You're always at Nvidia's mercy to design solutions to their customers needs. (And frankly, they've done that pretty well, I think.) Sure, you can pay almost $10,000 per Tesla V100 (which aren't going to become magically cheaper, all of a sudden), and buy 8x of them. That's a 256GB working set, for the price of like, $70k USD. It might make sense for some things. For everyone else? Pay $30,000 for a single server, run something like ClickHouse, and you'll have a better overall TCO for a vast majority of workloads. It'll saturate every NVMe drive and all the RAM (terabytes) you can give it, and will scale out too. It's got nothing to do with openness and everything to do with system architecture. You can replicate all of this with whatever AMD has and it won't make a single bit of difference in the market. I don't like the fact Nvidia keeps their software closed either (and in fact it was a motivating reason for replacing my old GTX in my headless server with a Radeon Pro card recently), but the problems you're talking about are not ones of openness. > If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. I think you vastly underestimate the complexity of these platforms and how to extract performance from them, if you think it's as simple as your list comprehension going faster now and you hang up your coat and you're done. Sure, when you're experimenting, that 5x raw wall clock time improvement is nice, and you don't think about whether or not you could have done it with comparable hardware under a different cost profile (5x faster is good, but 5x longer wall clock than the GPU but 15x lower power is a winner). But when you're paying millions of dollars for these systems, it's not a matter of "how to make this thing faster", it's "how do I utilize the resources I have, so 85% of this $300,000 machine isn't sitting idle". This thinking is what drives the design of the overall system, and that's much more complicated.
pjmlp 6y ago
NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent, while hip only works on Linux. As for Python, guess who is on the forefront of GPU Programming with Python, https://www.nvidia.cn/gtc/session-catalog/?search=python https://www.nvidia.cn/gtc/session-catalog/?search=python 41 results, including CUDA based JIT improvements. Meanwhile, at IWOCL & SYCLcon 2020, https://www.iwocl.org/iwocl-2020/conference-program https://www.iwocl.org/iwocl-2020/conference-program 2 sessions, where it is mentioned that PyFR might need OpenCL 3.0 extensions going forward. So if competition is not able to provide, most just get to buy NVidia.
- adev_ 6y ago
  > NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. I don't bite this argument. Nvidia made close to no effort to support OpenCL and promoted their own technology CUDA. Even in 2020, OpenCL support for Nvidia hardware is close to nonexistent. When the main actor of the market does not support a technology, why the hell would you use it or even develop its ecosystem (SYSCL) > The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent. OpenCL 3.0, presented by an Nvidia official(https://khr.io/ocl3slidedeck https://khr.io/ocl3slidedeck), head of the working group on it. A company that made close to 0 effort to support OpenCL 2.0 revert the spec to 1.2. Astonishing right ?
  - pjmlp 6y ago
    OpenCL isn't also supported on Android, where Google pushes Renderscript instead, their own C99 dialect, yet I don't see any uprising against Google. If the 139 member companies (taking NVidia out) listed here aren't able to provide the same quality in hardware, programming language and eco-system improvements against those from NVidia, and vote for a NVidia employee as chairman, then they deserve what they get. https://www.khronos.org/members/list https://www.khronos.org/members/list It is so easy to find a villain instead of acknowledging failure.
fulafel 6y ago
There's definitely a chicken-and-egg problem of the current user and developer base of GPGPU apps being a very tolerant bunch. "It's just a flesh wound" they say about a lot of things that are prohibitive to normal app developers. Maybe it's the natural order of things, or maybe the the "incompatible proprietary C++ dialects with different kinds of crashy drivers on each OS" approach will be suffiently unpalatable to some future generation of programmers.
solidasparagus 6y ago
> I haven't seen e.g. GPU-based databases reach broad adoption Because it's very questionable whether GPU-based databases are generally valuable. GPUs accelerate compute, not all of the other things that databases do and often GPUs are not cost effective. > I dabble in GPU, but never beyond prototypes, for those reasons And I bet if you dabbled a little further you still wouldn't use it because it isn't cost-effective outside of intensive-compute applications. > Technically, it makes sense. If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. This is a very questionable claim. There are tradeoffs to these things (e.g. clock-speed, startup time, etc) and your list comprehensions are probably not compute heavy enough that the tradeoffs are worth it.
- codemk8 6y ago
  Agree. The reason GPUs are not widely adopted in certain areas is not because of open sourced or not, but because it is not cost effective. GPUs are optimized for throughput, not latency.
- wegs 6y ago
  You're making a lot of lousy assumptions. As a few points of reference: * My list comprehensions run over gigabytes of data (but sometimes 3 orders of magnitude bigger or smaller). Stream processing of big data. It's not deep learning, but it's slow and potentially deeply parallel. It would move to MIMD trivially, and SIMD with just a little bit of work. * There are programming languages which support models almost exactly like this for data processing. Sun Labs Fortress comes to mind as an early example. This would generalize to a lot of contexts -- much smaller than you're giving credit for. * Most of the issues, like startup times, are implementation-specific, rather than fundamental, and could be mitigated for much smaller data too. You do need to wrap your head around changes in programming paradigms to make that work. There is some overhead for latency (you'll probably never do well moving a list with 10 items to a GPU), but most of those aren't where programs are performance-bound. * Many database operations map very well to MIMD. * You're making deep assumptions that you're talking to an idiot. That doesn't make you look smart or right, or lead to a constructive discussion.
fluffything 6y ago
> but I haven't seen e.g. GPU-based databases reach broad adoption NVIDIA announced today that Apache Spark 3.0 is built on RAPIDS and showed some benchmarks claiming same perf at 1/5 the cost. Not your classical database application though.
- threeseed 6y ago
  No they have built a GPU accelerated XGBoost library. Spark 3.0 is in preview stage already and there is no mention of RAPIDS anywhere in the code.
hodgesrm 6y ago
> Technically, it makes sense. If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. That assumes the operations, data types, and data arrangement are such that they are vectorizable. In databases at least this kind of optimization is typically done by hand because there are so many constraints. You can't just turn on a flag in the code and make it happen automatically. [1] [1] https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf

by wegs 6y ago

ethbro 6y ago

Hypothetically, from an ISA perspective, why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions? (as in, way more fully than MMX / SSE / AVX) Naive question, because I literally don't know the link between CPU instruction stream and GPGPU instruction stream. But it seems like there would be an opportunity to seize the higher (open) ground at the ISA level, and then force Nvidia to implement its own support for that standard. With the point of being able to run identical code across CPU / CPU-with-embedded-GPU / CPU + GPU. Understand we're talking about mind-boggling levels of complexity here, but it feels like the CPU shops ceded the role of graphical ISA to Nvidia & Microsoft (DirectX).

deleted 6y ago
[deleted]
bitL 6y ago
> why couldn't Intel and AMD extend x86-64 more fully with SIMD / MIMD instructions I think there is that latency vs bandwidth trade-off where CPU likes lower latency and GPU higher bandwidth, but you can't achieve the same with a single chip.
- throwaway2048 6y ago
  GPUs are all a single chip atm.
- ethbro 6y ago
  I guess this is fundamentally a homogeneous vs heterogeneous ISA question. I.e. is your ISA intended to operate one chip, or multiple cooperative chips / complexes?
  - enos_feedler 6y ago
    Does it make sense for a hardware ISA to express cooperation between chips? I would think HW ISA is meant to control it's local microarchitecture. I could see a virutal ISA or compiler IR built with a multi chip view.
pjmlp 6y ago
That is what Intel tried to do with Larabee and failed spectacularly.
- johncalvinyoung 6y ago
  I wouldn't say they failed. I'd say they gave up on it before product maturity.
  - ethbro 6y ago
    I'd say they aimed for the wrong market (graphics processing, where they were competing against very specialized and experienced competitors) and failed to partner. Maybe Intel ~1998 could have solo-launched a new architecture, but the only way they'd get uptake now is something in cooperation with AMD. And maybe the AMD partnership bridge is burnt from previous shenanigans, but it seems like both AMD and Intel would have incentive in more tightly coupling graphics compatibility to CPU ISA, vs Nvidia designing their own. That said, in that hypothetical reality, Nvidia wouldn't have been able to innovate and execute nearly as fast as they have. As one of my Comp-E professors once quipped, "If a structural engineer ever tells you programming close to processors is easy, ask them how they'd like their job if the physical properties of lumber changed every 2 years."

aseipp 6y ago

GPU based databases haven't reached broad adoption because sending things over the PCIe link is a huge waste of time if you can avoid it. Working around this with custom design like NVLink/NVSwitch do is ridiculously expensive (and why a DGX costs a gajillion dollars), and there is simply not enough volume to subsidize it. They are largely analytics focused, because the parallel hardware can obviously map onto primitives like sequential scan and filters relatively easily. Futhermore, data sizes are not small. Thus the architectures tend to emphasize things like in-memory (VRAM) workloads that get scaled horizontally via RDMA (or RoCE, whatever people are doing these days), which is expensive and limited. Major businesses (i.e. people with money, who nvidia are targeting) already pay for proprietary databases, regularly, every day. That's not the barrier. All of the actual true secret sauce is in the hardware design, and you can't replicate that. You're always at Nvidia's mercy to design solutions to their customers needs. (And frankly, they've done that pretty well, I think.) Sure, you can pay almost $10,000 per Tesla V100 (which aren't going to become magically cheaper, all of a sudden), and buy 8x of them. That's a 256GB working set, for the price of like, $70k USD. It might make sense for some things. For everyone else? Pay $30,000 for a single server, run something like ClickHouse, and you'll have a better overall TCO for a vast majority of workloads. It'll saturate every NVMe drive and all the RAM (terabytes) you can give it, and will scale out too. It's got nothing to do with openness and everything to do with system architecture. You can replicate all of this with whatever AMD has and it won't make a single bit of difference in the market. I don't like the fact Nvidia keeps their software closed either (and in fact it was a motivating reason for replacing my old GTX in my headless server with a Radeon Pro card recently), but the problems you're talking about are not ones of openness. > If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. I think you vastly underestimate the complexity of these platforms and how to extract performance from them, if you think it's as simple as your list comprehension going faster now and you hang up your coat and you're done. Sure, when you're experimenting, that 5x raw wall clock time improvement is nice, and you don't think about whether or not you could have done it with comparable hardware under a different cost profile (5x faster is good, but 5x longer wall clock than the GPU but 15x lower power is a winner). But when you're paying millions of dollars for these systems, it's not a matter of "how to make this thing faster", it's "how do I utilize the resources I have, so 85% of this $300,000 machine isn't sitting idle". This thinking is what drives the design of the overall system, and that's much more complicated.

pjmlp 6y ago

NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent, while hip only works on Linux. As for Python, guess who is on the forefront of GPU Programming with Python, https://www.nvidia.cn/gtc/session-catalog/?search=python https://www.nvidia.cn/gtc/session-catalog/?search=python 41 results, including CUDA based JIT improvements. Meanwhile, at IWOCL & SYCLcon 2020, https://www.iwocl.org/iwocl-2020/conference-program https://www.iwocl.org/iwocl-2020/conference-program 2 sessions, where it is mentioned that PyFR might need OpenCL 3.0 extensions going forward. So if competition is not able to provide, most just get to buy NVidia.

adev_ 6y ago
> NVidia is not to blame if the competition is stuck using C, printf debugging for computing shaders, cannot make their minds about which bytecode to support for heterogenous GPGPU programming. I don't bite this argument. Nvidia made close to no effort to support OpenCL and promoted their own technology CUDA. Even in 2020, OpenCL support for Nvidia hardware is close to nonexistent. When the main actor of the market does not support a technology, why the hell would you use it or even develop its ecosystem (SYSCL) > The situation is so bad that OpenCL 1.2 got promoted to OpenCL 3.0 and SYSCL is now backend independent. OpenCL 3.0, presented by an Nvidia official(https://khr.io/ocl3slidedeck https://khr.io/ocl3slidedeck), head of the working group on it. A company that made close to 0 effort to support OpenCL 2.0 revert the spec to 1.2. Astonishing right ?
- pjmlp 6y ago
  OpenCL isn't also supported on Android, where Google pushes Renderscript instead, their own C99 dialect, yet I don't see any uprising against Google. If the 139 member companies (taking NVidia out) listed here aren't able to provide the same quality in hardware, programming language and eco-system improvements against those from NVidia, and vote for a NVidia employee as chairman, then they deserve what they get. https://www.khronos.org/members/list https://www.khronos.org/members/list It is so easy to find a villain instead of acknowledging failure.

fulafel 6y ago

There's definitely a chicken-and-egg problem of the current user and developer base of GPGPU apps being a very tolerant bunch. "It's just a flesh wound" they say about a lot of things that are prohibitive to normal app developers. Maybe it's the natural order of things, or maybe the the "incompatible proprietary C++ dialects with different kinds of crashy drivers on each OS" approach will be suffiently unpalatable to some future generation of programmers.

solidasparagus 6y ago

> I haven't seen e.g. GPU-based databases reach broad adoption Because it's very questionable whether GPU-based databases are generally valuable. GPUs accelerate compute, not all of the other things that databases do and often GPUs are not cost effective. > I dabble in GPU, but never beyond prototypes, for those reasons And I bet if you dabbled a little further you still wouldn't use it because it isn't cost-effective outside of intensive-compute applications. > Technically, it makes sense. If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. This is a very questionable claim. There are tradeoffs to these things (e.g. clock-speed, startup time, etc) and your list comprehensions are probably not compute heavy enough that the tradeoffs are worth it.

codemk8 6y ago
Agree. The reason GPUs are not widely adopted in certain areas is not because of open sourced or not, but because it is not cost effective. GPUs are optimized for throughput, not latency.
wegs 6y ago
You're making a lot of lousy assumptions. As a few points of reference: * My list comprehensions run over gigabytes of data (but sometimes 3 orders of magnitude bigger or smaller). Stream processing of big data. It's not deep learning, but it's slow and potentially deeply parallel. It would move to MIMD trivially, and SIMD with just a little bit of work. * There are programming languages which support models almost exactly like this for data processing. Sun Labs Fortress comes to mind as an early example. This would generalize to a lot of contexts -- much smaller than you're giving credit for. * Most of the issues, like startup times, are implementation-specific, rather than fundamental, and could be mitigated for much smaller data too. You do need to wrap your head around changes in programming paradigms to make that work. There is some overhead for latency (you'll probably never do well moving a list with 10 items to a GPU), but most of those aren't where programs are performance-bound. * Many database operations map very well to MIMD. * You're making deep assumptions that you're talking to an idiot. That doesn't make you look smart or right, or lead to a constructive discussion.

fluffything 6y ago

> but I haven't seen e.g. GPU-based databases reach broad adoption NVIDIA announced today that Apache Spark 3.0 is built on RAPIDS and showed some benchmarks claiming same perf at 1/5 the cost. Not your classical database application though.

threeseed 6y ago
No they have built a GPU accelerated XGBoost library. Spark 3.0 is in preview stage already and there is no mention of RAPIDS anywhere in the code.

hodgesrm 6y ago

> Technically, it makes sense. If I type a list comprehension in Python, it would run at higher speed and lower power on a SIMD or MIMD platform. That assumes the operations, data types, and data arrangement are such that they are vectorizable. In databases at least this kind of optimization is typically done by hand because there are so many constraints. You can't just turn on a flag in the code and make it happen automatically. [1] [1] https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf