5 ms·

> If they make that chip I have no doubt that it will revolutionize the entire field. Ha, ha. Yeah… no. For one thing, there's no mention of its power consumpt

by PeCaN 10y ago

> If they make that chip I have no doubt that it will revolutionize the entire field. Ha, ha. Yeah… no. For one thing, there's no mention of its power consumption, no CUDA support, questionable memory design (sure, you can get a million TFlops without cache, now try to get that chip to do anything useful), etc. Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something.

modeless 10y ago
> Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. Ha, ha. Yeah… no. > there's no mention of its power consumption Can't be tremendously higher than Pascal for reasons of physics. > sure, you can get a million TFlops without cache, now try to get that chip to do anything useful "Without cache" is certainly an exaggeration. It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. Neural net workloads are incredibly predictable and homogeneous and are essentially the perfect scenario for hand optimization of data flows to beat automatic caching. > no CUDA support You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway.
- PeCaN 10y ago
  > > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. > Ha, ha. Yeah… no. Oh? You don't think they'd acquihire a machine learning startup to work on their compute/machine learning offering in Xeon Phi? I would be very, very surprised if Intel introduces a new machine-learning-oriented processor—though that would be quite interesting, so let's hope you're right. > Can't be tremendously higher than Pascal for reasons of physics. I wouldn't be so quick to assume that. Nvidia spends a lot of resources optimizing power efficiency. To think a much smaller company could match their power efficiency while scaling performace¹ is plausible, but not likely. If anything, physics would say that it can't be much lower than Pascal, but nothing about higher. For example, the large gap in power efficiency between Nvidia and AMD's 28nm GPUs last generation. It's not correct that similar size process nodes always have similar power consumption—that comes down a lot to architecture. > It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. You're right, the link² posted in another comment clears that up—and you're right that it could be very good for performance, but also very difficult to optimize for. > You're just being silly now. You're spouting stupid shit (like that bit about physics and power consumption… not to mention the ridiculous hyperbole in your original comment) and call me silly. I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. > CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway. Which is precisely why it won't revolutionize the field at all. Deep learning is relatively locked in to CUDA. Essentially, extraordinary claims (such as revolutionizing deep learning) require extraordinary evidence—evidence which neither you nor Nervana have shown the slightest signs of producing. -- ¹ I just noticed the nextplatform link specified teraops instead of teraflops. I don't know if that's referring to integer operations, would be… unusual, to say the least, for a deep learning chip. If it is, then we don't really know what its single-precision float performance is. ² https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
  - singhblom 10y ago
    They don't state flops because they don't do floats. The nervana chip has some weird fixed point format that they think works better for deep learning. I've heard similar noise out of Google (the other TPU company ...), so I wouldn't be surprised if we'll see the same in many of the specialized chips that people will build for deep learning. I really don't think CUDA support is important for nervana's offering. They think of themselves as the Apples of deep learning - they want to offer an integrated stack from the chip all the way to the APIs. The way most people use deep learning you don't really need to know CUDA, you just need to use a library that is fast. So it's enough that nervana's engineers know how to write deep learning libs for their own chip. Furthermore I can't see Intel caring that much about CUDA support, since CUDA is owned by Nvidia. What I've heard about the chip makes it sound really exciting. Many of the trade-offs in deep learning are different from the ones you do in graphics, so specialized hardware makes sense.
  - Alphasite_ 10y ago
    Google's TPU claims to have a >10x perf/w advantage over GPUs (they state competitors, but GPUs are the defacto standard) and thats frankly more important that raw throughput at scale. If Intel can ape Nvidia at deep learning then it is a big growth opportunity for them.
  - sctb 10y ago
    > I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. Personal attacks are not OK on Hacker News. Please leave these out of comments.
    - PeCaN 10y ago
      sieg heil
- mastazi 10y ago
  > You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA I would love to agree with you but the reality is that CUDA is the de-facto industry standard e.g. [1]. > and this isn't a general purpose processor anyway. The "GP" part in GPGPU would like to disagree... [2] [1] https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software https://en.wikipedia.org/wiki/Comparison_of_deep_learning_so... [2] http://www.nvidia.com/object/cuda_home_new.html http://www.nvidia.com/object/cuda_home_new.html
  - yvdriess 10y ago
    The GP part stands for "something else than graphics". CUDA is not general purpose in the sense that a lot of types of computation and data movement results in terrible performance. You buy GPU accelerators for performance reasons. Case in point, GPUs are standard (for now) for training DNNs, but the big server farms that actually do the scoring run on Xeons.
  - redcalx 10y ago
    What happened to OpenCL? https://en.wikipedia.org/wiki/OpenCL https://en.wikipedia.org/wiki/OpenCL
    - dagw 10y ago
      Nothing per se. It's just that it lags far enough behind CUDA that unless you have a specific reason to avoid NVIDIA hardware there are very few pragmatic reasons to use it.
  - Alphasite_ 10y ago
    Google has implemented their own deep learning chip with no CUDA support and its serving their purposes well. Why not another industry giant, Intel?
    - camuel 10y ago
      Google is very different from Intel on these fronts at least: 1. Google's open framework TensorFlow is exploding in popularity effectively moving the common interface up from CUDA. 2. Google has a huge need for internal AI/DeepLearning 3. Google has cloud and AI/DeepLearning as a service business 4. Google doesn't sells hardware However, Intel dominance is not to be underestimated, they definitely can make industry wide impact quickly. Just saying you can not easily draw parallels here

by PeCaN 10y ago

modeless 10y ago

> Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. Ha, ha. Yeah… no. > there's no mention of its power consumption Can't be tremendously higher than Pascal for reasons of physics. > sure, you can get a million TFlops without cache, now try to get that chip to do anything useful "Without cache" is certainly an exaggeration. It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. Neural net workloads are incredibly predictable and homogeneous and are essentially the perfect scenario for hand optimization of data flows to beat automatic caching. > no CUDA support You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway.

PeCaN 10y ago
> > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. > Ha, ha. Yeah… no. Oh? You don't think they'd acquihire a machine learning startup to work on their compute/machine learning offering in Xeon Phi? I would be very, very surprised if Intel introduces a new machine-learning-oriented processor—though that would be quite interesting, so let's hope you're right. > Can't be tremendously higher than Pascal for reasons of physics. I wouldn't be so quick to assume that. Nvidia spends a lot of resources optimizing power efficiency. To think a much smaller company could match their power efficiency while scaling performace¹ is plausible, but not likely. If anything, physics would say that it can't be much lower than Pascal, but nothing about higher. For example, the large gap in power efficiency between Nvidia and AMD's 28nm GPUs last generation. It's not correct that similar size process nodes always have similar power consumption—that comes down a lot to architecture. > It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. You're right, the link² posted in another comment clears that up—and you're right that it could be very good for performance, but also very difficult to optimize for. > You're just being silly now. You're spouting stupid shit (like that bit about physics and power consumption… not to mention the ridiculous hyperbole in your original comment) and call me silly. I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. > CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway. Which is precisely why it won't revolutionize the field at all. Deep learning is relatively locked in to CUDA. Essentially, extraordinary claims (such as revolutionizing deep learning) require extraordinary evidence—evidence which neither you nor Nervana have shown the slightest signs of producing. -- ¹ I just noticed the nextplatform link specified teraops instead of teraflops. I don't know if that's referring to integer operations, would be… unusual, to say the least, for a deep learning chip. If it is, then we don't really know what its single-precision float performance is. ² https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
- singhblom 10y ago
  They don't state flops because they don't do floats. The nervana chip has some weird fixed point format that they think works better for deep learning. I've heard similar noise out of Google (the other TPU company ...), so I wouldn't be surprised if we'll see the same in many of the specialized chips that people will build for deep learning. I really don't think CUDA support is important for nervana's offering. They think of themselves as the Apples of deep learning - they want to offer an integrated stack from the chip all the way to the APIs. The way most people use deep learning you don't really need to know CUDA, you just need to use a library that is fast. So it's enough that nervana's engineers know how to write deep learning libs for their own chip. Furthermore I can't see Intel caring that much about CUDA support, since CUDA is owned by Nvidia. What I've heard about the chip makes it sound really exciting. Many of the trade-offs in deep learning are different from the ones you do in graphics, so specialized hardware makes sense.
- Alphasite_ 10y ago
  Google's TPU claims to have a >10x perf/w advantage over GPUs (they state competitors, but GPUs are the defacto standard) and thats frankly more important that raw throughput at scale. If Intel can ape Nvidia at deep learning then it is a big growth opportunity for them.
- sctb 10y ago
  > I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. Personal attacks are not OK on Hacker News. Please leave these out of comments.
  - PeCaN 10y ago
    sieg heil
mastazi 10y ago
> You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA I would love to agree with you but the reality is that CUDA is the de-facto industry standard e.g. [1]. > and this isn't a general purpose processor anyway. The "GP" part in GPGPU would like to disagree... [2] [1] https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software https://en.wikipedia.org/wiki/Comparison_of_deep_learning_so... [2] http://www.nvidia.com/object/cuda_home_new.html http://www.nvidia.com/object/cuda_home_new.html
- yvdriess 10y ago
  The GP part stands for "something else than graphics". CUDA is not general purpose in the sense that a lot of types of computation and data movement results in terrible performance. You buy GPU accelerators for performance reasons. Case in point, GPUs are standard (for now) for training DNNs, but the big server farms that actually do the scoring run on Xeons.
- redcalx 10y ago
  What happened to OpenCL? https://en.wikipedia.org/wiki/OpenCL https://en.wikipedia.org/wiki/OpenCL
  - dagw 10y ago
    Nothing per se. It's just that it lags far enough behind CUDA that unless you have a specific reason to avoid NVIDIA hardware there are very few pragmatic reasons to use it.
- Alphasite_ 10y ago
  Google has implemented their own deep learning chip with no CUDA support and its serving their purposes well. Why not another industry giant, Intel?
  - camuel 10y ago
    Google is very different from Intel on these fronts at least: 1. Google's open framework TensorFlow is exploding in popularity effectively moving the common interface up from CUDA. 2. Google has a huge need for internal AI/DeepLearning 3. Google has cloud and AI/DeepLearning as a service business 4. Google doesn't sells hardware However, Intel dominance is not to be underestimated, they definitely can make industry wide impact quickly. Just saying you can not easily draw parallels here