7 ms·

Intel to buy deep-learning startup Nervana Systems for at least $350M

143 points by flinner 10y ago

p1esk 10y ago
Here are some details about their upcoming chip: http://www.nextplatform.com/2016/08/08/deep-learning-chip-upstart-set-take-gpus-task/ http://www.nextplatform.com/2016/08/08/deep-learning-chip-up... Summary: - 28nm - looks similar to P100 (interposer with HBM) - 55 teraops/s performance - custom number format (variable length fixed point?) - simplified memory architecture (no cache?) - no info about power consumption
- modeless 10y ago
  Wow, really important details here; very interesting. Now imagine this chip, but shrunk to Intel's latest 14nm process. If they make that chip I have no doubt that it will revolutionize the entire field.
  - PeCaN 10y ago
    > If they make that chip I have no doubt that it will revolutionize the entire field. Ha, ha. Yeah… no. For one thing, there's no mention of its power consumption, no CUDA support, questionable memory design (sure, you can get a million TFlops without cache, now try to get that chip to do anything useful), etc. Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something.
    - modeless 10y ago
      > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. Ha, ha. Yeah… no. > there's no mention of its power consumption Can't be tremendously higher than Pascal for reasons of physics. > sure, you can get a million TFlops without cache, now try to get that chip to do anything useful "Without cache" is certainly an exaggeration. It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. Neural net workloads are incredibly predictable and homogeneous and are essentially the perfect scenario for hand optimization of data flows to beat automatic caching. > no CUDA support You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway.
      - PeCaN 10y ago
        > > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. > Ha, ha. Yeah… no. Oh? You don't think they'd acquihire a machine learning startup to work on their compute/machine learning offering in Xeon Phi? I would be very, very surprised if Intel introduces a new machine-learning-oriented processor—though that would be quite interesting, so let's hope you're right. > Can't be tremendously higher than Pascal for reasons of physics. I wouldn't be so quick to assume that. Nvidia spends a lot of resources optimizing power efficiency. To think a much smaller company could match their power efficiency while scaling performace¹ is plausible, but not likely. If anything, physics would say that it can't be much lower than Pascal, but nothing about higher. For example, the large gap in power efficiency between Nvidia and AMD's 28nm GPUs last generation. It's not correct that similar size process nodes always have similar power consumption—that comes down a lot to architecture. > It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. You're right, the link² posted in another comment clears that up—and you're right that it could be very good for performance, but also very difficult to optimize for. > You're just being silly now. You're spouting stupid shit (like that bit about physics and power consumption… not to mention the ridiculous hyperbole in your original comment) and call me silly. I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. > CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway. Which is precisely why it won't revolutionize the field at all. Deep learning is relatively locked in to CUDA. Essentially, extraordinary claims (such as revolutionizing deep learning) require extraordinary evidence—evidence which neither you nor Nervana have shown the slightest signs of producing. -- ¹ I just noticed the nextplatform link specified teraops instead of teraflops. I don't know if that's referring to integer operations, would be… unusual, to say the least, for a deep learning chip. If it is, then we don't really know what its single-precision float performance is. ² https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
        singhblom 10y ago
        They don't state flops because they don't do floats. The nervana chip has some weird fixed point format that they think works better for deep learning. I've heard similar noise out of Google (the other TPU company ...), so I wouldn't be surprised if we'll see the same in many of the specialized chips that people will build for deep learning. I really don't think CUDA support is important for nervana's offering. They think of themselves as the Apples of deep learning - they want to offer an integrated stack from the chip all the way to the APIs. The way most people use deep learning you don't really need to know CUDA, you just need to use a library that is fast. So it's enough that nervana's engineers know how to write deep learning libs for their own chip. Furthermore I can't see Intel caring that much about CUDA support, since CUDA is owned by Nvidia. What I've heard about the chip makes it sound really exciting. Many of the trade-offs in deep learning are different from the ones you do in graphics, so specialized hardware makes sense.
  - argonaut 10y ago
    What PeCaN is driving at, is that saying something like you "have no doubt that it will revolutionize the entire field," is simply hyperbole that is not supported. We'll see. Nervana didn't really gain that much traction, which is why it's being acquired. Intel hasn't had much of a deep learning presence so far, which is why they are acquiring.
- pjscott 10y ago
  > simplified memory architecture (no cache?) No automatic cache hierarchy. There is local memory, which you manage explicitly. More info here: https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
- Katydid 10y ago
  Most of the questions asked in these threads on HN are answered in an update article they posted this morning about 14 nanometer, the Neon framework's future and such. Also, looks like everyone who broke the news did got the acquisition amount wrong. http://www.nextplatform.com/2016/08/10/nervana-ceo-intel-acquisition-future-technology-outlook/ http://www.nextplatform.com/2016/08/10/nervana-ceo-intel-acq...
singularity2001 10y ago
@OP: that's Nervana with 'e' ;)
- flinner 10y ago
  Thank you! Updated
mgreg 10y ago
Likely a good move for Intel to get in on one of the faster growing areas of computing. Let's the compete with the GPU folks and offer something to the cloud players eventually (a la Tensor Processing Unit). Nice to see the competition. Also lets Nervana scale and potentially get their Neon deep learning framework out there in the face of bigger players (a la TensorFlow). All in all it's good to see the competition in this space.
- dharma1 10y ago
  Neon has been fast in benchmarks because of hand optimised kernels, but their kernel guy (Scott Gray) recently left to OpenAI. I wonder why
  - scott-gray 10y ago
    I was aware of this deal before leaving. I made out pretty well with my vested shares but was more interested in working on the cutting edge of research than in continuing on with pure hardware optimization and design. I wish Nervana/Intel the best and I'm looking forward to seeing their hardware come to market. I'm mainly working on TensorFlow now, but would love to see Nervana finish the graph backend they've been working on.
    - ericjang 10y ago
      Love your work. We welcome open source contributions to TensorFlow :)
cs702 10y ago
Finally it looks like Intel is getting serious about competing with Nvidia GPUs in the nascent deep/machine learning market! As good as this could be, it would be even better if we also get an open-source software stack that can compete with Nvidia's proprietary CUDA stack, which currently dominates everywhere (except maybe in Google's data centers).
- p1esk 10y ago
  it would be even better if we also get an open-source software stack that can compete with Nvidia's proprietary CUDA stack You've already got that. It's called OpenCL.
  - mastazi 10y ago
    Parent said "that can compete" so no, in practice, we don't have that. (Of course I hope the situation will improve!!) https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software https://en.wikipedia.org/wiki/Comparison_of_deep_learning_so... (see column "Open CL support")
    - p1esk 10y ago
      Oh, so you (or OP) want Intel/Nervana to create yet another programming model which will succeed where OpenCL fails (e.g. beat CUDA)? Seems unlikely...
  - gcr 10y ago
    OpenCL is not well supported by any major learning framework out there. Theano has had WIP support for years with little visible progress, Torch has an unofficial fork for it, and IIUC Caffe doesn't use OpenCL at all. Like it or not, Nvidia's currently cornered the market. With their outstanding work on CUDNN et al, they are milking that cow for all it's worth.
atemerev 10y ago
Their Neon platform is awesome. I hope it will not be abandoned. Congratulations!
modeless 10y ago
Very exciting that Intel sees a need to compete with NVIDIA here in a way that isn't just more x86. NVIDIA certainly needs the competition. Now can AMD get in on the action? They should be in a pretty good place to compete but so far they seem to have missed the boat, with approximately zero software support.
- flybirdx101 10y ago
  Unfortunately, I am afraid that AMD is to far behind Intel technologically in the moment. What is so powerful in this deal, is that Nervana will be able to leverage Intel's capabilities and expertise in chip making, and they are definitely a leader in this area.
lettergram 10y ago
Having done a technical review of what Nervana has to offer... Why Intel? My assumption, is that Intel will lease the chips + software, similar to the way HP, Dell, etc. Lease GPU machines. Nervana honestly woukd have been obsolete and dead pretty soon without this. I see this as an acuire-hire.
- byebyetech 10y ago
  >> an acquire-hire. For $350 million is not an acquire-hire.
  - confiscate 10y ago
    I thought so too. Until I saw the HN thread on the recent acquisition of Quip by Salesforce
etrautmann 10y ago
Congrats Naveen! Very nice work
zump 10y ago
I'm suffering from FOMO !
scottlegrand 10y ago
I figured Nervana was mostly dead if they were stuck at 28 nm I figured Intel was down and out in Santa Clara without a strong deep learning play That all changed today. Intel has been bouncing all over the place trying to break into deep learning. If they don't screw this up, they just found their way in IMO.
- nibnib 10y ago
  >I figured Nervana was mostly dead if they were stuck at 28 nm Why? Is there a competitor that has leveraged a more modern process technology?
  - scottlegrand 10y ago
    Yep, NVIDIA. Nervana's dedicated ASIC will deliver 55 (mostly) int16 TOps in 2017. In contrast, the two Titan XP GPUs I bought last week for a total of $2400 deliver 44 such TOps. Next year, a single Volta GPU will deliver at least 36 so I saw no way for them to win on their own with NVIDIA's GPU roadmap merrily marching along since 2007. However, getting access to Intel's fabs makes them a lot more interesting and competitive. It's not a slamdunk for Intel yet because they still have to incorporate this into their product line (anyone seen Altera's Stratix 10 yet? Because that was supposed to be 2014's 10+ TFLOP GPU killer), but it's a fantastic acquisition and I wish them the best.
mastazi 10y ago
@OP: the article's title has changed (to reflect the fact that the sum to be paid will be in excess of $400M)
rdlecler1 10y ago
Now compare this to Walmart'a $3B acquisition of Jet. This seems like a brilliant move and maybe a very good price given what it could do for Intel.
deleted 10y ago
[deleted]
jeffwass 10y ago
Addendum from the bottom of the article : Update: An earlier version of the story indicated the purchase price was more than $350 million, according to a source. Multiple investors told Recode the purchase price was significantly above that price, with one pegging it at $408 million.

143 points by flinner 10y ago

p1esk 10y ago

Here are some details about their upcoming chip: http://www.nextplatform.com/2016/08/08/deep-learning-chip-upstart-set-take-gpus-task/ http://www.nextplatform.com/2016/08/08/deep-learning-chip-up... Summary: - 28nm - looks similar to P100 (interposer with HBM) - 55 teraops/s performance - custom number format (variable length fixed point?) - simplified memory architecture (no cache?) - no info about power consumption

modeless 10y ago
Wow, really important details here; very interesting. Now imagine this chip, but shrunk to Intel's latest 14nm process. If they make that chip I have no doubt that it will revolutionize the entire field.
- PeCaN 10y ago
  > If they make that chip I have no doubt that it will revolutionize the entire field. Ha, ha. Yeah… no. For one thing, there's no mention of its power consumption, no CUDA support, questionable memory design (sure, you can get a million TFlops without cache, now try to get that chip to do anything useful), etc. Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something.
  - modeless 10y ago
    > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. Ha, ha. Yeah… no. > there's no mention of its power consumption Can't be tremendously higher than Pascal for reasons of physics. > sure, you can get a million TFlops without cache, now try to get that chip to do anything useful "Without cache" is certainly an exaggeration. It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. Neural net workloads are incredibly predictable and homogeneous and are essentially the perfect scenario for hand optimization of data flows to beat automatic caching. > no CUDA support You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway.
    - PeCaN 10y ago
      > > Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something. > Ha, ha. Yeah… no. Oh? You don't think they'd acquihire a machine learning startup to work on their compute/machine learning offering in Xeon Phi? I would be very, very surprised if Intel introduces a new machine-learning-oriented processor—though that would be quite interesting, so let's hope you're right. > Can't be tremendously higher than Pascal for reasons of physics. I wouldn't be so quick to assume that. Nvidia spends a lot of resources optimizing power efficiency. To think a much smaller company could match their power efficiency while scaling performace¹ is plausible, but not likely. If anything, physics would say that it can't be much lower than Pascal, but nothing about higher. For example, the large gap in power efficiency between Nvidia and AMD's 28nm GPUs last generation. It's not correct that similar size process nodes always have similar power consumption—that comes down a lot to architecture. > It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. You're right, the link² posted in another comment clears that up—and you're right that it could be very good for performance, but also very difficult to optimize for. > You're just being silly now. You're spouting stupid shit (like that bit about physics and power consumption… not to mention the ridiculous hyperbole in your original comment) and call me silly. I'm not sure if you're a troll, an overly zealous brogrammer buying into deep learning buzzwords, or retarded, but maybe you should cut it out. > CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway. Which is precisely why it won't revolutionize the field at all. Deep learning is relatively locked in to CUDA. Essentially, extraordinary claims (such as revolutionizing deep learning) require extraordinary evidence—evidence which neither you nor Nervana have shown the slightest signs of producing. -- ¹ I just noticed the nextplatform link specified teraops instead of teraflops. I don't know if that's referring to integer operations, would be… unusual, to say the least, for a deep learning chip. If it is, then we don't really know what its single-precision float performance is. ² https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
      - singhblom 10y ago
        They don't state flops because they don't do floats. The nervana chip has some weird fixed point format that they think works better for deep learning. I've heard similar noise out of Google (the other TPU company ...), so I wouldn't be surprised if we'll see the same in many of the specialized chips that people will build for deep learning. I really don't think CUDA support is important for nervana's offering. They think of themselves as the Apples of deep learning - they want to offer an integrated stack from the chip all the way to the APIs. The way most people use deep learning you don't really need to know CUDA, you just need to use a library that is fast. So it's enough that nervana's engineers know how to write deep learning libs for their own chip. Furthermore I can't see Intel caring that much about CUDA support, since CUDA is owned by Nvidia. What I've heard about the chip makes it sound really exciting. Many of the trade-offs in deep learning are different from the ones you do in graphics, so specialized hardware makes sense.
- argonaut 10y ago
  What PeCaN is driving at, is that saying something like you "have no doubt that it will revolutionize the entire field," is simply hyperbole that is not supported. We'll see. Nervana didn't really gain that much traction, which is why it's being acquired. Intel hasn't had much of a deep learning presence so far, which is why they are acquiring.
pjscott 10y ago
> simplified memory architecture (no cache?) No automatic cache hierarchy. There is local memory, which you manage explicitly. More info here: https://www.nervanasys.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
Katydid 10y ago
Most of the questions asked in these threads on HN are answered in an update article they posted this morning about 14 nanometer, the Neon framework's future and such. Also, looks like everyone who broke the news did got the acquisition amount wrong. http://www.nextplatform.com/2016/08/10/nervana-ceo-intel-acquisition-future-technology-outlook/ http://www.nextplatform.com/2016/08/10/nervana-ceo-intel-acq...

singularity2001 10y ago

@OP: that's Nervana with 'e' ;)

flinner 10y ago
Thank you! Updated

mgreg 10y ago

Likely a good move for Intel to get in on one of the faster growing areas of computing. Let's the compete with the GPU folks and offer something to the cloud players eventually (a la Tensor Processing Unit). Nice to see the competition. Also lets Nervana scale and potentially get their Neon deep learning framework out there in the face of bigger players (a la TensorFlow). All in all it's good to see the competition in this space.

dharma1 10y ago
Neon has been fast in benchmarks because of hand optimised kernels, but their kernel guy (Scott Gray) recently left to OpenAI. I wonder why
- scott-gray 10y ago
  I was aware of this deal before leaving. I made out pretty well with my vested shares but was more interested in working on the cutting edge of research than in continuing on with pure hardware optimization and design. I wish Nervana/Intel the best and I'm looking forward to seeing their hardware come to market. I'm mainly working on TensorFlow now, but would love to see Nervana finish the graph backend they've been working on.
  - ericjang 10y ago
    Love your work. We welcome open source contributions to TensorFlow :)

cs702 10y ago

Finally it looks like Intel is getting serious about competing with Nvidia GPUs in the nascent deep/machine learning market! As good as this could be, it would be even better if we also get an open-source software stack that can compete with Nvidia's proprietary CUDA stack, which currently dominates everywhere (except maybe in Google's data centers).

p1esk 10y ago
it would be even better if we also get an open-source software stack that can compete with Nvidia's proprietary CUDA stack You've already got that. It's called OpenCL.
- mastazi 10y ago
  Parent said "that can compete" so no, in practice, we don't have that. (Of course I hope the situation will improve!!) https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software https://en.wikipedia.org/wiki/Comparison_of_deep_learning_so... (see column "Open CL support")
  - p1esk 10y ago
    Oh, so you (or OP) want Intel/Nervana to create yet another programming model which will succeed where OpenCL fails (e.g. beat CUDA)? Seems unlikely...
- gcr 10y ago
  OpenCL is not well supported by any major learning framework out there. Theano has had WIP support for years with little visible progress, Torch has an unofficial fork for it, and IIUC Caffe doesn't use OpenCL at all. Like it or not, Nvidia's currently cornered the market. With their outstanding work on CUDNN et al, they are milking that cow for all it's worth.

atemerev 10y ago

Their Neon platform is awesome. I hope it will not be abandoned. Congratulations!

modeless 10y ago

Very exciting that Intel sees a need to compete with NVIDIA here in a way that isn't just more x86. NVIDIA certainly needs the competition. Now can AMD get in on the action? They should be in a pretty good place to compete but so far they seem to have missed the boat, with approximately zero software support.

flybirdx101 10y ago
Unfortunately, I am afraid that AMD is to far behind Intel technologically in the moment. What is so powerful in this deal, is that Nervana will be able to leverage Intel's capabilities and expertise in chip making, and they are definitely a leader in this area.

lettergram 10y ago

Having done a technical review of what Nervana has to offer... Why Intel? My assumption, is that Intel will lease the chips + software, similar to the way HP, Dell, etc. Lease GPU machines. Nervana honestly woukd have been obsolete and dead pretty soon without this. I see this as an acuire-hire.

byebyetech 10y ago
>> an acquire-hire. For $350 million is not an acquire-hire.
- confiscate 10y ago
  I thought so too. Until I saw the HN thread on the recent acquisition of Quip by Salesforce

etrautmann 10y ago

Congrats Naveen! Very nice work

zump 10y ago

I'm suffering from FOMO !

scottlegrand 10y ago

I figured Nervana was mostly dead if they were stuck at 28 nm I figured Intel was down and out in Santa Clara without a strong deep learning play That all changed today. Intel has been bouncing all over the place trying to break into deep learning. If they don't screw this up, they just found their way in IMO.

nibnib 10y ago
>I figured Nervana was mostly dead if they were stuck at 28 nm Why? Is there a competitor that has leveraged a more modern process technology?
- scottlegrand 10y ago
  Yep, NVIDIA. Nervana's dedicated ASIC will deliver 55 (mostly) int16 TOps in 2017. In contrast, the two Titan XP GPUs I bought last week for a total of $2400 deliver 44 such TOps. Next year, a single Volta GPU will deliver at least 36 so I saw no way for them to win on their own with NVIDIA's GPU roadmap merrily marching along since 2007. However, getting access to Intel's fabs makes them a lot more interesting and competitive. It's not a slamdunk for Intel yet because they still have to incorporate this into their product line (anyone seen Altera's Stratix 10 yet? Because that was supposed to be 2014's 10+ TFLOP GPU killer), but it's a fantastic acquisition and I wish them the best.

mastazi 10y ago

@OP: the article's title has changed (to reflect the fact that the sum to be paid will be in excess of $400M)

rdlecler1 10y ago

Now compare this to Walmart'a $3B acquisition of Jet. This seems like a brilliant move and maybe a very good price given what it could do for Intel.

deleted 10y ago

[deleted]

jeffwass 10y ago

Addendum from the bottom of the article : Update: An earlier version of the story indicated the purchase price was more than $350 million, according to a source. Multiple investors told Recode the purchase price was significantly above that price, with one pegging it at $408 million.