5 ms·

Sure, but the really lucky break IMO was a decade of INTC repeatedly punching itself in the face w/r to manycore computation. I was at Nvidia from 2001 to 2011

Sure, but the really lucky break IMO was a decade of INTC repeatedly punching itself in the face w/r to manycore computation. I was at Nvidia from 2001 to 2011 (and now back) and I spent 2006-2011 as one of the very first CUDA programmers. It was pretty obvious within a week or two that this technology was going to be huge, and I more or less made my career with it. But instead of stepping up to the plate and igniting a Red Queen's Race that would have benefited everyone, INTC first tried to discredit the technology repeatedly, then they built an absolutely dreadful series of decelerators that demonstrated how badly they didn't understand manycore. Eventually, they gave up, and now they're playing catch-up by buying companies that get within striking distance of NVDA rather than building really cool technology from within. Now if someone threw a large pile of money at AMD again, things could get really interesting IMO. But the piles of stupid money seem biased towards throwing ~$5M per layer at the pets.com of AI companies these days.

doh 7y ago
> But instead of stepping up to the plate and igniting a Red Queen's Race that would have benefited everyone, INTC first tried to discredit the technology repeatedly, then they built an absolutely dreadful series of decelerators that demonstrated how badly they didn't understand manycore. Eventually, they gave up, and now they're playing catch-up by buying companies that get within striking distance of NVDA rather than building really cool technology from within. Reminds me of "First they ignore you, then they laugh at you, then they fight you, then you win". So many companies have this reaction (e.g. RIM with BlackBerry). Wondering if this is some kind of "corporate instinct".
- marcosdumay 7y ago
  This is "The innovator's dilemma" material. TL.DR. yes, it's a perfectly rational (but short term biased) reaction that corporations mostly can not resist having.
  - doh 7y ago
    I was just thinking about it as I wrote it. I think that's correct and glad you brought it up.
  - asdf21 7y ago
    The thing is, it works 98% of the time, the scrappy upstart gets laughed out of business. There is a sort of survivors bias in focusing in the 2% and assuming that's the norm. Corporations act this way because it generally works. Like the OP said though, it does lead to arrogance over time, and that's when a fall happens.
    - icelancer 7y ago
      But all it takes is a 1/50 shot to sink your business or overlook a key feature. That's why it's a real thing, and a 98% success rate is not very good at all.
    - close04 7y ago
      A corporation (the management, that is) also act this way because they know their maneuverability is just about the same as that of the Titanic. So while they (maybe) prepare a response knowing full well they're already late too the party they try to discredit the startup hoping it will at the very least slow it down. Corporations can't really be as successful at innovating as a startup. A startup is free to build or reshape itself into anything, focus on a single thing, and pivot on a dime. In a corporation the same structures that hold it up and moving are the ones that resist it changing direction or promoting something new. Easy to lose focus and get lost in the red tape. And that's before you consider the risk a CEO sees in potentially cannibalizing their own (currently successful) business or just throwing money down the drain at 98 losing ideas. Like you said, 2% of ideas may be successful so a corporation would rather let the startup play it out and then buy it if it has potential. Easier to justify to investors. So corporations innovate when they have nothing to lose and any risk is worth taking. See MS trying to somewhat successfully reinventing themselves after seeing mobile and FOSS booming. Private companies also have an easier time innovating because they have no investor pressure. They may be behemoths but at least they can avoid suffering from the "too many cooks in the kitchen" syndrome.
jcranmer 7y ago
Nvidia's coup was in getting people to switch to a different programming model and rewrite their code to achieve the necessary performance. Intel, especially in upper management, is stuffed with people who assumed that was impossible. And faced with new competition from GPUs, the only acceptable response to management was the many-x86, "you don't have to rewrite your code to get performance" approach which didn't actually work out. It's not that Intel doesn't have people that recognize the issues, but rather the people who do have that foresight are drowned out by people who don't realize the game has changed. Intel, to be fair, does have the best autovectorizer--but designing vector code from scratch in a purpose-built vector language is still going to produce better results, as shown when ispc beat the vectorizer. But Nvidia can also get drunk on its kool-aid, just as Intel has been. Nvidia's marketing would have you believe that switching to GPUs magically makes you gain performance, but if your code isn't really amenable to a vector programming style, then GPUs aren't going to speed your code up, and the shift from CPU-based supercomputers to GPU-based supercomputers are going to leave you happy. There's still room for third-way architectures that is anyone's game.
- mjevans 7y ago
  Intel also got burned by Itanium; though maybe if it had been a many-core design as well the payoff would have been worth it. (Looking (about a generation later, Itanium first released in 2001 and cell development started then) at the Cell processor the PS3 used, the idea was probably around at the time and didn't seem to pay off very well their either...) Arguably one of the things that Nvidia really got right was learning from those past failures at other companies and making it easier for developers to utilize the platform starting from a standpoint that they were familiar with and helpfully nudging them towards what would run fast in parallel.
- varelse 7y ago
  If you don't have a fundamentally serial workload (and usually either you don't or you have a lot of them you can parallelize across tasks) and you are willing to write bespoke CUDA code for that workload, Nvidia is telling the truth. CUDA's sweet spot lies between embarrassingly parallel (for which ASICs and FPGAs rule the world because these are generally pure compute with low memory bandwidth overhead) and serial (for which CPUs are still best), a place I call "annoyingly parallel." There are a lot of workloads in this space in my experience. But if you don't satisfy both of the aforementioned requirements and/or you insist on doing this all from someone else's code interfaced through a weak-typed garbage collected global interpreter locked language, your mileage will vary greatly cough deep learning frameworks cough. Finally, it doesn't matter who's doing it, marchitecturing(tm) drives me nuts too.
bryanlarsen 7y ago
> Now if someone threw a large pile of money at AMD again, things could get really interesting IMO. That somebody might be Nvidia. I believe that Nvidia is still battling and has not yet paid the 1.06 billion Euro fine to AMD imposed on it by the UK courts in 2009. Hearsay claims that it was the similar US fine that basically paid for Zen R&D...
jacobolus 7y ago
Is there a reason for referring to Intel as INTC and Nvidia as NVDA? Is this just the internal Nvidia jargon or something? It seems gratuitously confusing for readers, and doesn’t seem to have any benefit I can see.
- bradleyjg 7y ago
  > Is this just the internal Nvidia jargon or something? Stock symbols
  - jacobolus 7y ago
    Yes it’s obvious that these cryptic 4-letter abbreviations are ticker symbols, but my point is that most people refer to companies by their names, not their ticker symbols. I’m wondering if using the latter is something common internal to Nvidia, or if there’s some other explanation.
    - finebalance 7y ago
      Possibly a hangover from when sending data across the wire was expensive and being able to uniquely identify companies with just a few letters was a huge cost-saving innovation.
    - bradleyjg 7y ago
      In my experience it is common when talking with people that invest in individual stocks--either personally or professionally.
- antisemiotic 7y ago
  I always wonder what people who feel it worthwhile to shave off 2 letters from a word do with all the free time they gain.
person_of_color 7y ago
Are you hiring?
folloup 7y ago
Do you mind putting an email in your profile (or by reply)? I'd like to get your opinion on something but would rather not ask here.
megablast 7y ago
Why would you say INTC and not Intel? Are you deliberately trying to be confusing?

doh 7y ago

> But instead of stepping up to the plate and igniting a Red Queen's Race that would have benefited everyone, INTC first tried to discredit the technology repeatedly, then they built an absolutely dreadful series of decelerators that demonstrated how badly they didn't understand manycore. Eventually, they gave up, and now they're playing catch-up by buying companies that get within striking distance of NVDA rather than building really cool technology from within. Reminds me of "First they ignore you, then they laugh at you, then they fight you, then you win". So many companies have this reaction (e.g. RIM with BlackBerry). Wondering if this is some kind of "corporate instinct".

marcosdumay 7y ago
This is "The innovator's dilemma" material. TL.DR. yes, it's a perfectly rational (but short term biased) reaction that corporations mostly can not resist having.
- doh 7y ago
  I was just thinking about it as I wrote it. I think that's correct and glad you brought it up.
- asdf21 7y ago
  The thing is, it works 98% of the time, the scrappy upstart gets laughed out of business. There is a sort of survivors bias in focusing in the 2% and assuming that's the norm. Corporations act this way because it generally works. Like the OP said though, it does lead to arrogance over time, and that's when a fall happens.
  - icelancer 7y ago
    But all it takes is a 1/50 shot to sink your business or overlook a key feature. That's why it's a real thing, and a 98% success rate is not very good at all.
  - close04 7y ago
    A corporation (the management, that is) also act this way because they know their maneuverability is just about the same as that of the Titanic. So while they (maybe) prepare a response knowing full well they're already late too the party they try to discredit the startup hoping it will at the very least slow it down. Corporations can't really be as successful at innovating as a startup. A startup is free to build or reshape itself into anything, focus on a single thing, and pivot on a dime. In a corporation the same structures that hold it up and moving are the ones that resist it changing direction or promoting something new. Easy to lose focus and get lost in the red tape. And that's before you consider the risk a CEO sees in potentially cannibalizing their own (currently successful) business or just throwing money down the drain at 98 losing ideas. Like you said, 2% of ideas may be successful so a corporation would rather let the startup play it out and then buy it if it has potential. Easier to justify to investors. So corporations innovate when they have nothing to lose and any risk is worth taking. See MS trying to somewhat successfully reinventing themselves after seeing mobile and FOSS booming. Private companies also have an easier time innovating because they have no investor pressure. They may be behemoths but at least they can avoid suffering from the "too many cooks in the kitchen" syndrome.

jcranmer 7y ago

Nvidia's coup was in getting people to switch to a different programming model and rewrite their code to achieve the necessary performance. Intel, especially in upper management, is stuffed with people who assumed that was impossible. And faced with new competition from GPUs, the only acceptable response to management was the many-x86, "you don't have to rewrite your code to get performance" approach which didn't actually work out. It's not that Intel doesn't have people that recognize the issues, but rather the people who do have that foresight are drowned out by people who don't realize the game has changed. Intel, to be fair, does have the best autovectorizer--but designing vector code from scratch in a purpose-built vector language is still going to produce better results, as shown when ispc beat the vectorizer. But Nvidia can also get drunk on its kool-aid, just as Intel has been. Nvidia's marketing would have you believe that switching to GPUs magically makes you gain performance, but if your code isn't really amenable to a vector programming style, then GPUs aren't going to speed your code up, and the shift from CPU-based supercomputers to GPU-based supercomputers are going to leave you happy. There's still room for third-way architectures that is anyone's game.

mjevans 7y ago
Intel also got burned by Itanium; though maybe if it had been a many-core design as well the payoff would have been worth it. (Looking (about a generation later, Itanium first released in 2001 and cell development started then) at the Cell processor the PS3 used, the idea was probably around at the time and didn't seem to pay off very well their either...) Arguably one of the things that Nvidia really got right was learning from those past failures at other companies and making it easier for developers to utilize the platform starting from a standpoint that they were familiar with and helpfully nudging them towards what would run fast in parallel.
varelse 7y ago
If you don't have a fundamentally serial workload (and usually either you don't or you have a lot of them you can parallelize across tasks) and you are willing to write bespoke CUDA code for that workload, Nvidia is telling the truth. CUDA's sweet spot lies between embarrassingly parallel (for which ASICs and FPGAs rule the world because these are generally pure compute with low memory bandwidth overhead) and serial (for which CPUs are still best), a place I call "annoyingly parallel." There are a lot of workloads in this space in my experience. But if you don't satisfy both of the aforementioned requirements and/or you insist on doing this all from someone else's code interfaced through a weak-typed garbage collected global interpreter locked language, your mileage will vary greatly cough deep learning frameworks cough. Finally, it doesn't matter who's doing it, marchitecturing(tm) drives me nuts too.

bryanlarsen 7y ago

> Now if someone threw a large pile of money at AMD again, things could get really interesting IMO. That somebody might be Nvidia. I believe that Nvidia is still battling and has not yet paid the 1.06 billion Euro fine to AMD imposed on it by the UK courts in 2009. Hearsay claims that it was the similar US fine that basically paid for Zen R&D...

jacobolus 7y ago

Is there a reason for referring to Intel as INTC and Nvidia as NVDA? Is this just the internal Nvidia jargon or something? It seems gratuitously confusing for readers, and doesn’t seem to have any benefit I can see.

bradleyjg 7y ago
> Is this just the internal Nvidia jargon or something? Stock symbols
- jacobolus 7y ago
  Yes it’s obvious that these cryptic 4-letter abbreviations are ticker symbols, but my point is that most people refer to companies by their names, not their ticker symbols. I’m wondering if using the latter is something common internal to Nvidia, or if there’s some other explanation.
  - finebalance 7y ago
    Possibly a hangover from when sending data across the wire was expensive and being able to uniquely identify companies with just a few letters was a huge cost-saving innovation.
  - bradleyjg 7y ago
    In my experience it is common when talking with people that invest in individual stocks--either personally or professionally.
antisemiotic 7y ago
I always wonder what people who feel it worthwhile to shave off 2 letters from a word do with all the free time they gain.

person_of_color 7y ago

Are you hiring?

folloup 7y ago

Do you mind putting an email in your profile (or by reply)? I'd like to get your opinion on something but would rather not ask here.

megablast 7y ago

Why would you say INTC and not Intel? Are you deliberately trying to be confusing?