10 ms·

This is awful. Out of all the big tech companies, Nvidia is probably least friendly to open source and cross-platform comparability. It seems to me that their g

This is awful. Out of all the big tech companies, Nvidia is probably least friendly to open source and cross-platform comparability. It seems to me that their goal is to monopolize AI hardware over the next 20 years, the same way Intel effectively monopolized cloud hardware over the last 20. Expect to see less choice in the chip market and more and more propietary software frameworks like CUDA. A sad day for CS and for AI.

sillysaurusx 6y ago
AI training is moving away from CUDA and toward TPUs anyway. DGX clusters can't keep up.
- ladberg 6y ago
  And Nvidia's GPUs now include the same type of hardware that TPUs have, so there's no reason to believe that TPUs will win out over GPUs.
  - sillysaurusx 6y ago
    The key difference between a TPU and a GPU is that a TPU has a CPU. It's an entire computer, not just a piece of hardware. Is nVidia moving in that direction?
    - newsclues 6y ago
      They just bought ARM for 40$billion. I think they want to integrate CPU,GPU and high speed networks
    - dikei 6y ago
      In term of cutting edge tech, They have their own GPUs, CPUs from ARM, Networking from Mellanox, so I'd say they're pretty much set to build a kick ass TPU.
    - shaklee3 6y ago
      A TPU is a chip you cannot program. It's purpose built and can't run the fraction of the type of workloads that a GPU can.
      - sillysaurusx 6y ago
        I don't know where all of this misinformation is coming from or why, but, as someone who has spent the last year programming TPUs to do all kinds of things that a GPU can't do, this isn't true. Are we going to simply say "Nu uh" at each other, or do you want to throw down some specific examples so I can show you how mistaken they are?
        shaklee3 6y ago
        Please show me the API where I can write a generic function on a TPU. I'm talking about writing something like a custom reduction or a peak search, not offloading a tensor flow model. I'll make it easier for you, directly from Google's website: TPUs Cloud TPUs are optimized for specific workloads. In some situations, you might want to use GPUs or CPUs on Compute Engine instances to run your machine learning workloads. Please tell me a workload a gpu can't do that a TPU can.
        sillysaurusx 6y ago
        Sure, here you go: https://www.tensorflow.org/api_docs/python/tf/raw_ops https://www.tensorflow.org/api_docs/python/tf/raw_ops In my experience, well over 80% of these operations are implemented on TPU CPUs, and at least 60% are implemented on TPU cores. Again, if you give a specific example, I can simply write a program demonstrating that it works. What kind of custom reduction do you want? What's a peak search? As for workloads that GPUs can't do, we regularly train GANs at 500+ examples/sec across a total dataset size of >3M photos. Rather hard to do that with GPUs.
        shaklee3 6y ago
        Well, there you go. For one TensorFlow is not a generic framework like cuda is, so you lose a whole bunch of the configurability you have with cuda. So, for example, even though there is an FFT raw function, there doesn't appear to be a way to do more complicated FFTs, such as an overlap-save. This is trivial to do on a GPU, and is built into the library. The raw functions it provides is not direct access to the hardware and memory subsystem. It's a set of raw functions that is a small subset of the total problem space. And certainly if you are saying that running something on a TPU's CPU cores are in any way going to compete with a gpu, then I don't know what to tell you. You did not give an example of something GPUs can't do. all you said was that TPUs are faster for a specific function in your case.
- lostmsu 6y ago
  Where did you get this from? AFAIK GPT-3 (for example) was trained on a GPU cluster, not TPUs.
  - sillysaurusx 6y ago
    Experience, for one. TPUs are dominating MLPerf benchmarks. That kind of performance can't be dismissed so easily. GPT-2 was trained on TPUs. (There are explicit references to TPUs in the source code: https://github.com/openai/gpt-2/blob/0574c5708b094bfa0b0f6dfe3fd284d9a045acd9/src/model.py#L61 https://github.com/openai/gpt-2/blob/0574c5708b094bfa0b0f6df...) GPT-3 was trained on a GPU cluster probably because of Microsoft's billion-dollar Azure cloud credit investment, not because it was the best choice.
    - option 6y ago
      no they are not. Go read recent MLPerf results more carefully and not Google’s blogpost. NVIDIA won 8/8 benchmarks for publicly available SW/HW combo. Also 8/8 on per chip performance. Google did show better results with some “research” system which is not available to anyone other then them yet.
      - sillysaurusx 6y ago
        This is a weirdly aggressive reply. I don't "read Google's blogpost," I use TPUs daily. As for MLPerf benchmarks, you can see for yourself here: https://mlperf.org/training-results-0-6 https://mlperf.org/training-results-0-6 TPUs are far ahead of competitors. All of these training results are openly available, and you can run them yourself. (I did.) For MLPerf 0.7, it's true that Google's software isn't available to the public yet. That's because they're in the middle of transitioning to Jax (and by extension, Pytorch). Once that transition is complete, and available to the public, you'll probably be learning TPU programming one way or another, since there's no other practical way to e.g. train a GAN on millions of photos. You'd think people would be happy that there are realistic alternatives to nVidia's monopoly for AI training, rather than rushing to defend them...
- make3 6y ago
  this is just false
therealmarv 6y ago
The problem is more that AMD is sleeping in regard of GPU AI and good software interfaces.
- burnte 6y ago
  Hardly sleeping, they have a fraction of the revenue and profit of NVidia and Intel. Revenue was half of Nvidia, profit was 5% of NVidia. Intel is even bigger. They only have so much in the way of R&D money.
  - reader_mode 6y ago
    Seeing their market performance I have no doubt they could get capital for R&D
    - zrm 6y ago
      Their market performance is a relatively recent development. R&D has a lead time.
  - Polylactic_acid 6y ago
    ROCm is their product for compute and it still can't run on their latest NAVI cards which have been out for over a year while CUDA works on every nvidia card on day one.
  - adventured 6y ago
    > Revenue was half of Nvidia, profit was 5% of NVidia. For operating income it's 25%, for net income it's 18%; not 5%. Last four quarters operating income for AMD: $884 million Last four quarters operating income for Nvidia: $3.5 billion This speaks to the dramatic improvement in AMD's operating condition over the last several years. For contrast, in fiscal 2016 AMD's operating income was negative $382 million. Op income has increased by over 300% in just ~2 1/2 years. Increasingly AMD is no longer a profit lightweight.
    - burnte 6y ago
QuixoticQuibit 6y ago
NVIDIA’s hardware works on x86, PowerPC, and ARM platforms. Many of their AI libraries/tools are in fact open source. They stand to be a force that could propel ARM’s strength in data center and desktop computing. For some reason you’re okay with the current x86 duopoly held by AMD and Intel, both who have their own destiny over CPUs and GPUs. The HN crowd is incredibly biased against certain companies. Why not look at some of the potential bright sides to this for a more nuanced and balanced opinion?
- ianai 6y ago
  Yes I wonder the same thing about the sentiment against nvidia. It’s be helpful if there were some wiki about things they’ve killed or instances they’ve acted against foss systems.
  - mixmastamyk 6y ago
    Linus famously gave them the middle-finger.
    - paulmd 6y ago
      "linus went on a hyperbolic rant" isn't sufficient evidence for anything. linus is a hyperbolic jerk (as he admitted himself for a few months before resuming his hyperbolic ways) who is increasingly out of touch with anything outside the direct sphere of his projects. Like his misguided and completely unnecessary rants about ZFS or AVX. if there are technical merits to discuss you can post those instead of just appealing to linus' hyperbole. (I won't even say "appeal to authority" because that's not what you're appealing to. You're literally appealing to his middle finger.)
      - andrewprock 6y ago
        The notion that the maintainer if Linux has a narrow focus role that isn't capable of advocating and serving the broader community is belied by the fact that Linux has grown from being a niche hobbyist platform to the de facto cloud standard OS.
m3kw9 6y ago
Lol with the italics awful and your “seems to me” angle
- starpilot 6y ago
  Hyperbole is how you get upvotes. No one really reacts to wishy-washy answers, it's the "OMG world is ending" provocative stuff that gets people engaged.
tony 6y ago
Surprisingly - they have a working driver for FreeBSD. Never had an issue with it - and the performance is fantastic. As far back as early 2000's I remember installing proprietary nvidia drivers on Linux and playing UT2004. Maybe Nintendo/Sony uses Nvidia cards on their developer machines? I imagine FreeBSD drivers aren't simply altruism on their part. On the other hand, stagnation on other fronts: - Nouveau (tried recently) is basically unusable on Ubuntu. As in the mouse/keyboard locks every 6 seconds. - Proprietary drivers won't work with wayland And since their stuff isn't open, the community can't do much to push Nouveau forward.
- gautamcgoel 6y ago
  Is the FreeBDD driver open source? Also, somehow I was under the impression they stopped maintaining the FreeBSD driver. Is that correct?
  - Conan_Kudo 6y ago
    They did. The proprietary NVIDIA driver is required on FreeBSD as well. The FreeBSD community just doesn't care about this.
  - magic_quotes 6y ago
    > impression they stopped maintaining the FreeBSD driver. It's maintained while simultaneously not receiving any new features.
    - throwaway2048 6y ago
      Note this includes support for newer cards, so its only a matter of time before FreeBSD support is EOL'd
      - magic_quotes 6y ago
        New cards are fully supported. Why do you think they aren't?
      - 6y ago
twblalock 6y ago
On the other hand, if Nvidia wants ARM to succeed (and why else would they acquire it?), they can be a source of more competition in the CPU market. I don't really see how this deal makes the CPU market worse -- wasn't the ARM market for mobile devices basically dominated by Qualcomm for years? Plus, the other existing ARM licensees don't seem to be impacted. On the other hand, I do see a lot of potential if Nvidia is serious about innovating in the CPU space.

sillysaurusx 6y ago

AI training is moving away from CUDA and toward TPUs anyway. DGX clusters can't keep up.

ladberg 6y ago
And Nvidia's GPUs now include the same type of hardware that TPUs have, so there's no reason to believe that TPUs will win out over GPUs.
- sillysaurusx 6y ago
  The key difference between a TPU and a GPU is that a TPU has a CPU. It's an entire computer, not just a piece of hardware. Is nVidia moving in that direction?
  - newsclues 6y ago
    They just bought ARM for 40$billion. I think they want to integrate CPU,GPU and high speed networks
  - dikei 6y ago
    In term of cutting edge tech, They have their own GPUs, CPUs from ARM, Networking from Mellanox, so I'd say they're pretty much set to build a kick ass TPU.
  - shaklee3 6y ago
    A TPU is a chip you cannot program. It's purpose built and can't run the fraction of the type of workloads that a GPU can.
    - sillysaurusx 6y ago
      I don't know where all of this misinformation is coming from or why, but, as someone who has spent the last year programming TPUs to do all kinds of things that a GPU can't do, this isn't true. Are we going to simply say "Nu uh" at each other, or do you want to throw down some specific examples so I can show you how mistaken they are?
      - shaklee3 6y ago
        Please show me the API where I can write a generic function on a TPU. I'm talking about writing something like a custom reduction or a peak search, not offloading a tensor flow model. I'll make it easier for you, directly from Google's website: TPUs Cloud TPUs are optimized for specific workloads. In some situations, you might want to use GPUs or CPUs on Compute Engine instances to run your machine learning workloads. Please tell me a workload a gpu can't do that a TPU can.
        sillysaurusx 6y ago
        Sure, here you go: https://www.tensorflow.org/api_docs/python/tf/raw_ops https://www.tensorflow.org/api_docs/python/tf/raw_ops In my experience, well over 80% of these operations are implemented on TPU CPUs, and at least 60% are implemented on TPU cores. Again, if you give a specific example, I can simply write a program demonstrating that it works. What kind of custom reduction do you want? What's a peak search? As for workloads that GPUs can't do, we regularly train GANs at 500+ examples/sec across a total dataset size of >3M photos. Rather hard to do that with GPUs.
        shaklee3 6y ago
        Well, there you go. For one TensorFlow is not a generic framework like cuda is, so you lose a whole bunch of the configurability you have with cuda. So, for example, even though there is an FFT raw function, there doesn't appear to be a way to do more complicated FFTs, such as an overlap-save. This is trivial to do on a GPU, and is built into the library. The raw functions it provides is not direct access to the hardware and memory subsystem. It's a set of raw functions that is a small subset of the total problem space. And certainly if you are saying that running something on a TPU's CPU cores are in any way going to compete with a gpu, then I don't know what to tell you. You did not give an example of something GPUs can't do. all you said was that TPUs are faster for a specific function in your case.
lostmsu 6y ago
Where did you get this from? AFAIK GPT-3 (for example) was trained on a GPU cluster, not TPUs.
- sillysaurusx 6y ago
  Experience, for one. TPUs are dominating MLPerf benchmarks. That kind of performance can't be dismissed so easily. GPT-2 was trained on TPUs. (There are explicit references to TPUs in the source code: https://github.com/openai/gpt-2/blob/0574c5708b094bfa0b0f6dfe3fd284d9a045acd9/src/model.py#L61 https://github.com/openai/gpt-2/blob/0574c5708b094bfa0b0f6df...) GPT-3 was trained on a GPU cluster probably because of Microsoft's billion-dollar Azure cloud credit investment, not because it was the best choice.
  - option 6y ago
    no they are not. Go read recent MLPerf results more carefully and not Google’s blogpost. NVIDIA won 8/8 benchmarks for publicly available SW/HW combo. Also 8/8 on per chip performance. Google did show better results with some “research” system which is not available to anyone other then them yet.
    - sillysaurusx 6y ago
      This is a weirdly aggressive reply. I don't "read Google's blogpost," I use TPUs daily. As for MLPerf benchmarks, you can see for yourself here: https://mlperf.org/training-results-0-6 https://mlperf.org/training-results-0-6 TPUs are far ahead of competitors. All of these training results are openly available, and you can run them yourself. (I did.) For MLPerf 0.7, it's true that Google's software isn't available to the public yet. That's because they're in the middle of transitioning to Jax (and by extension, Pytorch). Once that transition is complete, and available to the public, you'll probably be learning TPU programming one way or another, since there's no other practical way to e.g. train a GAN on millions of photos. You'd think people would be happy that there are realistic alternatives to nVidia's monopoly for AI training, rather than rushing to defend them...
make3 6y ago
this is just false

therealmarv 6y ago

The problem is more that AMD is sleeping in regard of GPU AI and good software interfaces.

burnte 6y ago
Hardly sleeping, they have a fraction of the revenue and profit of NVidia and Intel. Revenue was half of Nvidia, profit was 5% of NVidia. Intel is even bigger. They only have so much in the way of R&D money.
- reader_mode 6y ago
  Seeing their market performance I have no doubt they could get capital for R&D
  - zrm 6y ago
    Their market performance is a relatively recent development. R&D has a lead time.
- Polylactic_acid 6y ago
  ROCm is their product for compute and it still can't run on their latest NAVI cards which have been out for over a year while CUDA works on every nvidia card on day one.
- adventured 6y ago
  > Revenue was half of Nvidia, profit was 5% of NVidia. For operating income it's 25%, for net income it's 18%; not 5%. Last four quarters operating income for AMD: $884 million Last four quarters operating income for Nvidia: $3.5 billion This speaks to the dramatic improvement in AMD's operating condition over the last several years. For contrast, in fiscal 2016 AMD's operating income was negative $382 million. Op income has increased by over 300% in just ~2 1/2 years. Increasingly AMD is no longer a profit lightweight.
  - burnte 6y ago

QuixoticQuibit 6y ago

NVIDIA’s hardware works on x86, PowerPC, and ARM platforms. Many of their AI libraries/tools are in fact open source. They stand to be a force that could propel ARM’s strength in data center and desktop computing. For some reason you’re okay with the current x86 duopoly held by AMD and Intel, both who have their own destiny over CPUs and GPUs. The HN crowd is incredibly biased against certain companies. Why not look at some of the potential bright sides to this for a more nuanced and balanced opinion?

ianai 6y ago
Yes I wonder the same thing about the sentiment against nvidia. It’s be helpful if there were some wiki about things they’ve killed or instances they’ve acted against foss systems.
- mixmastamyk 6y ago
  Linus famously gave them the middle-finger.
  - paulmd 6y ago
    "linus went on a hyperbolic rant" isn't sufficient evidence for anything. linus is a hyperbolic jerk (as he admitted himself for a few months before resuming his hyperbolic ways) who is increasingly out of touch with anything outside the direct sphere of his projects. Like his misguided and completely unnecessary rants about ZFS or AVX. if there are technical merits to discuss you can post those instead of just appealing to linus' hyperbole. (I won't even say "appeal to authority" because that's not what you're appealing to. You're literally appealing to his middle finger.)
    - andrewprock 6y ago
      The notion that the maintainer if Linux has a narrow focus role that isn't capable of advocating and serving the broader community is belied by the fact that Linux has grown from being a niche hobbyist platform to the de facto cloud standard OS.

m3kw9 6y ago

Lol with the italics awful and your “seems to me” angle

starpilot 6y ago
Hyperbole is how you get upvotes. No one really reacts to wishy-washy answers, it's the "OMG world is ending" provocative stuff that gets people engaged.

tony 6y ago

Surprisingly - they have a working driver for FreeBSD. Never had an issue with it - and the performance is fantastic. As far back as early 2000's I remember installing proprietary nvidia drivers on Linux and playing UT2004. Maybe Nintendo/Sony uses Nvidia cards on their developer machines? I imagine FreeBSD drivers aren't simply altruism on their part. On the other hand, stagnation on other fronts: - Nouveau (tried recently) is basically unusable on Ubuntu. As in the mouse/keyboard locks every 6 seconds. - Proprietary drivers won't work with wayland And since their stuff isn't open, the community can't do much to push Nouveau forward.

gautamcgoel 6y ago
Is the FreeBDD driver open source? Also, somehow I was under the impression they stopped maintaining the FreeBSD driver. Is that correct?
- Conan_Kudo 6y ago
  They did. The proprietary NVIDIA driver is required on FreeBSD as well. The FreeBSD community just doesn't care about this.
- magic_quotes 6y ago
  > impression they stopped maintaining the FreeBSD driver. It's maintained while simultaneously not receiving any new features.
  - throwaway2048 6y ago
    Note this includes support for newer cards, so its only a matter of time before FreeBSD support is EOL'd
    - magic_quotes 6y ago
      New cards are fully supported. Why do you think they aren't?
    - 6y ago

twblalock 6y ago

On the other hand, if Nvidia wants ARM to succeed (and why else would they acquire it?), they can be a source of more competition in the CPU market. I don't really see how this deal makes the CPU market worse -- wasn't the ARM market for mobile devices basically dominated by Qualcomm for years? Plus, the other existing ARM licensees don't seem to be impacted. On the other hand, I do see a lot of potential if Nvidia is serious about innovating in the CPU space.