3 ms·

> I'm dying to hear what you think is more efficient for floating-point computation yup, flops are the only thing that matter. Nothing else makes a difference.

> I'm dying to hear what you think is more efficient for floating-point computation yup, flops are the only thing that matter. Nothing else makes a difference. 700watts a unit, plus the ancillary heat from the processor and infiniband/fancy ethernet is challenging to house at scale. as to my original point, they are fucking expensive. Some of the ones we have are literally gold plated.

talldayo 2y ago
What is your use-case where you need a GB200 but can't house it? These are datacenter processors; they consume a lot of power, output a lot of heat, and do an impressive amount of work. The same problem would arise with an Instinct or Gaudi accelerator. The real "only thing" that matters is CUDA. CUDA scales from Raspberry Pi boards to desktop cards to datacenters without skipping a beat. It's a killer product in an age where OpenCL is rotting and dead.
- KaiserPro 2y ago
  > where you need a GB200 but can't house it? density. Datacentre space is expensive. 9.6kw per machine means putting liquid cooling to the die, if we want to have more than 3 machines per rack. We've already lost a load of space in order to house an unspecified thousands of "h100" We need to have some space left to put in storage, otherwise we're going to wasting power for funsies. We need that storage to be close, because otherwise it gets _really_ expensive and really slow quickly.
  - talldayo 2y ago
    I don't know what to tell you. Jetson Orin exists, GPU cards in all form factors support CUDA, and the servers represent the apex of that power. There are options for almost every form factor and power profile you can imagine; the goddamn Nintendo Switch supports CUDA running at 7w. Literally, what are your other options? Hot aisles get hot, I don't know what to tell you there.
    - KaiserPro 2y ago
      Yes, Orins exist, and they are frankly awful to develop on. Sure I could spend a boatload of time porting my stuff to an outdated version of ubuntu, with no practical HSM support, or I could not. Mind you, its no worse than dealing with our internal shit. You are misunderstanding what I originally saying: there is an opportunity for other people to bring in new types of accelerators. You took that to mean that I was saying nvidia are shit. GPUs TPUs and other accelerators are a pain in the arse to integrate at scale. NVIDIA have the current winner because CUDA is well understood, and the GPUs they make are fast and plentiful for the desktop. They also make a loverly goldplated "mini computer" in the form of the DGX. Again great for any startup that's just got funding. But, just like IBM, DEC & HP, who also made really fast, capable, large scalable machines, they were eaten by slower, cheaper pizzabox clusters. Not because they were better, or more efficient, but because they were cheap, plentyful and easy to sling in a room.
      - talldayo 2y ago
        Orins are fine. If your use-case is robotics and edge compute, your alternative is exclusively worse ARM boards or x86 boards. Strictly speaking, it is the "pizzabox computer" disrupting everyone else's acceleration racket. And my point isn't that I resent people insulting Nvidia; I think the world would be a better place if CUDA didn't exist. But there literally are not HPC alternatives that exist in the mainstream today. Seriously, what are you suggesting? 20,000w Beowulf clusters that accelerate everything on 10-year-old Xeons? A few thousand Macs running GPU-shader accelerated int8 inference? These solutions just aren't out there, and it's why Nvidia owns the industry right now. The sheer incompetence of every other commercial hardware vendor has cemented their dominance for the foreseeable future. The industry would rather watch each other die fighting Nvidia than work together to stand a solitary chance.

talldayo 2y ago

What is your use-case where you need a GB200 but can't house it? These are datacenter processors; they consume a lot of power, output a lot of heat, and do an impressive amount of work. The same problem would arise with an Instinct or Gaudi accelerator. The real "only thing" that matters is CUDA. CUDA scales from Raspberry Pi boards to desktop cards to datacenters without skipping a beat. It's a killer product in an age where OpenCL is rotting and dead.

KaiserPro 2y ago
> where you need a GB200 but can't house it? density. Datacentre space is expensive. 9.6kw per machine means putting liquid cooling to the die, if we want to have more than 3 machines per rack. We've already lost a load of space in order to house an unspecified thousands of "h100" We need to have some space left to put in storage, otherwise we're going to wasting power for funsies. We need that storage to be close, because otherwise it gets _really_ expensive and really slow quickly.
- talldayo 2y ago
  I don't know what to tell you. Jetson Orin exists, GPU cards in all form factors support CUDA, and the servers represent the apex of that power. There are options for almost every form factor and power profile you can imagine; the goddamn Nintendo Switch supports CUDA running at 7w. Literally, what are your other options? Hot aisles get hot, I don't know what to tell you there.
  - KaiserPro 2y ago
    Yes, Orins exist, and they are frankly awful to develop on. Sure I could spend a boatload of time porting my stuff to an outdated version of ubuntu, with no practical HSM support, or I could not. Mind you, its no worse than dealing with our internal shit. You are misunderstanding what I originally saying: there is an opportunity for other people to bring in new types of accelerators. You took that to mean that I was saying nvidia are shit. GPUs TPUs and other accelerators are a pain in the arse to integrate at scale. NVIDIA have the current winner because CUDA is well understood, and the GPUs they make are fast and plentiful for the desktop. They also make a loverly goldplated "mini computer" in the form of the DGX. Again great for any startup that's just got funding. But, just like IBM, DEC & HP, who also made really fast, capable, large scalable machines, they were eaten by slower, cheaper pizzabox clusters. Not because they were better, or more efficient, but because they were cheap, plentyful and easy to sling in a room.
    - talldayo 2y ago
      Orins are fine. If your use-case is robotics and edge compute, your alternative is exclusively worse ARM boards or x86 boards. Strictly speaking, it is the "pizzabox computer" disrupting everyone else's acceleration racket. And my point isn't that I resent people insulting Nvidia; I think the world would be a better place if CUDA didn't exist. But there literally are not HPC alternatives that exist in the mainstream today. Seriously, what are you suggesting? 20,000w Beowulf clusters that accelerate everything on 10-year-old Xeons? A few thousand Macs running GPU-shader accelerated int8 inference? These solutions just aren't out there, and it's why Nvidia owns the industry right now. The sheer incompetence of every other commercial hardware vendor has cemented their dominance for the foreseeable future. The industry would rather watch each other die fighting Nvidia than work together to stand a solitary chance.