4 ms·

I'm hoping when the shit hits the fan, you can get a sick GPU for cheap.

gonzo41 2mo ago
it'll be a rack based GPU without video ports. The only thing getting sick GPU's will be landfill when they all burn out and the data center rationalization happens.
- kees99 2mo ago
  More like rack-sized GPU that takes in half-megawatt of power as 800 volts DC, and requires 10 gallons per second of liquid cooling or it'll catch fire.
  - SturgeonsLaw 2mo ago
    Looks like I'll have to get back into casemodding then
- trescenzi 2mo ago
  Totally agree these won’t go to consumers but even after a crash I’d home they are able to be sold for parts and not just to landfills. If this all goes into the trash that would be even more tragic than it already is.
  - palmotea 2mo ago
    > Totally agree these won’t go to consumers but even after a crash I’d home they are able to be sold for parts and not just to landfills. If this all goes into the trash that would be even more tragic than it already is. I'm not an accountant (so I could be wrong), but I'm vaguely under the impression it's sometimes financially beneficial to "write off" inventory, and to do that you have to destroy the items (e.g. https://en.wikipedia.org/wiki/Atari_video_game_burial https://en.wikipedia.org/wiki/Atari_video_game_burial, https://appleinsider.com/articles/23/05/30/apples-lisa-entombment-was-just-the-beginning-shows-new-documentary https://appleinsider.com/articles/23/05/30/apples-lisa-entom...). So I kinda wouldn't be surprised if a bunch of these GPUs go in the dump, given that it sounds like they'd be extremely power-hungry and difficult to use outside a hyperscale data center setting.
- leoqa 2mo ago
  I went down a rabbit hole of refurbing burnt out crypto gpus. It’s really not too complicated but can be hit or miss. Need to get a box full.
  - riknos314 2mo ago
    The proper server AI chips come on a board that isn't compatible with consumer pcie lanes and have no built-in fans. Completely different ball game in the ultra dense server world.
- veqq 2mo ago
  > rack based GPU without video ports Still fine for running APL with the dfns compiler, Futhark or other array languages directly hosted on the GPU itself!
- GuestFAUniverse 2mo ago
  Nope. The scientific community is eagerly waiting for prices to drop. Single 5090, single H100, a few 4090s, a few Blackwell PRO 6000... that's all there often is as dedicated devices for a faculty (apart from the oversubscribed bigger clusters with big stuff). Apart from the chairs doing vision, most of them use them headless. So, landfill will be highly unlikely, if things get sold fast enough.
  - dijksterhuis 2mo ago
    as the ml phd student who volunteered to be the admin for the gpu servers, that list is pretty accurate.
cmiles8 2mo ago
More likely is communities will end up with a bunch of half-built abandoned datacenter projects.
- fwipsy 2mo ago
  Isn't that what they said?
nswango 2mo ago
I think a more likely scenario is that GPU time in the cloud will be very cheap. If LLM use/profitability doesn't follow the expected path, new deep learning applications could be heavily subsidised.

it'll be a rack based GPU without video ports. The only thing getting sick GPU's will be landfill when they all burn out and the data center rationalization happens.

kees99 2mo ago
More like rack-sized GPU that takes in half-megawatt of power as 800 volts DC, and requires 10 gallons per second of liquid cooling or it'll catch fire.
- SturgeonsLaw 2mo ago
  Looks like I'll have to get back into casemodding then
trescenzi 2mo ago
Totally agree these won’t go to consumers but even after a crash I’d home they are able to be sold for parts and not just to landfills. If this all goes into the trash that would be even more tragic than it already is.
- palmotea 2mo ago
  > Totally agree these won’t go to consumers but even after a crash I’d home they are able to be sold for parts and not just to landfills. If this all goes into the trash that would be even more tragic than it already is. I'm not an accountant (so I could be wrong), but I'm vaguely under the impression it's sometimes financially beneficial to "write off" inventory, and to do that you have to destroy the items (e.g. https://en.wikipedia.org/wiki/Atari_video_game_burial https://en.wikipedia.org/wiki/Atari_video_game_burial, https://appleinsider.com/articles/23/05/30/apples-lisa-entombment-was-just-the-beginning-shows-new-documentary https://appleinsider.com/articles/23/05/30/apples-lisa-entom...). So I kinda wouldn't be surprised if a bunch of these GPUs go in the dump, given that it sounds like they'd be extremely power-hungry and difficult to use outside a hyperscale data center setting.
leoqa 2mo ago
I went down a rabbit hole of refurbing burnt out crypto gpus. It’s really not too complicated but can be hit or miss. Need to get a box full.
- riknos314 2mo ago
  The proper server AI chips come on a board that isn't compatible with consumer pcie lanes and have no built-in fans. Completely different ball game in the ultra dense server world.
veqq 2mo ago
> rack based GPU without video ports Still fine for running APL with the dfns compiler, Futhark or other array languages directly hosted on the GPU itself!
GuestFAUniverse 2mo ago
Nope. The scientific community is eagerly waiting for prices to drop. Single 5090, single H100, a few 4090s, a few Blackwell PRO 6000... that's all there often is as dedicated devices for a faculty (apart from the oversubscribed bigger clusters with big stuff). Apart from the chairs doing vision, most of them use them headless. So, landfill will be highly unlikely, if things get sold fast enough.
- dijksterhuis 2mo ago
  as the ml phd student who volunteered to be the admin for the gpu servers, that list is pretty accurate.

cmiles8 2mo ago

More likely is communities will end up with a bunch of half-built abandoned datacenter projects.

fwipsy 2mo ago
Isn't that what they said?

nswango 2mo ago

I think a more likely scenario is that GPU time in the cloud will be very cheap. If LLM use/profitability doesn't follow the expected path, new deep learning applications could be heavily subsidised.