3 ms·

I guess somewhere this year having your own GPU might be cheaper than renting.

by ozim 10mo ago

speedgoose 10mo ago
I wonder what will happen with cloud GPU prices once the AI bubble pops. Will they keep having high prices, or will they drop prices significantly ?
- deleted 10mo ago
  [deleted]
- ezst 10mo ago
  My guess is, the bubble popping means the demand wasn't real. No amount of price reduction will motivate it. Anybody who has to get theirs will be in for a steep premium.
BloodyIron 10mo ago
It already is. It's why I launched this very recently: https://it.lanified.com/Solutions/Managed-AI-Solutions https://it.lanified.com/Solutions/Managed-AI-Solutions
Aachen 10mo ago
When was buying a device from the manufacturer not cheaper than paying a middleman to manage rentals of said device?
- noname120 10mo ago
  When you need to scale up and down dynamically.
  - Aachen 10mo ago
    Then the claim above can never be falsified because there will always be a value you can plug into the equation at which it's cheaper to rent (like at 50% occupancy, at 5%, at 0.5%... so long as you're not the only person in the world who wants to use the device being offered for loan). I would assume we're talking about a common workload where one actually needs the device that one considers purchasing for some reasonable fraction of the time and not less than half the time
    - lucketone 10mo ago
      So in summary: - own the hardware, if you have constant load and have enough expertise (personally or a team) to use/maintain it at your required reliability level - rent the cloud, if your usage is opportunistic/ spike’y/rapidly changing - also rent the cloud, if required expertise/maintenance would cost you more than the hardware (if you want to have gazillion of nines, you need somebody who’s there to deal with smoking stuff, in several locations)
      - Aachen 10mo ago
        That sounds like good common sense, though I'd reword the "constant" with like <30% occupancy because cloud premiums are rather large My question was (to what ozim wrote), how's that new in 2026
- efreak 10mo ago
  When you barely need said device, or you occasionally need many of said device. Not to mention you're renting more than just a GPU from them, you're getting the rest of the server too. I rent from runpod a handful of days a year (maybe a week or so total?) and I've estimate that I've spent at most $300 total in the last 2 years. - I get a 3090 or 4090, sometimes a 5090 or a pro GPU with 40+gb vram. (I have no dGPU). - Usually they have >100gb ram (my new PC has 64gb, the old one had 32gb) - 20-30 cores available (my new PC has 20, old was 4gb ddr2(!)) - 1-2tb of disposable nvme storage (my old PC didn't have nvme at all, my current PC has ~800gb free of 2tb) - fast network. I don't always test it, but I'm usually done downloading ~700gb of models in under 2 hours; locally this would take around 15 hours. - electricity is far cheaper in random east European/Asian City than it is in Los Angeles. Some GPUs cost less to rent than they do to run locally, never mind the cost of buying the GPU itself. Paying runpod or vast or just about anyone else is _far_ cheaper than buying this hardware myself.

I guess somewhere this year having your own GPU might be cheaper than renting.

by ozim 10mo ago

I guess somewhere this year having your own GPU might be cheaper than renting.

I wonder what will happen with cloud GPU prices once the AI bubble pops. Will they keep having high prices, or will they drop prices significantly ?

deleted 10mo ago
[deleted]
ezst 10mo ago
My guess is, the bubble popping means the demand wasn't real. No amount of price reduction will motivate it. Anybody who has to get theirs will be in for a steep premium.

BloodyIron 10mo ago

It already is. It's why I launched this very recently: https://it.lanified.com/Solutions/Managed-AI-Solutions https://it.lanified.com/Solutions/Managed-AI-Solutions

Aachen 10mo ago

When was buying a device from the manufacturer not cheaper than paying a middleman to manage rentals of said device?

noname120 10mo ago
When you need to scale up and down dynamically.
- Aachen 10mo ago
  Then the claim above can never be falsified because there will always be a value you can plug into the equation at which it's cheaper to rent (like at 50% occupancy, at 5%, at 0.5%... so long as you're not the only person in the world who wants to use the device being offered for loan). I would assume we're talking about a common workload where one actually needs the device that one considers purchasing for some reasonable fraction of the time and not less than half the time
  - lucketone 10mo ago
    So in summary: - own the hardware, if you have constant load and have enough expertise (personally or a team) to use/maintain it at your required reliability level - rent the cloud, if your usage is opportunistic/ spike’y/rapidly changing - also rent the cloud, if required expertise/maintenance would cost you more than the hardware (if you want to have gazillion of nines, you need somebody who’s there to deal with smoking stuff, in several locations)
    - Aachen 10mo ago
      That sounds like good common sense, though I'd reword the "constant" with like <30% occupancy because cloud premiums are rather large My question was (to what ozim wrote), how's that new in 2026
efreak 10mo ago
When you barely need said device, or you occasionally need many of said device. Not to mention you're renting more than just a GPU from them, you're getting the rest of the server too. I rent from runpod a handful of days a year (maybe a week or so total?) and I've estimate that I've spent at most $300 total in the last 2 years. - I get a 3090 or 4090, sometimes a 5090 or a pro GPU with 40+gb vram. (I have no dGPU). - Usually they have >100gb ram (my new PC has 64gb, the old one had 32gb) - 20-30 cores available (my new PC has 20, old was 4gb ddr2(!)) - 1-2tb of disposable nvme storage (my old PC didn't have nvme at all, my current PC has ~800gb free of 2tb) - fast network. I don't always test it, but I'm usually done downloading ~700gb of models in under 2 hours; locally this would take around 15 hours. - electricity is far cheaper in random east European/Asian City than it is in Los Angeles. Some GPUs cost less to rent than they do to run locally, never mind the cost of buying the GPU itself. Paying runpod or vast or just about anyone else is _far_ cheaper than buying this hardware myself.