4 ms·
>This largely depends on whether you own or rent your hardware. At $40,000 per B200, your lifetime cost per user is 40_000/num_users. In the 100% duty cycle cas
by smalltorch 4mo ago
>This largely depends on whether you own or rent your hardware. At $40,000 per B200, your lifetime cost per user is 40_000/num_users.
In the 100% duty cycle case (worst for cost), that's 6k$ per user. Realistically, serving 300 users per GPU you'll spend a lifetime cost of about $133 per user, plus the datacenter/upkeep bill.
If you rent the GPU, the cost is more straightforward. At an hourly rate of $43, your hourly cost per user is 4/num_users. For num_users=300 you get an hourly rate of about $0.013 per user, or $9.36 per month.
This leads me to believe you can buy a GPU but leave it at a data center?
Do people do this? I don't understand. Or are you equating upkeep bill to electricity on premises?
- __s 4mo agoYou can, people do. https://www.linkedin.com/posts/activity-7409593739138060288-OFmi https://www.linkedin.com/posts/activity-7409593739138060288-...
- smalltorch 4mo agoSo what's the cost separating them from placing this box at their premise? Network throughout?
- deleted 4mo ago[deleted]
- namibj 4mo agoPlus power and cooling.
- BadBadJellyBean 4mo agoPlus space, manpower and security.
- smalltorch 4mo agoThe napkin math should figure in the base amount of GPU's i need to buy in order to be able to place it in a data center otherwise this math is not relevant. Because surely any place can cool, power, and protect a single gpu easily. Not having physical access to my assets doesn't sound secure at all, and even a residential internet connection could handle this throughout.
- injuly 4mo agoYes. You can either rent an entire blade, or purchase a dis-assembled box (majority share of the price will be GPU), and place it at a datacenter. This cannot be done on most premises because of power, noise, and cooling.