4 ms·
Presumably the 32GB of VRAM is what makes it compelling, as you could cram some fairly substantial AI models on there.
by 22c 3y ago
Presumably the 32GB of VRAM is what makes it compelling, as you could cram some fairly substantial AI models on there.
- iaw 3y ago33% increase in VRAM over a consumer card at 150% increase in price? If CUDA is 100% compliant on AMD we'll actually see competition in this space.
- darklycan51 3y agoYeah, because price is a deciding factor for corporations, lol
- photonbeam 3y agoIn the long-term it is
- frognumber 3y agoNot really. If a SWE salary is $200k, total cost is probably around $400k. A $4k GPU, if bought each year, increases SWE cost by 1-2%. Buying a nice GPU was the best purchase my employer made, even if I use it every 6 months. It's already paid for itself many times over.
- PeterStuer 3y agoYou should never compare an additional cost to the overall c ost in a production setting. You should compare it to the profit margin. If you are running a 10% margin, then a 2% cost increase is -20% profit, a quite substantial amount.
- frognumber 3y agoThe key question is output per dollar. If employees are more efficient, I need fewer of them. People cost a lot of money relative to capital expenses like nice laptops, good chairs, or individual GPUs. This kind of spending almost always pays off very quickly.
- PeterStuer 3y agoYes, I also believe that. The only thing I argue against is the 'negligeable' marginal cost statement fed to devs by dev gurus because it is a fallacy. I'm all for investing in quality work experience. Decent equipment being the most obvious. Not only for the potential higher productivity, but also for its definite hr/retention effects. Investing in tangible production improvements and comfort is money spent 10x more effective than the hr semi-mandatory fun/social events that have metastasised throughout tech. Where asking wasn't an outcome, in contrast to some other colleague that just went "I guess we will just deliver below potential then" I used to buy my own stuff because I respected my time, but it definitely didn't want to keep me working there. Now this does not mean you have to shell out for every whim. If you want a 10.000€ PC setup instead of a 4.000€ one, I will need a better justification than 'custom loop and RGB'.
- frognumber 3y agoLet me be very clear: Because I have a fancy GPU in my computer, I was able to prototype algorithms in my spare time (e.g. nights and weekends), learn a lot more about the state of machine learning, and use this in production. I can think of one specific use -- processing a data set -- which allowed me to do something automatically which would have used a few hours manual time. My hourly salary times hours saved on that one use already paid for the GPU. I use Stable Diffusion many times to make images for presentations before it was mainstream (yes, there were a few months before everyone was using DALLE and SD). That had a noticeable impact which went well beyond the cost of the GPU. And so on. Yes, I could have done this at lower cost on cloud-based services with only a little bit more overhead, but: 1) I wouldn't have put in the extra work 2) If I did, I wouldn't have been able to justify the expense when things were at the half-baked stage Things like Hugging Face are at a level where this DOES increase productivity by far more than it costs, and not in the abstract hr/retention effects way, but in terms of concrete output. More critically, these tools are now part of our tool set. That means when new opportunities come up, we can leverage them.
- wkat4242 3y agoHave you worked at an enterprise? Everything is about money. Don't forget the scale at which these operate. $1000 extra in a GPU adds up quickly when you buy a thousand of them. Perhaps in big tech it's different but in our huge multinational anything IT is seen as a cost center only and has to be on the cheap. Which a real focus on short term savings because who knows if this year's SVP will still be in that seat next year. Nobody ever told me "let's take the most expensive option because it's the best in the long run" :(
- syntaxing 3y agoThat's kinda what top parent comment is talking about though. Money doesn't matter when you only have one choice. Enterprise can ONLY buy Nvidia cards so money really doesn't matter in this case. Technical execution is more important than anything else, including money. If you can't afford or make profit to buy cards to support what you need (more VRAM in this case), the project is dead in the water to begin with.
- valine 3y agoYou can split your AI models between two 4090s and have access to 48GB of VRAM for less money. You do take a slight performance hit compared to running on one card, but the 4090 also has more CUDA cores which help mitigate the slowdown.