11 ms·
Train an AI model once and deploy on any cloud
- ommz 3y agoIt would be nice if Nvidia did not enforce artificial driver and legal kneecaps to consumer Geforce cards for cloud usage to prop up their enterprise ones... but shareholder rights come before anyone.
- konschubert 3y agoIf they were not such a monopoly they could not pull this off.
- sanxiyn 3y agoNVIDIA became a monopoly by building superior products. It's not like they became a monopoly by anti-competitive practices.
- konschubert 3y agoI’m not saying they did anything bad. But a monopoly can be harmful for a market without anyone doing anything illegal.
- sanxiyn 3y agoTrue. But it is also self-correcting, since monopoly profit will attract competitors. AMD seems to be the most likely candidate.
- deleted 3y ago[deleted]
- ChuckNorris89 3y agoBut then what's stopping cloud customers from scalping up all the consumer GeForce stocks for cheap and putting those in the data center like in the crypto mining days? Cloud customers can afford to pay more for those GPUs than gamers because they generate revenue with them, gamers don't. So it make sense to have some product segmentation in place to prevent one market completely cannibalizing the other while leaving Nvidia with less profits. The current situation is still caused by manufacturing constraints at TSMC for the cutting edge nodes which both the consumer and data center parts occupy so it makes sense for Nvidia to prioritize the higher margin parts. There have been great points made that Nvidia should split into Nvidia, the general compute company oriented to data center customers with deep pockets, and in GeForce, the gaming GPU company with access to all the cutting edge tech of Nvidia but seeks to be more scrappy and optimize designs for rasterization performance rather than generic compute and chases smaller die sizes on cheaper nodes to be price competitive. This way the data center compute market will stop cannibalizing consumer gaming one and we'll be back to having better GPUs at competitive prices.
- kkielhofner 3y agoThere are some debatable licensing terms in various Nvidia driver releases that prohibit the use of consumer cards being hosted in "datacenters". But the real issue is physical form factor and power. As has been noted in the press, etc, something like an RTX 3090 (and more so 4090) is literally designed to push frames as fast as possible power and heat be damned. They're multi-slot (which results in poor density), have card design/cooling challenges, power configuration issues, etc. There's a story out there about the only dual-slot RTX 3090. Gigabyte came up with one (I have several - they're great) but supposedly Nvidia put pressure on them to pull them from the market[0] because people were putting them in x8 server configurations and using them instead of their much more expensive datacenter products. [0] - https://www.tomshardware.com/news/gigabyte-rains-partners-parade-cancelling-geforce-rtx-3090-turbo https://www.tomshardware.com/news/gigabyte-rains-partners-pa...
- neximo64 3y agoYou could always use a Geforce card at home. Are you saying the cloud should use those Geforce cards and completely distort the price of the GPUs for home use?
- izacus 3y agonVidia making sure that their consumer business isn't outscalped and destroyed by VC funded companies is a good thing. This is how they also came out on top from the crypto craze without destroying their gaming market.
- sdflhasjd 3y agoThey didn't come out on top, they revelled in it. What brought us back to some relative normalcy was the crypto crash & Etherium's switch away from PoW; even after that, the 40 series pricing and range seems to be nVidia cashing in on the scalper prices
- KaoruAoiShiho 3y agonvidia maintained MSRP of 30 series cards during the WFH boom and did not allow AIBs to increase prices, this was one of the main complaints from EVGA that ended up with them pulling out of the GPU market. The scalping was done by third parties.
- rmbyrro 3y agoThey're just trying to eat the consumer surplus from enterprise customers, which are higher up in the demand curve. Everyone does that. An individual developer is happy to charge a higher salary for its services from a larger corporation in comparison to working for an SME, simple because in a large org its services generate more value, allowing it to capture more of it.
- cj 3y agoI don’t disagree, but I think that’s a poor analogy. I don’t think devs take into account the business value their future job will bring their employer when negotiating salary. And if they do, they only do so when the balance is in their favor and they definitely wouldn’t lower their salary if they think the job has less impact than another job.
- rmbyrro 3y agoThey do when they decide to interview for large orgs. They do it because they get better pay. It's the same service. Why not work for a small org that pays less?
- __MatrixMan__ 3y agoAs a human, I do not want a level playing field when it comes to humans exploiting corporations vs corporations exploiting humans.
- smoldesu 3y agoYou have long since missed the boat on changing that. This is how business is done: "well we can charge you 5x the market price for the RAM/SSD upgrade, so we will!"
- __MatrixMan__ 3y agoIn some cases, yes. But not entirely. Open source exists to give people a way to opt out of would-be exploitation of a related kind. Things can still get a lot worse: The fight isn't over until all roads are toll roads and you have to pay for the oxygen you consume.
- villgax 3y agoIt's more on the framework that you use than nvidia at this point. Anything dockerized works with any compatible underlying hardware with no issues. Any optimization is again fragmented with FasterTransformer or TensorRT conversion with half baked layer supports which lags by 6months or more pretty much. NVAIE license is what nvidia wants enterprises to pay for using their bespoke cards in shared VRAM configuration by knee capping consumer cards which can very well do the same job better with more cuda cores but lesser memory. And don't even get me started on RIVA stack FP8 emulation is also never going to get backported instead only H100 & 4090s can make use of it
- homarp 3y agoNVAIE aka Nvidia AI enterprise, https://docs.nvidia.com/ai-enterprise/overview/0.1.0/platform-overview.html https://docs.nvidia.com/ai-enterprise/overview/0.1.0/platfor... RIVA: NVIDIA® Riva, a premium edition of NVIDIA AI Enterprise software, is a GPU-accelerated speech and translation AI SDK FasterTransformer: https://github.com/NVIDIA/FasterTransformer https://github.com/NVIDIA/FasterTransformer an highly optimized transformer-based encoder and decoder component, supported on pytorch, tensorflow and triton TensorRT, custom ml framework/ inference runtime from nvidia, https://developer.nvidia.com/tensorrt https://developer.nvidia.com/tensorrt, but you have to port your models
- codethief 3y ago> Any optimization is again fragmented with FasterTransformer or TensorRT conversion with half baked layer supports which lags by 6months or more pretty much. Thanks, I came here to see whether anything had changed since I last did ML stuff on Nvidia GPUs, and it looks like things are still the same.
- villgax 3y agoAt this point the benefits of a GPU get outmatched by CPUs even if the latency is 5-10X since you can scale CPU cores cheaper than GPUs both on prem or on public cloud
- 3y ago
- politelemon 3y agoI'm failing to see why k8s needs to be involved here - it's overkill for most model serving cases but its involvement here now adds additional overhead. So it's not really any cloud, it's any cloud where you're running your EKS/AKS etc.
- ianpurton 3y agoKubernetes means you don't have to learn each clouds way of doing a deployment. You just learn the k8s way then use that with Google, Azure or whatever. So your skillset is reusable.
- finikytou 3y agono it doesnt mean that. you still need to know how to operate k8s on a specific cloud.
- quickthrower2 3y agohell yes.
- jml78 3y agoJust like you have to know how to operate each cloud in general. There is no free lunch. But if you learn k8s, moving from AWS EKS to Google GKE to DigitalOceans hosted k8s is easy.
- Art9681 3y agoNo you don't have to. You can deploy your own cluster instead of using the managed option if you want to. A good SRE can deploy and manage EKS. A great SRE can deploy and manage a cluster to any Cloud without ever touching the dashboards.
- hhh 3y agoIdeally the developer doesn’t. At scale some platform or infra team should.
- artdigital 3y ago
- thih9 3y agoVery off topic, every time I see nvidia expand towards AI products I'm reminded that they had every opportunity to expand towards crypto products and didn't. I like that they work on what they believe in - and skip if they don't. In a time when AI is becoming a buzzword, this feels refreshing.
- quickthrower2 3y agoMaybe they are hype immune - clearly crypto is zero sum and somewhat seasonal. Machine learning (and matmul and relu in particular) is here to stay and will expand.
- Culonavirus 3y ago> and didn't Uh huh. > Nvidia will pay $5.5 million to settle charges that it unlawfully obscured how many of its graphics cards were sold to cryptocurrency miners... And > The CMP HX is a pro-level cryptocurrency mining GPU that provides maximum performance... Just a quick google away. Nvidia will develop and sell whatever will make Nvidia more money. They just think the world of AI is two or three orders of magnitude more lucrative than mining ever was. Hence the maximum push on the AI front.
- bushbaba 3y agoCrypto mining using GPUs has crashed. Ether was the main source of profit, and the shift away from proof of work dried that up. Bitcoin requires ASICs without the market for nvidia, and recent conditions made this only worse. Nvidia knows their biggest revenue sources today, which are growing, and is investing into their business units based on that data. It’s just smart business.
- KaoruAoiShiho 3y ago> Nvidia will pay $5.5 million to settle charges that it unlawfully obscured how many of its graphics cards were sold to cryptocurrency miners... This is because they didn't serve the market... so they didn't understand how many buyers were coming from crypto.
- archerx 3y ago
- zaalps 3y ago[flagged]
- paganel 3y agoThe AI shovels industry is doing good business. Other than that, any major use-case behind the recent AI hype? One that has brought tangible benefits, or at the very least a positive ROI.
- andrewcamel 3y agoI'm starting to outline them here: ctlresearch.com . Upcoming interviews with Chief Architect at Intuit, Head of Procurement at DoD, etc. DoD already shortened process of writing structured "requests from industry" from 3 months to 1 day. Makes it far easier to get requests out to vendors. Next step is an auto-complete bot that helps vendors respond with required language to RFPs. I have 20 interviews coming down the pipe -- all of which have highly tactical / near term valuable ideas like this.
- moneywoes 3y agoDo you have a blog or are these market research ideas
- andrewcamel 3y agoIt's a collection of interviews posted in the form of a library. So bit blog-like in structure, but just a collection of ideas on how to leverage this new tech.
- moneywoes 3y agoWhat info do you digest daily?
- throwawaybbq1 3y agoI work at an industry research lab. Key challenge for LLMs is the legality and massive resources needed to train. I have research colleagues that are convinced that even OpenAI may be on shaky legal ground. A lot of non-profit and academic liasoning helps to muddy the issue (academics have fair-use exceptions). If you don't see the potential of the tech and the rapid advances, I can't help you. But the issue around deployment is more legal (and perhaps not enough GPUs to go around).
- csears 3y agoCongrats to the Run:ai team. This looks like a pretty big endorsement from Nvidia.
- hospitalJail 3y agoWe need local models for our confidential data. Nvidia, we already can train using OpenAI or a beefy hosted server. But this particular data is air gapped.
- jokethrowaway 3y agoCool! Is the cost AWS level of waste - or something reasonable? I can get an A4000 with 16GB vram which can run some models for 140$ per month. I can't say the setup is anything special really but not having to do that has some value
- lee101 3y ago[dead]