6 ms·
Are they buying them to try and slow down open source models and protect the massive amounts of money they make from OpenAI, Anthropic, Meta ect? It quite obvi
by impulser_ 10mo ago
Are they buying them to try and slow down open source models and protect the massive amounts of money they make from OpenAI, Anthropic, Meta ect?
It quite obvious that open source models are catching up to closed source models very fast they about 3-4 months behind right now, and yeah they are trained on Nvidia chips, but as the open source models become more usable, and closer to closed source models they will eat into Nvidia profit as these companies aren't spending tens of billion dollars on chips to train and run inference. These are smaller models trained on fewer GPUs and they are performing as good as the pervious OpenAI and Anthropic models.
So obviously open source models are a direct threat to Nvidia, and they only thing open source models struggle at is scaling inference and this is where Groq and Cerberus come into the picture as they provide the fastest inference for open source models that make them even more usable than SOTA models.
Maybe I'm way off on this.
- matthewfcarlson 10mo agoIdk- cheaper inference seems to be a huge industry secret and providing the best inference tech that only works with nvidia seems like a good plan. Makes nvidia the absolute king of compute against AWS/AMD/Intel seems like a no brainer.
- SkyPuncher 10mo agoThey need to vertically integrate the entire stack or they die. All of the big players are already making plans for their own chips/hardware. They see everyone else competing for the exact same vendor’s chips and need to diversify.
- Workaccount2 10mo agoShy of an algo breakthrough, open source isn't going to catch up with SOTA, their main trick for model improvement is distilling the SOTA models. That's why they they have perpetually been "right behind".
- impulser_ 10mo agoThey don't need to catch up. They just need to be good enough and fast as fuck. Vast majority of useful tasks of LLMs has nothing to do with how smart they are. GPT-5 models have been the most useless models out of any model released this year despite being SOTA, and it because it slow as fuck.
- nineteen999 10mo agoBullseye.
- dontwannahearit 10mo agoConfused. Is ‘fuck’ fast or slow? Or both at the same time? Is there a sort of quantum superposition of fuck?
- ThrowawayTestr 10mo agoIt's an intensifier
- 867-5309 10mo agowell, it's not slow as fuck! it's quick as lightning and speedy as hell
- deleted 10mo ago[deleted]
- aschobel 10mo agoFor coding I don’t use any of the previous gen models anymore. Ideally I would have both fast and SOTA; if I would have to pick one I’d go with SOTA. There a report by OpenRouter on what folks tend to pay for it; it generally is SOTA in the coding domain. Folks are still paying a premium for them today. There is a question if there is a bar where coding models are “good enough”; for myself I always want smarter / SOTA.
- _fizz_buzz_ 10mo ago> their main trick for model improvement is distilling the SOTA models Could you elaborate? How is this done and what does this mean?
- MobiusHorizons 10mo agoI am by no means an expert, but I think it is a process that allows training LLMs from other LLMs without needing as much compute or nearly as much data as training from scratch. I think this was the thing deepseek pioneered. Don’t quote me on any of that though.
- tickerticker 10mo agoYes. They bounced millions of queries off of ChatGPT to teach/form/train their DeepSeek model. This bot-like querying was the "distillation."
- SirMaster 10mo agoWhy would OpenAI allow someone to do that?
- qcnguy 10mo agoThey don't anymore. They introduced ID verification shortly after, but it's hard to stop completely while also scaling fast.
- MadnessASAP 10mo agoThey didn't, but how do you stop it? Presuming the scale that OpenAI is running at?
- orbital-decay 10mo agoThey definitely didn't. They demonstrated their stuff long before OAI and the models were nothing like each other.
- 10mo ago
- stx5 10mo ago[dead]
- mistercheph 10mo agoToo bad, so sad for the Mister Krabs secret recipe-pilled labs. Shy of something fundamental changing, it will always be possible to make a distillation that is 98% as good as a frontier model for ~1% of the cost of training the SOTA model. Some technology just wants to be free :)
- blackoil 10mo agoWe trust in our lord and savior China and Zuck to keep the peasants fed.
- __mharrison__ 10mo agoHow does this work considering the Nemotron models?
- nl 10mo agoNVIDIA release some of the best open source models around. Almost all open source models are trained and mostly run on NVIDIA hardware. Open source is great for NVIDIA. They want more open source, not less. Commoditize your complement is business 101.
- impulser_ 10mo agoThen why are they spending $20 billion dollars to handicap an inference company that giving open source models a major advantage over closed source models?
- credit_guy 10mo ago> to handicap an inference company That's a non-charitable interpretation of what happened. The are not "spending $20 billion to handicap Groq". They are handing Groq $20 billion to do whatever they want with it. Groq can take this money and build more chips, do more R&D, hire more people. $20 billion is truly a lot of money. It's quite hard to "handicap" someone by giving them $20 billion.
- wmf 10mo agoGroq doesn't have any employees. They can't do R&D because there's no one to do it. The $20B goes to Groq's investors.
- credit_guy 10mo agoFrom the article: > Groq added that it will continue as an “independent company,” led by finance chief Simon Edwards as CEO. The $20B does not go to Groq's investors. It goes to Groq. You can say that Groq is owned by its investors, and this is the same thing, but it's not. In order for the money to go to the investors, Groq needs to disburse a dividend, or to buy back shares. There is no indication that this will happen. And what's more, the investors don't even need this to happen. I'm sure any investor that wants to sell their shares in Groq will now find plenty of buyers at a very advantageous price.
- ymck 10mo agoI'd say that it's probably not a play against open source, but more trying to remove/change the bottlenecks in the current chip production cycle. Nvidia likely doesn't care who wins, they just want to sell their chips. They literally can't make enough to meet current demand. If they split off the inference business (and now own one of the only purchasable alternatives) they can spin up more production. That said, it's completely anti-competitive. Nvidia could design a inference chip themselves, but instead the are locking down one of the only real independents. But... Nobody was saying Groq was making any real money. This might just be a rescue mission.
- ilaksh 10mo agoYes, you are way off, because Groq doesn't make open source models. Groq makes innovative AI accelerator chips that are significantly faster than Nvidia's.
- zamalek 10mo ago> Groq makes innovative AI accelerator chips that are significantly faster than Nvidia's. Yeah I'm disappointed by this, this is clearly to move them out of the market. Still, that leaves a vacuum for someone else to fill. I was extremely impressed by Groq last I messed about with it, the inference speed was bonkers.
- wmf 10mo agoIf it's that good Nvidia can just keep selling it.
- allovertheworld 10mo agomore like now Nvidia wants to release their own ASIC to combat google
- fragmede 10mo agoUmm... no one tell them, okay?
- LoganDark 10mo agoFor inference, but yes. Many hundreds of tokens per second of output is the norm, in my experience. I don't recall the prompt processing figures but I think it was somewhere in the low hundreds of tokens per second (so slightly slower than inference).
- heavyset_go 10mo agoNvidia just released their Nemotron models, and in my testing, they are the best performing models on low-end consumer hardware in both terms of speed and accuracy.
- ramoz 10mo agoThey acquired in order to have an ASICs competitor to Google TPU.
- Kiboneu 10mo ago>Are they buying them to try and slow down open source models The opposite, I think. Why do you think that local models are a direct threat to Nvidia? Why would Nvidia let a few of their large customers have more leverage by not diversifying to consumers? Openai decided to eat into Nvidia's manufacturing supply by buying DRAM; that's concretely threatening behavior from one of Nvidia's larger customers. If Groq sells technology that allows for local models to be used better, why would that /not/ be a profit source for Nvidia to incorporate? Nvidia owes a lot of their success on the consumer market. This is a pattern in the history of computer tech development. Intel forgot this. AMD knows this. See where everyone is now. Besides, there are going to be more Groqs in the future. Is it worth spending ~20B for each of them to continue to choke-hold the consumer market? Nvidia can afford to look further. It'd be a lot harder to assume good faith if Openai ended up buying Groq. Maybe Nvidia knows this.
- deaux 10mo ago> Besides, there are going to be more Groqs in the future. And likely some of them are going to be in countries that won't let them sell out to Nvidia.
- vachina 10mo agoMore like they’re trying to snuff out potential competitors. Why work as hard to push your own products if NVIDIA gave you money to retire for the rest of your life?
- karmasimida 10mo agoWith RAM/memory price this high, open source is not going to catch up with closed source. The open source economy relies on the wisdom of crowds. But that implies and equal access to experimentation platforms. The democratization of PC and consumer hardware brings the previous open source era that we all love, I am afraid the tech mongols had identified the chokehold of LLM ecosystem and found ways to successfully monopolized it
- PunchyHamster 10mo agoYou still need hardware to run open source models. It might eat into OpenAI profit but I doubt it will eat into NVIDIA's If anything more companies in making models business the higher NVIDIA chip demand will be, till we get some proper competition at least. We badly need some open CUDA equivalent so moving off to competition isn't a problem
- HPsquared 10mo agoNvidia's dream would be for everyone to buy a personal DGX H100 for private local inference. That's where open source could lead. Datacenters are much more efficient in their use of chips.
- mr_toad 10mo agoThe constant threat of open source (and other competitors) is what keeps the big fish from getting complacent. It’s why they’re spending trillions on new data centers, and that benefits Nvidia. When there’s an arms-race on it’s good to be an arms dealer.
- nbardy 10mo agoYour way off, this reads more like anti capitalist political rhetoric than real reasoning. Look at Nvidia nemotron series. They hav become a leading open source training lab themselves and they’re releasing the best training data, training tooling, and models at this point.
- AmazingTurtle 10mo agoShow me an affordable open source coding model thats closet to GPT-5.2-codex capabilities. Note: I do not have tons of HBM lying around
- epolanski 10mo agoI don't see where is the benefit for Nvidia to limit the open source models. The more competition, the more shovels they sell. It's like saying that Intel would've benefited if only Dell and few others sold servers because they brought in multiple billions per year.
- jayanmn 10mo agoChina may take over the open source part. That is the only country with exposure to hardware, software and political might.
- nurettin 10mo ago> It quite obvious that open source models are catching up to closed source models very fast they about 3-4 months behind > Maybe I'm way off on this. If by open source, you mean downloadable from huggingface and SOTA you mean opus 4.5, yes you are way off.
- baconner 10mo agoNVIDIA makes money no matter if the model is open weights or not. I don't think open is a concern for them and they'd very much like to be servicing China and their batch of open models I think. what's concerning them more likely is A. The inevitable breakdown of their massive head start with CUDA and data center hardware. A serious competitor at real scale. B. Anything that'll cool off the massive data center buildouts that are fueling them. Seems clear that locking up a major potential competitor especially the minds behind it solves for A. And their ongoing machinations with circular funding of companies funding data centers is all about B - keeping the momentum before it fizzles.