17 ms·
OpenAI builds first chip with Broadcom and TSMC, scales back foundry ambition
- DeathArrow 2y agoWill OpenAI only use the chips internally? Are they usable by other models like Llama or Claude? If not, will everyone else have to build their own chips to compete?
- whaleofatw2022 2y agoI feel like them working with broadcom is another warning sign
- mgh2 2y agoWhy? Care for references instead of opinions?
- observationist 2y agoSam seems to be finding as many devils to bargain with as he can; Broadcom is a particularly devilish company.
- crowcroft 2y agoMakes sense to work with them if they’re trying to design some kind of ASIC that would work for training or inference though?
- worldsayshi 2y agoWhat have they done?
- nick__m 2y agoThey buy companies like VMWare and Computer-Associate, gut them and jack the price until they only have captive consumers from the fortune 500.
- alephnerd 2y agoCA and VMWare were already dead on acquisition. Splunk+Datadog and AWS+Nutanix+Cohesity respectively ate much of CA and VMWare's marketshare.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- ZeroCool2u 2y agoI'm not saying that I necessarily agree, but the general consensus on HN seems to be that Broadcom is now less of a tech company and more of a holding company that raids others, for example VMWare, and extracts all value to the detriment of customers and the acquired company. I don't think that's completely wrong, but it's a big company and I'm sure there are some better areas of the company than others.
- alephnerd 2y ago> general consensus on HN General consensus on HN is generally wrong.
- OrigamiPastrami 2y agoIs there somewhere the general consensus is generally right?
- Workaccount2 2y ago"Journalists" seem to trust twitter an awful lot...
- ants_everywhere 2y agomath departments
- formerly_proven 2y agoBroadcom is on track to spend 10bn on RND this year. Basically a pure-play patent troll!
- deleted 2y ago[deleted]
- plegresl 2y agoWhy? Google also partners with Broadcom for TPU.
- nabla9 2y agoBecause Google's TPU' are so bad? https://www.theregister.com/2023/09/22/google_broadcom_tpus/ https://www.theregister.com/2023/09/22/google_broadcom_tpus/
- mhandley 2y agoBroadcom also builds xPUs for Google, Meta and Bytedance. Maybe these companies know a thing or two.
- kccqzy 2y agoBroadcom is a vendor. They provide the main skeleton of a chip so that the client only needs to focus on the product differentiator, which is the compute part of the chip. They can worry less about other parts that are not their core competency.
- high_na_euv 2y agoSo TSMC will eventually work with podcasting bro >TSMC execs allegedly dismissed Sam Altman as ‘podcasting bro’ — OpenAI CEO made absurd requests for 36 fabs for $7 trillion
- CryptoBanker 2y agoI’m sure they’ll work with anyone willing to pay
- jsheard 2y ago> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network That framing massively undersells how insane Sams ambitions were there, he was floating the idea of somehow raising seven trillion dollars to build thirty six fabs dedicated to making AI silicon. The TSMC execs reported more or less laughed in his face when he brought it up it them.
- cal5k 2y agoIt's kind of admirable, though. If you start asking for $7T, only asking for $1T becomes quite reasonable ;-)
- deleted 2y ago[deleted]
- lyu07282 2y agoOnly if they stop laughing long enough for them to hear your second offer
- yreg 2y agoWell they did stop laughing, hence this thread.
- discreteevent 2y agoNot really. The plans to build a foundry have been dropped. It's more a cautionary tale that at some point you will end up in an environment that's closer to reality where your bullshit is a liability. It was reported that the TSMC people called him a 'podcasting bro'. In other words, they lost all respect for anything he was saying.
- latexr 2y agoThat only works if the initial request isn’t so bonkers that no one can trust subsequent ones.
- latchkey 2y ago"while adding AMD (AMD.O), opens new tab chips alongside Nvidia (NVDA.O), opens new tab chips to meet its surging infrastructure demands" Jensen has been saying that demand is "insane" and we're hearing rumors of low yields. This equates to supply issues in the coming months/years. No fortune 500 puts all their eggs into one basket. Diversifying away from a single source for all of AI hardware and software, is a smart thing to do.
- vineyardmike 2y ago> Diversifying away from a single source for all of AI hardware and software, is a smart thing to do. I wonder how this squares with the exclusivity contract with Microsoft. Even the OpenAI/Oracle deal requires Oracle to run Azure stack on their datacenter so MSFT can mediate the relationship. The AMD chips mentioned are also purchased by MSFT. I wonder if this really means that OpenAI is accepting the risk/capital expense while providing a variety of hardware to Microsoft, or if there are other terms at play.
- latchkey 2y agoIt is just more of this... https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd-instinct-mi300x-accelerators-power-microsoft-a.html https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd...
- buildbot 2y agoThat's interesting given Microsoft also has it's own accelerator, Maia...
- bloodyplonker22 2y agoThat is exactly what the execs at my company are telling us when asked about not using Nvidia -- diversifying away. It's funny though because we have no Nvidia for training at all. We use Trainium because we could not get our hands on Nvidia.
- latchkey 2y ago
- qubitly 2y agoSpending $7 trillion on in-house fabs sounded both ambitious and crazy. Reality finally kicked in. If they’re done dreaming big, let’s hope they keep the quality
- stravant 2y agoMore like he's finally done price anchoring and ready to make some actual deals.
- asadotzler 2y agoIt's hardly price anchoring when they laugh at you.
- fuddle 2y agoHow long would take them to get a new chip into production and then used for training/inference?
- wmf 2y agoThe first generation chip won't be good enough to use in production so then you have the second generation... maybe 3-4 years.
- KaiserPro 2y agotwoish years, if you're good. Then the software to make it work properly. its not a quick thing to do. You can throw more money at it to make it go faster, but it also might fuck it up and take longer.
- disqard 2y agoCorrect! AMD has been trying to fix "the software to make it work properly" part for years -- more than two years.
- latchkey 2y agoAMD hardware is good. The software is getting better daily. Training still sucks (mostly due to unoptimized libraries), but inference is looking pretty decent with recent advances in tuning... https://shisa.ai/blog/posts/tuning-vllm-mi300x/ https://shisa.ai/blog/posts/tuning-vllm-mi300x/ https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html
- KaiserPro 2y ago> AMD hardware is good. The software is getting better daily. Which is the story of AMD for the last ~15 years. getting support so we could develop apps for their graphics card was dispiriting. at the time they were faster and very much cheaper than nvidia (but no cuda) But every time we found an issue, the poor devs who were contracted to fix it were left struggling. Its the same where I am now. We were trying to qualify some motherboard/TPM/other issue, but they didn't have enough bandwidth to help us in time. Its better now, we do have some AMD stuff in the fleet.
- b112 2y agoBroadcom! But that's where IP goes to languish and die.
- textlapse 2y agoOther than GH Copilot (which is amazing) and the like, I am yet to see a social/cultural impact of LLMs. I am ignoring the hype cycle fueled social media posts where influencers show cool stuff or model releases. I am talking about this stuff permeating through the society.. am I immune to it? Or is it hidden in saas/enterprise products layers deep? I would imagine some productive cultural impact that’s wide spread, sticky, high retention not like ‘oh cool 3d tv or vr’ kinda thing? Given how much ‘money’ has been burnt through the GPU heatsinks.
- ryukoposting 2y agoGo ask an English teacher how LLMs are affecting them.
- bobsyourbuncle 2y agoIt’s kinda easy to check it though. Ask your friends if they use it, preferably non developer. When I find out my accounting friends and marketing friends are using it for basic analytics/promotion write ups I realized this is being used by non developers already on a regular basis. The use by non tech folks is significant for this . Crypto as a counter example of hype hasn’t lived up to its purported use case as non tech ppl I know only “own” it through an exchange.
- textlapse 2y agoI did. Most of them if not all have tried it once just out of curiosity. None of them is using it on a regular basis- not even once a week. Maybe I am in the wrong group? Includes family / friends (non tech). Tech people (friends) use it regularly for coding though.
- surfingdino 2y agoIt's the generation currently in schools and universities that are making LLMs a part of their daily routine. Apart from people in education, a whole SEO industry has grown around LLMs and they love the newly found powers to produce spam at scale. In business / government space people are cautiously introducing LLMs to improve productivity in the comms area where slight hallucinations or robotic corporate language are acceptable, because the final text is still produced by a human. On the customer support / sales side companies are investing money in "AI" versions of the good old phone menu trees that inevitably make you scream "I want to speak to a human" before you get connected to a human being. Elsewhere adoption is slow, because we expect deterministic results and these are just not to be expected. I write software that interacts with LLMs via APIs and have to implement defensive coding techniques I haven't used for over a decade, because the output may switch from the requested structure at random. This makes use of structured output or function calling problematic.
- AdieuToLogic 2y agoConsidering the cogent financial analysis detailed in "The Subprime AI Crisis"[0], all I can add is a quote from the movie "Dodgeball"[1]: It's a bold strategy, Cotton. Let's see if it pays off for 'em. 0 - https://www.wheresyoured.at/subprimeai/ https://www.wheresyoured.at/subprimeai/ 1 - https://www.imdb.com/title/tt0364725/quotes/ https://www.imdb.com/title/tt0364725/quotes/
- _sys49152 2y agoreplying to the wheresyouredat article: >>OpenAI launched o1 — codenamed Strawberry — on Thursday night, with all the excitement of a dentist’s appointment. Across a series of tweets, Sam Altman described o1 as OpenAi’s “most capable and aligned models yet.” Though he conceded that o1 was “still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it,” he promised it would deliver more accurate results when performing the kinds of activities where there is a definitive right answer — like coding, math problems, or answering science questions. it seems to me like they nerfed the regular free ai model to give me imperfect replies on my benchmark consistently while the o1 model nails it consistently. before the free model would, depending on the day, give me more accurate replies to my benchmark . thus widening the divide to make the o1 model look more useful. openai can do whatever they want and thats fine but all it seems to me they did was make 01 look good by comparison by nerfing the free product.
- AdieuToLogic 2y ago> it seems to me like they nerfed the regular free ai model ... My main takeaway from the article was more about the unsustainable business model due to a massive cash burn rate. To wit: OpenAI’s models and products — and we'll get into their utility in a bit — are deeply unprofitable to operate, with the Information reporting that OpenAI is paying Microsoft an estimated $4 billion in 2024 to power ChatGPT and its underlying models, and that's with Microsoft giving it a discounted $1.30-per-GPU-an-hour cost, as opposed to the regular $3.40 to $4 that other customers pay. This means that OpenAI would likely be burning more like $6 billion a year on server costs if it wasn’t so deeply wedded to Microsoft — and that's before you get into costs like staffing ($1.5 billion a year), and, as I've discussed, training costs that are currently $3 billion for the year and will almost certainly increase. Whether or not the "free model" is better or worse than the "o1 model" is moot IMHO.
- Incipient 2y agoI expect they wanted to go down the fab and chip design path to build a moat, as they've realised they really don't have one outside of brand recognition. If they had proprietary fabs and chips that were magically superior, and they could build them for cheap rather than paying the team green tax, they'd have a huge advantage in inference cost. Perhaps they've realised just starting with a proprietary chip is enough?
- HarHarVeryFunny 2y agoSo presumably they're building their own data centers to use these chips, but odd that the article doesn't actually say so. Are any of the western AI/matmul chips NOT being fabbed by TSMC? Amazon or Meta's perhaps? We've got NVIDIA, AMD, Google (TPU - also designed by Broadcom), and now OpenAI all using them. Are Samsung not competitive? I wonder to what extent total chip/chipset volume is limited by TSMC capacity as opposed to memory or packaging?
- wmf 2y agoSamsung is not competitive so everything good has to be made by TSMC now. It has been reported that packaging and HBM are the bottlenecks for AI chips although TSMC is also running at high utilization.
- YetAnotherNick 2y agoSamsung is ~2-3 years behind, which is not as bad as HN claims it to be. But if you can sell the chips at 80% profit, you would want the best.
- subarctic 2y agoThis is the first I've heard of any of this, and it's funny how making their oen ships with TSMC is a "scaled back" plan
- kccqzy 2y agoNot surprising at all. As a former Google employee, I'm regularly in contact with friends still at Google and the number of hardware engineers OpenAI has hired out of Google have drastically increased in the last few months. You can readily see for yourself just by browsing LinkedIn and looking at profiles of hardware engineers currently at OpenAI and noticing how many of them are former Google employees. Google's own TPU team is now a shell of its former self. I wouldn't be surprised if future Google TPUs are inferior to the new OpenAI chips. Also, did you know that Google's TPU efforts previously also relied on help from Broadcom? https://www.theregister.com/AMP/2023/09/22/google_broadcom_tpus/ https://www.theregister.com/AMP/2023/09/22/google_broadcom_t... It's absolutely unsurprising that former Google employees would bring this vendor relationship into their new company, given Broadcom did a good job helping Google with TPUs.
- NitpickLawyer 2y agoIt's also worth mentioning that google spent a lot of time (and probably money) and human capital on their TPUs and slowly got there. Other companies that have a proven track record of hyper-scaling also tried and missed (tesla) so it's not as easy as locking a bunch of phds in a room and have them crank out LLMs. Hardware is finicky, slow to develop, slow to prove itself and extremely expensive.
- mindentropy 2y agoBRCM is becoming more like a service company designing ASICs for other companies who want custom ASICs. There are many companies who do this? How is this news? BRCM is a leader in SerDes along with Marvell and other companies. There is so many IPs from Synopsys and Cadence that in turn many of these chip companies use. Not sure what is hoopla here. Nobody is going to completely vertically integrate all these IPs.
- wmf 2y agoIt's news to people who know nothing about hardware.
- benreesman 2y agoPodcast bro.
- niggaman62420 2y ago[flagged]
- DeathArrow 2y ago> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. The company has dropped the ambitious foundry plans for now due to the costs and time needed to build a network So they planned to transform everything into paperclip factories?
- rsynnott 2y ago> OpenAI considered building everything in-house and raising capital for an expensive plan to build a network of factories known as "foundries" for chip manufacturing. ... Wait, what, _really_? _Why_? I mean, it'd probably take about a decade to get going.
- osigurdson 2y agoI keep thinking the transistor is being wasted in AI. We use binary states to remove entropy from the system. In AI, however, much more entropy is acceptable. It seems that we should be able to do a lot better than NAND gates for this use case.
- nickpsecurity 2y agoThe other reason for binary was automating synthesis vs hand-drawn circuits. That’s really important at scale. It both reduces NRE costs and increases velocity. Ditching it might mean ditching the tooling that makes it so effective.