11 ms·
OpenAI is exploring making its own AI chips
- cpersona 3y agoWhen building out an initiative like this, how do companies avoid IP issues? They are looking to build technology that competes with the best in class to make it worth the effort without having to reinvent the wheel.
- cmrdporcupine 3y agoWish I could buy stock in Tenstorrent.
- makestuff 3y agoMeta was rumored to want to layoff its chip fab team for VR. I wonder if OpenAI will work out a deal with them.
- vhiremath4 3y agoIt’s interesting to think, when a company starts to vertically integrate, how deep do you go? Seems like OpenAI is exploring its own devices/OS as well, which makes sense to me, but it’s a vertical integration bet. This seems to be another big bet, but they could benefit from having their own optimized chips regardless of whether the device/OS bet wins out. Extremely exciting times for OpenAI!
- hashtag-til 3y agoVery exciting (from an engineer point of view), but there is much much more in a chips/devices/OS than "AI". It's a trap! To me, it seems like a distraction for them to go into devices stuff, rather than making their stuff so relevant that device vendors (Android/iOS) can't ignore. At the moment they can because they got good enough competitor solutions.
- baq 3y agoDepends what they've got under the hood. If they found a theoretical way to infer something like GPT-3.5 without using so much RAM and can build a chip which makes this feasible in laptops or (holy grail) phones, they've got their moat for a 12-24 months, possibly more if they manage to patent it. Big if though.
- leetcodesucks 3y ago[dead]
- Legend2440 3y agoThere's almost certainly no way to avoid using massive amounts of memory. But you could build a device with more memory...
- vineyardmike 3y agoConsidering the unit prices of NVidia chips, and the growing cuda software lock-in, it seems “obvious” to work on something to hedge against price increases. That might be the performance or cost edge they need. If they just make android phones better, they risk Google benefitting off their efforts and replacing the models. Besides that’s a lot more work to partner and manage across the industry. It seems risky to bet they can release consumer hardware (instead of server side) but that’s the ultimate big bet. Ask Meta how it feels to be “just an app”.
- theptip 3y ago> how deep do you go? As far as gives you a competitive edge! For AI, to win you need better data and compute than your competitors. Going up the stack to consumer devices seems like a somewhat speculative move, though I understand the underlying desire to secure a data moat. Going down the stack to chips makes a lot of sense; if you can secure an edge in compute efficiency then you will beat anybody that doesn't have substantially more data than you do.
- causi 3y agoAm I the only person who thinks OpenAI just doesn't know where to go from here?
- bcjordan 3y agoStill feels like they're laser-focused on achieving AGI/aligned ASI. Improving chips and evolving the UX feel like cohesive (possibly requisite) intermediate steps
- baq 3y agoIt might be that, or they might know exactly where to go from here, just need the hardware.
- hcks 3y agoThey stumbled upon a mass market product while releasing a rough research poc… If anything it vindicates even more their initial thesis about pursuing AGI as a business goal.
- vb-8448 3y agoI guess they are trying to find a path to profitability and scalability: the current setup will not to scale much further, and they need some more energy efficient solution.
- theptip 3y agoAre we watching the same company? They have been shipping new features like crazy. Code interpreter blew peoples' minds, they are training GPT-5 and I bet they are leaning into multi-modal more than with GPT-4V. Strategically multimodal is the big frontier on which they are going to continue expanding datasets. With that in mind, expanding their apps to ingest more audio and image data is an obvious strong move. (And you can see why a consumer device would help them get even more data from the real world, though it's less obvious to me that this is a win vs. just shipping apps.)
- ikeashark 3y agoOpenAI confirms they aren't working on or training a a "GPT-5" https://techcrunch.com/2023/06/07/openai-gpt5-sam-altman/#:~:text=OpenAI%20is%20still%20not%20training,Sam%20Altman's%20large%20language%20models https://techcrunch.com/2023/06/07/openai-gpt5-sam-altman/#:~....
- johngossman 3y agoGiven the cost of gpus it would be negligent if they weren’t at least looking for alternatives. A story like this could also help negotiate prices with their suppliers. And everyone is looking at the success Apple had with custom silicon. But I suspect they’d prefer to partner or find alternative (cheaper) suppliers
- amelius 3y agoI'm so tired of all this vertical integration. Can't we have hardware companies that make hardware, software (AI) companies that make software, and data companies (or government institutions) that run the software on the hardware and deal with our data?
- icapybara 3y agoWe do have those. It sounds more like you're saying that we shouldn't have vertically integrated companies.
- amelius 3y agoThe vertical integration allows them to corner the market and destroy the competition or prevent them from entering the market, which is bad for the consumer and bad for society, eventually.
- rootusrootus 3y agoIf we're making bets on OpenAI vs Nvidia in the hardware space, I know who I'm picking.
- icapybara 3y agoThat's a broad claim, you have to prove your case. You're asking for a ban on vertical integration as a business strategy.
- notaustinpowers 3y agoVertical integration is already under scrutiny by the antitrust folks. Even more so now after everything Google was able to get away with. OpenAI wanting to vertically merge to make their own AI chips may seem harmless enough (it's a good business move, we can cut expenses)! But we can't forget that Sam Altman just a few months ago told Congress he supports making an organization that companies need permission from to being creating/utilizing advanced AI systems. And he's such a kind man he's willing to lead that organization himself. Obviously someone integrating the chips to train AI, and having the ability to approve/deny his own competition is a huge red flag.
- Oras 3y agoIt means Microsoft is involved too. From consumer perspective, following the performance of Apple silicon, I am excited for this news.
- wing-_-nuts 3y agoI said a while back that I expect the major cloud vendors (Azure, AWS, GCP, etc) to start trying to develop their own chips for AI work. Google already does to some extent with their tpus. At the very least, this is saber rattling trying to convince nvidia to lower prices.
- seydor 3y agothey should also buy reddit
- drexlspivey 3y agoMasayoshi Son is funding them so you know they’re gonna lose a ton of money
- deleted 3y ago[deleted]
- n40487171 3y agoI was just thinking the inference cost could be reduced by making hardware with less error correction in specific areas to get higher density, and let the NN work around the limitations.
- btbuildem 3y agoThis makes a lot of sense. When ChatGPT initially broke the mold, I was hoping someone would find a way to repurpose all the silicon the crypto-bros' nonsense has commandeered -- alas, the problems are too different. Making specialized chips to run LLMs is the logical next step.
- trident5000 3y agoThe economy is in an interesting place when in house chip making efforts or startups are popping up now. It used to be a task like landing on the moon. Still a difficult effort but it looks like this industry is expanding. It speaks to the rapid nature of technology in general where insurmountable tasks over time become closer to trivial.
- aportnoy 3y agoNote Sam Altman is a Cerebras investor.
- the-dude 3y agoThe first valuable comment imho, thanks! Let me add : there are numerous other AI chip startups.
- asciimike 3y agoFrom the last thread on this (https://news.ycombinator.com/item?id=32610780 https://news.ycombinator.com/item?id=32610780): - https://sambanova.ai/ https://sambanova.ai/ (Enterprise AI and dataflow-as-a-service for established models) - https://www.cerebras.net/ https://www.cerebras.net/ (AI accelerator, trying to compete with Nvidia) - https://www.graphcore.ai/ https://www.graphcore.ai/ (Another AI accelerator company, UK based) - https://femtosense.ai/ https://femtosense.ai/ (Sparse NNs on very low power chips, cool hardware and software challenges) - https://sima.ai/ https://sima.ai/ (ML accelerators for embedded applications) - https://ambiq.com/ https://ambiq.com/ (Not AI, but low power chips for wireless using some fancy tech that reduces energy leakage) - https://www.esperanto.ai/ https://www.esperanto.ai/ (RISC-V based Tensor computes chip, founded by Intel Hybrid Parallel Computing Vice President Dave Ditzel) - https://www.furiosa.ai/ https://www.furiosa.ai/ (AI accelerator company which show good results in MLPerf benchmark) - https://groq.com/ https://groq.com/ (From the team that built the original TPU at Google) - https://lightmatter.co/ https://lightmatter.co/ (Light tubes instead of copper) - https://www.untether.ai/ https://www.untether.ai/
- rvz 3y agoExactly. We are now starting to slowly realize... [0] [0] https://news.ycombinator.com/item?id=35490837 https://news.ycombinator.com/item?id=35490837
- hef19898 3y agoAh, so that is how Sam and Co. cash out on the 10 billion from MS!
- hamilyon2 3y agoI think that endgame for generative language models is models embedded directly into chips. Computers that run english language instead of machine code and for which CPU, GPU and what is currently known as PC is more like a peripheral IO device.
- dragonwriter 3y ago> I think that endgame for generative language models is models embedded directly into chips. Computers that run english language instead of machine code and for which CPU, GPU and what is currently known as PC is more like a peripheral IO device. That may be the endgame, but I think if it is there is a long time before attempts to jump to it aren't going to fail like every high-level-system-in-hardware for other than very niche applications, because general purpose (comparatively, even if specialized for running AI models) hardware will be good enough that the value of being able to upgrade the models it is running will outweigh any marginal temporary edge that current-models-in-hardware have.
- intrasight 3y agoSuch a model could not be updated nor learn. Even very simple biological organisms can learn. The end-game is growing brains ;) (only partially kidding)
- dragonwriter 3y ago> Such a model could not be updated nor learn. It might have some limited ability for updates if the hardware had the model code but the weights were in memory that was updatable. It might do in-context learning even without that.
- intrasight 3y ago> weights in memory That's basically how our neurons work. New neuron growth and connection isn't much of a factor in learning. Rather it's the synaptic restructuring (equivalent to AI model weights) that change relatively quickly. So we need to figure how how to "grow" mechanical brains. I envision this being done with a new generation of FPGAs tailored to this task.
- jedberg 3y agoMy guess is that this leak is in response to the announcement from Anthropic that they will be using Amazon's custom AI silicon.
- mwbajor 3y agoAs soon as the investors and boardmembers realize that chips mean "hardware design" they will quickly put an end to any of these efforts.
- hskalin 3y agoWhy so?
- simne 3y agoThis is not answering question, on what OpenAI will earn money. As even using only Nvidia chips, industry need somewhere gather ~$1000B (half of this is OpenAI share now), but with custom chips will be at least 3-4 times larger numbers.