10 ms·
I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI
by fnordpiglet 2y ago
I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI
- gunalx 2y agoI Don't know. Being able to get azure credits has payed out really well for openai as a business in constant need of computer.
- fnordpiglet 2y agoWhich is a very short term advantage. And Anthropic gets aws credits which would you rather have?
- Closi 2y agoNever discount the value of short term advantages. Being first at the start (i.e. first mover advantage) is huge.
- luma 2y agoGiven Amazon's no show in the AI space? Azure. By a mile.
- cthalupa 2y agoExcept if you're Anthropic or OpenAI you don't care about what your compute provider has done in the AI space - you care about the compute power they can give you.
- luma 2y agoThat's exactly what I'm talking about: https://i.imgur.com/sZ3tniY.jpeg https://i.imgur.com/sZ3tniY.jpeg
- cthalupa 2y agoBut how many of those are ordered specifically for OpenAI, and are on order as a result of them to begin with? Do you think if we were in a parallel universe where OpenAI ended up partnering with Google or Amazon instead, the GPU shipments would look the same? I think they would reflect wherever OpenAI ended up doing all their compute showing a pretty similar lion's share. Your claim was that people should care about compute based on what the provider has done in the AI space, but Microsoft was pretty far behind on that side until OpenAI - Google was really the only player in town. Should they have wanted GCP credits instead? Do you care about their AI results or the ex post facto GPU shipments? Or, if what you actually want to argue is that Anthropic would be able to get more GPUs with Azure than AWS or GCP then this is a different argument which is going to require different evidence than raw GPU shipments.
- luma 2y agoThe claim being implied was that Anthropic was in a better position because they had partnered with AWS versus Azure and thus they would have more access to GPU. That isn't the case, at all. All I'm stating is what the chart clearly shows - Azure has invested deeply in this technology and at a rate that far exceeds AWS.
- nabla9 2y agoThere are no moats in deep learning, everything changes so fast. They have the next iteration of GPT Sutskever helped to finalize. OpenAI lost it's future unless they find new same caliber people.
- sk11001 2y ago> They have the next iteration of GPT Sutskever helped to finalize How do you know that they have the next GPT? How do you know what Sutskever contributed? (There was talk that the most valuable contributions came from the less well known researchers not from him)
- criddell 2y agoIsn't access to massive datasets and computation the moat? If you and your very talented friends wanted to build something like GPT-4, could you? It's going to get orders of magnitude less expensive, but for now, the capital requirements feel like a pretty deep moat.
- Gud 2y agoHow do you know massive datasets are required? Just because that’s how current LLMs operate, doesn’t mean it’s necessarily the only solution.
- 2y ago
- cm2187 2y agoBut is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).
- benterix 2y agoExtrapolating 2 years might give you a wrong idea, but extrapolating the last year suggests making another leap that was GPT3 or GPT4 is much, much more difficult. The only considerable breakthrough I can think of is Google's huge context window which I hope will be the norm one day, but in terms of actual results they're not mind-blowing yet. We see little improvements everyday and for sure there will be some leaps, but I wouldn't count on a revolution.
- trashtester 2y agoUnlike AI in the past, there is now massive amounts of money going into AI. And the number things humans are still doing significantly better than AI is going down continously now. If something like Q* is provided organically with GPT5 (which may have a different name), and allows proper planning, error correction and direct interaction with tools, that gaps is getting really close to 0.
- varjag 2y agoAI in the past (adjusted for 1980s) was pretty well funded. It's just that fundamental scientific discovery bears little relationship to the pallets of cash.
- mark_l_watson 2y agoFunding in the 1980s was sometimes very good. My company bought me an expensive Lisp Machine in 1982 and after that, even in “AI winters” it mostly seemed that money was available. AI has a certain mystique that helps get money. In the 1980s I was on a DARPA neural network tools advisory panel, and I concurrently wrote a commercial product that included the 12 most common network architectures. That allowed me to step in when a project was failing (a bomb detector we developed for the FAA) that used a linear model, with mediocre results. It was a one day internal consult to provide software for a simple one hidden layer backprop model. During that time I was getting mediocre results using symbolic AI for NLP, but the one success provided runway internally in my company to keep going.
- bamboozled 2y agoThey seem to have a huge "money moat" now. Partnerships with Apple and MS mean they have a LOT of money to try a lot of things I guess. Before the Apple partnership, maybe it seemed like the moat was shrinking, but I'm tno sure now. Likely they have access to a LOT of data now too.
- boringg 2y agoOpenAI most definitely needs the compute from MSFT. It could certainly swap out to another service but given that microsoft invested via credits it would be problematic. They have enmeshed their future.