5 ms·

>Open models... are still behind GPT-4 by some distance but they are closing in at a rate that leads me to believe there’s not much of a moat there Commerciall

by yyyk 3y ago

>Open models... are still behind GPT-4 by some distance but they are closing in at a rate that leads me to believe there’s not much of a moat there Commercially, this doesn't matter even if true. As I keep reminding here: moats are rarely about raw tech. Moats are much more about integrations, brand ("Nobody got fired for buying X"), access to API/raw compute... If OpenAI have a superior Office 365 integration they'll have a de facto moat. If OpenAI have a larger plugin ecosystem they'll have a de facto moat. If OpenAI has access to better compute (that's far from a certainty) they'll have a moat. If it's much easier to use OpenAI than install a local model they'll have a moat, etc. And that's true even if they don't improve their model at all. What will open source offer? Privacy? You can see for how little and how easily people barter that. Moats do fall - but for that, FOSS will have to think product. We're not there at all at the moment. [EDIT: Oh, I didn't notice the author's name. We've had that conversation in the past here. Sorry for being repetitive.]

ithkuil 3y ago
> Open models... are still behind GPT-4 by some distance but they are closing in at a rate that leads me to believe there’s not much of a moat there This assumes that openai internally doesn't also "closing-in" towards something even more impressive to be released next year or whatnot.
l33tman 3y ago
I agree with you on that a good integrator can get successful, but we should not forget here that the underlying code running all of this is in the order of hundreds of lines (see alpaca.cpp for example) in combination with a boatload of money required for training (that is quickly going down), to simplify the argument a bit. It's hardly comparable to the gigantic ecosystem of services and micro-services that your comparison alludes to with IBM and Microsoft. OpenAI is nowhere near such a brand recognition, and grassroots support for it within a company would quickly move to the next free chatgpt-clone that is better or cheaper or faster or more accessible just as what happened with Dalle-2. The value of the models themselves will quickly come down to just a slight bit over the underlying hardware costs and Altman knows this. Microsoft bought themselves a $10B time window to try to do what you're saying, so let's see :) But even for them, when they've built LLM-adaptations to their most popular products, it's fairly simple to just swap it out with something new and more shiny and cheaper that's not OpenAI, and the end customer won't notice as it's the Microsoft or Office brand that they buy into. They are not going to advertise what's inside their products with big banners "Powered by OpenAI" in the long run, I think (do they now?)
- yyyk 3y ago
  What you refer as 'the underlying code' isn't the real underlying code, the underlying code is the model. Your underlying code is just the runtime, which is commercially the least interesting part (which is why it is/will be/ open sourced). Both the model and the integration are way more interesting commercially. I'm sure the code Microsoft has to integrate their assistant to Windows is way more than hundreds of lines. How long until GNOME/KDE even start on something similar? Probably not before 2030. Anyway, I treat OpenAI and Microsoft as two sides of the same coin given level of integration between the two. It's arguable the Microsoft has the upper hand here but OpenAI is their main LLM talent. [EDIT: I don't see MS switching backends from a backend they control, especially when performance apparently is adequate enough already and the real cost isn't licensing the code, but Azure, so open source doesn't necessarily have an advantage here.]
zirgs 3y ago
Open models can talk about "forbidden" topics and can be extended by users. Both of them are significant advantages.
- yyyk 3y ago
  The ability to talk about 'forbidden' topics is also a significant disadvantage. Just wait for the first moral panic targeting open source GPTs. I think that Open Source will exist (barring a legal ban), but the triumphalism is in my mind very unjustified. There's a fair chance closed source will get 99% of this market.
  - zirgs 3y ago
    It's an advantage, because each user can apply different restrictions to Open Source AI models. If I'm writing a crime novel for adults - I want the assistant to be able to talk about sex and violence. But obviously - a corporate customer support chatbot should not talk about those things. I understand why ClosedAI added those restrictions. But they are too inflexible.
api 3y ago
I alluded to what you say in the last paragraph: the average user isn't going to play with open source models, but the existence of the open source AI ecosystem will enable more competitors. There's not going to just be one AI company. There will be thousands and thousands, each addressing different use cases and market niches. In a world where OpenAI has a powerful technological moat, all of these companies would end up having to pay rent to OpenAI. In a world with a strong open source AI ecosystem that's not the case. They can take open source models and even train them themselves and refine them for specific use cases. Winner take all dynamics in general are overstated. They exist in a few niches but not most. How many networking, database, file sync, cloud, gaming, banking, or hosting companies are there? There's even been markets that once looked winner take all like social media that have recently experienced a flurry of diversification. Edit: there's one more reason I'm not sure moats are strong in AI: AI can write code and can process "messy" inputs. One of the thing that strengthens moats built around integrations and such is that the difficulty of doing the integration is part of the barrier. Integrations are frankly annoying and labor intensive to create. With AI you can just tell it to integrate in natural language and schlep messy imperfect data into it. That makes integrations significantly less labor intensive, making it easier for a competitor to pop up and add them very easily and quickly.
- yyyk 3y ago
  I agree OpenAI won't and can't have anywhere near a monopoly, excluding very unlikely legal shenanigans. The field is too approachable, and ultimately, to pinch into their market one doesn't need GPT-4 level performance. Besides, apparently OpenAI can't scale compute fast enough, so users will end up by default requiring other providers. There's a very good chance of OpenAI/Microsoft being major players though IMHO. I can however see a possible future where open source is not going to have any significant impact on LLMs, say like Desktop Linux. Either because it gets stuck in a technical realm and doesn't make anything too approachable to ordinary users, or because it lacks the necessary integrations, or because developers get stuck arguing about the license (raw LLAMA not being good enough due to the non-commercial requirement), or because a moral panic ("4chanGPT is radicalizing people!") leads to a form of legal restrictions that makes open source efforts difficult to sustain. This doesn't have to be, so long as the hacker community can avoid falling into complacency. On integrations, you're thinking about input, but there are still significant challenges there, the output step, API keys, rate limits, various crazy API corners, certifications... LLMs will help, but I expect integration to still be annoying. For example, the Microsoft example where the Assistant changes the system to Dark Mode. You can't use LLM messy input to get that output on a generic level (and if you could, that would risk the LLM as an attack vector). You might be able to use a software development LLM to help write the code to do that specific thing and make it available to the product LLM, but ultimately that's a generic software productivity speedup - which also 'helps' those writing the API to make it more complicated and do more stuff we'll need to implement...

by yyyk 3y ago

ithkuil 3y ago

> Open models... are still behind GPT-4 by some distance but they are closing in at a rate that leads me to believe there’s not much of a moat there This assumes that openai internally doesn't also "closing-in" towards something even more impressive to be released next year or whatnot.

l33tman 3y ago

I agree with you on that a good integrator can get successful, but we should not forget here that the underlying code running all of this is in the order of hundreds of lines (see alpaca.cpp for example) in combination with a boatload of money required for training (that is quickly going down), to simplify the argument a bit. It's hardly comparable to the gigantic ecosystem of services and micro-services that your comparison alludes to with IBM and Microsoft. OpenAI is nowhere near such a brand recognition, and grassroots support for it within a company would quickly move to the next free chatgpt-clone that is better or cheaper or faster or more accessible just as what happened with Dalle-2. The value of the models themselves will quickly come down to just a slight bit over the underlying hardware costs and Altman knows this. Microsoft bought themselves a $10B time window to try to do what you're saying, so let's see :) But even for them, when they've built LLM-adaptations to their most popular products, it's fairly simple to just swap it out with something new and more shiny and cheaper that's not OpenAI, and the end customer won't notice as it's the Microsoft or Office brand that they buy into. They are not going to advertise what's inside their products with big banners "Powered by OpenAI" in the long run, I think (do they now?)

yyyk 3y ago
What you refer as 'the underlying code' isn't the real underlying code, the underlying code is the model. Your underlying code is just the runtime, which is commercially the least interesting part (which is why it is/will be/ open sourced). Both the model and the integration are way more interesting commercially. I'm sure the code Microsoft has to integrate their assistant to Windows is way more than hundreds of lines. How long until GNOME/KDE even start on something similar? Probably not before 2030. Anyway, I treat OpenAI and Microsoft as two sides of the same coin given level of integration between the two. It's arguable the Microsoft has the upper hand here but OpenAI is their main LLM talent. [EDIT: I don't see MS switching backends from a backend they control, especially when performance apparently is adequate enough already and the real cost isn't licensing the code, but Azure, so open source doesn't necessarily have an advantage here.]

zirgs 3y ago

Open models can talk about "forbidden" topics and can be extended by users. Both of them are significant advantages.

yyyk 3y ago
The ability to talk about 'forbidden' topics is also a significant disadvantage. Just wait for the first moral panic targeting open source GPTs. I think that Open Source will exist (barring a legal ban), but the triumphalism is in my mind very unjustified. There's a fair chance closed source will get 99% of this market.
- zirgs 3y ago
  It's an advantage, because each user can apply different restrictions to Open Source AI models. If I'm writing a crime novel for adults - I want the assistant to be able to talk about sex and violence. But obviously - a corporate customer support chatbot should not talk about those things. I understand why ClosedAI added those restrictions. But they are too inflexible.

api 3y ago

I alluded to what you say in the last paragraph: the average user isn't going to play with open source models, but the existence of the open source AI ecosystem will enable more competitors. There's not going to just be one AI company. There will be thousands and thousands, each addressing different use cases and market niches. In a world where OpenAI has a powerful technological moat, all of these companies would end up having to pay rent to OpenAI. In a world with a strong open source AI ecosystem that's not the case. They can take open source models and even train them themselves and refine them for specific use cases. Winner take all dynamics in general are overstated. They exist in a few niches but not most. How many networking, database, file sync, cloud, gaming, banking, or hosting companies are there? There's even been markets that once looked winner take all like social media that have recently experienced a flurry of diversification. Edit: there's one more reason I'm not sure moats are strong in AI: AI can write code and can process "messy" inputs. One of the thing that strengthens moats built around integrations and such is that the difficulty of doing the integration is part of the barrier. Integrations are frankly annoying and labor intensive to create. With AI you can just tell it to integrate in natural language and schlep messy imperfect data into it. That makes integrations significantly less labor intensive, making it easier for a competitor to pop up and add them very easily and quickly.

yyyk 3y ago
I agree OpenAI won't and can't have anywhere near a monopoly, excluding very unlikely legal shenanigans. The field is too approachable, and ultimately, to pinch into their market one doesn't need GPT-4 level performance. Besides, apparently OpenAI can't scale compute fast enough, so users will end up by default requiring other providers. There's a very good chance of OpenAI/Microsoft being major players though IMHO. I can however see a possible future where open source is not going to have any significant impact on LLMs, say like Desktop Linux. Either because it gets stuck in a technical realm and doesn't make anything too approachable to ordinary users, or because it lacks the necessary integrations, or because developers get stuck arguing about the license (raw LLAMA not being good enough due to the non-commercial requirement), or because a moral panic ("4chanGPT is radicalizing people!") leads to a form of legal restrictions that makes open source efforts difficult to sustain. This doesn't have to be, so long as the hacker community can avoid falling into complacency. On integrations, you're thinking about input, but there are still significant challenges there, the output step, API keys, rate limits, various crazy API corners, certifications... LLMs will help, but I expect integration to still be annoying. For example, the Microsoft example where the Assistant changes the system to Dark Mode. You can't use LLM messy input to get that output on a generic level (and if you could, that would risk the LLM as an attack vector). You might be able to use a software development LLM to help write the code to do that specific thing and make it available to the product LLM, but ultimately that's a generic software productivity speedup - which also 'helps' those writing the API to make it more complicated and do more stuff we'll need to implement...