6 ms·

Kinda interesting to see where the moat is in AI space. Good base models can always distilled when you have access to API. System prompts can get leaked, and UI

Kinda interesting to see where the moat is in AI space. Good base models can always distilled when you have access to API. System prompts can get leaked, and UI tricks can be copied. In the end, the moat might be in the hardware and vertical integration.

deelowe 2y ago
How many times have we been down this path? Tcp/IP, dos/windows, Linux, virtualization, and on and on. Open platforms always seem to find a way to usurp everyone else. In the end, it's better to be a service provider.
- runlevel1 2y ago
  Open source finds a way. Good enough + open (and free) is a very appealing proposition.
suraci 2y ago
ecosystem
tonyhart7 2y ago
there is no open source alternative to GPU farm, that's the moat that's why they can open source their model and be fine because running this shit is actually hard, let alone maintaining SLA for millions of users??
- randomvariable 2y ago
  How long until laptops are able to run high end models? What's the use case that requires a server farm for end user's?
  - tonyhart7 2y ago
    maybe next 5 - 10 years??? but even then the frontier would be push further and people would get used to lets say 10 trillion model cloud host and using 600B model would feel stupid
vineyardmike 2y ago
> the moat might be in the hardware and vertical integration. The moat is the products that can be built. The moat is always the product - because a differentiated product can't be a commodity. And an LLM is not a product. Google and MSFT and Meta have already "won" because they have profitable products they can build LLMs onto. Every other company seems to be burning cash to build a product, and only ChatGPT is getting the brand recognition to realistically compete. Building an LLM is like building a database. Sure a good one unlocks new uses, but consumers aren't buying something for the database. Meanwhile enterprise customers will shop around and drive the price of a commodity down while open source alternatives grow from in-house uses to destroy moats. Even hardware isn't a true moat. Only Google has strong vertical integration with their TPUs, and that gives them a lead. BUT Microsoft, AWS, Meta and a whole bunch of startups are building out custom silicon which will surely put pressure on them and Nvidia to keep innovating and earning that price edge.
- dsco 2y ago
  See I kind of buy the database argument but also kind of don't. A database needs an operator whereas a LLM doesn't. You're basically melting the product into a piece of goo and the UI can be approached using natural language. For products that still need a UI you could claim that LLM operators take over, so that's still a tax you pay to the incumbents as you interact with a product. It's sort of like we take the money which was paid to SQL operators and engineers and instead pay it to the hyperscalers.
  - vineyardmike 2y ago
    LLMs absolutely need an operator - who runs the servers and GPUs that hosts the models? Who writes the system prompts? Who fine tunes and trains the models? This can be a big cloud api like AWS, but it can also be a custom-in-house service for a company. Users of LLMs don’t quite have an equivalent employee to a DBA, but neither do most customers of AWS DynamoDB or RDS or whatever. Many use cases of LLMs won’t be chat bots like ChatGPT. They’re be tools for automated summarizations, classifications, etc. They’ll be automated assistance and basic tool calling, etc. They’ll perform OCR and documentation analysis. Automated translations etc.
rfoo 2y ago
Why do we need a moat?
- lompad 2y ago
  _We_ don't. Investors do. Because without being able to gatekeep the rest of the world, there is little money in LLMs.
  - rfoo 2y ago
    Indeed. I guess investors should stop pouring money into LLMs, then. Just like how they don't pour money into pure mathematics.
- FergusArgyll 2y ago
  So a company can make enough money to fund the next breakthrough/training run
sumedh 2y ago
> Good base models can always distilled when you have access to API. What does that mean?
- mptest 2y ago
  You can use the outputs of a closed source model (or deepseek -> llama. see llama 70b deepseek distilled) to create a synthetic training data set which lets you fine tune (distill) most of the benefits of the "smarter" model in to a "dumber" model. This is why openAi does not show the actual full chain of thought but a summarized version. To stop exfiltration of their IP which has proven immensely difficult.* *disclaimer; i am an expert of nothing
panny 2y ago
>Kinda interesting to see where the moat is in AI space. Where we're going, we don't need moats.

deelowe 2y ago

How many times have we been down this path? Tcp/IP, dos/windows, Linux, virtualization, and on and on. Open platforms always seem to find a way to usurp everyone else. In the end, it's better to be a service provider.

runlevel1 2y ago
Open source finds a way. Good enough + open (and free) is a very appealing proposition.

suraci 2y ago

ecosystem

tonyhart7 2y ago

there is no open source alternative to GPU farm, that's the moat that's why they can open source their model and be fine because running this shit is actually hard, let alone maintaining SLA for millions of users??

randomvariable 2y ago
How long until laptops are able to run high end models? What's the use case that requires a server farm for end user's?
- tonyhart7 2y ago
  maybe next 5 - 10 years??? but even then the frontier would be push further and people would get used to lets say 10 trillion model cloud host and using 600B model would feel stupid

vineyardmike 2y ago

> the moat might be in the hardware and vertical integration. The moat is the products that can be built. The moat is always the product - because a differentiated product can't be a commodity. And an LLM is not a product. Google and MSFT and Meta have already "won" because they have profitable products they can build LLMs onto. Every other company seems to be burning cash to build a product, and only ChatGPT is getting the brand recognition to realistically compete. Building an LLM is like building a database. Sure a good one unlocks new uses, but consumers aren't buying something for the database. Meanwhile enterprise customers will shop around and drive the price of a commodity down while open source alternatives grow from in-house uses to destroy moats. Even hardware isn't a true moat. Only Google has strong vertical integration with their TPUs, and that gives them a lead. BUT Microsoft, AWS, Meta and a whole bunch of startups are building out custom silicon which will surely put pressure on them and Nvidia to keep innovating and earning that price edge.

dsco 2y ago
See I kind of buy the database argument but also kind of don't. A database needs an operator whereas a LLM doesn't. You're basically melting the product into a piece of goo and the UI can be approached using natural language. For products that still need a UI you could claim that LLM operators take over, so that's still a tax you pay to the incumbents as you interact with a product. It's sort of like we take the money which was paid to SQL operators and engineers and instead pay it to the hyperscalers.
- vineyardmike 2y ago
  LLMs absolutely need an operator - who runs the servers and GPUs that hosts the models? Who writes the system prompts? Who fine tunes and trains the models? This can be a big cloud api like AWS, but it can also be a custom-in-house service for a company. Users of LLMs don’t quite have an equivalent employee to a DBA, but neither do most customers of AWS DynamoDB or RDS or whatever. Many use cases of LLMs won’t be chat bots like ChatGPT. They’re be tools for automated summarizations, classifications, etc. They’ll be automated assistance and basic tool calling, etc. They’ll perform OCR and documentation analysis. Automated translations etc.

rfoo 2y ago

Why do we need a moat?

lompad 2y ago
_We_ don't. Investors do. Because without being able to gatekeep the rest of the world, there is little money in LLMs.
- rfoo 2y ago
  Indeed. I guess investors should stop pouring money into LLMs, then. Just like how they don't pour money into pure mathematics.
FergusArgyll 2y ago
So a company can make enough money to fund the next breakthrough/training run

sumedh 2y ago

> Good base models can always distilled when you have access to API. What does that mean?

mptest 2y ago
You can use the outputs of a closed source model (or deepseek -> llama. see llama 70b deepseek distilled) to create a synthetic training data set which lets you fine tune (distill) most of the benefits of the "smarter" model in to a "dumber" model. This is why openAi does not show the actual full chain of thought but a summarized version. To stop exfiltration of their IP which has proven immensely difficult.* *disclaimer; i am an expert of nothing

panny 2y ago

>Kinda interesting to see where the moat is in AI space. Where we're going, we don't need moats.