3 ms·

Even Microsoft knows that there is no moat in AI models given that it is fundamentally a race to zero. The difference is, they finally realized that after Appl

by rvz 2y ago

Even Microsoft knows that there is no moat in AI models given that it is fundamentally a race to zero. The difference is, they finally realized that after Apple almost invested but stepped back. Perhaps they know that this hype is a repeat of being in the middle of the year 1999; or even we are close to the start of the year 2000.

devsda 2y ago
Isn't the training data a moat by itself? Due to sneaky website scraping by OpenAI many sites now have their content locked behind login, have their APIs severely restricted and watchout for scraping. Unless you want to build a specialized AI built on specific open or closed training corpus, huge training data is required to build something similar to OpenAI's models. Google with it's crawlers and associated relaxations should not be affected much by those restrictions. Miscrosoft can easily update its browser/OS ToS and say "we'll use all your browsing data for training, anonymously ofcourse" and there's not much users will do except shrug. Is there anyone else who is better positioned to collect the content for training or am I placing too much importance on training data?
- a_bonobo 2y ago
  I agree, it's far more the 'training data' than the model architecture. Given that Windows 11 is such a surveillance/advertisement machine I'm sure they sit on years of user movements MS can use to build a OS-running AI, similar to what Anthropic has been trying with 'computer use'. That's where I see Microsoft's niche in the future, and it's an enormous niche: how many people do 'data entry' as a full-time job who'd be unemployed once an LLM can move data from random sources into an Excel sheet by operating a simulated computer?
- Davidzheng 2y ago
  Disagree, now that there are great open models are available I think there's less need of huge training data--can just post train
est31 2y ago
Companies like Uber live off solely from the "we have a large war chest of money to kill competition" moat model where any competition in some area can be killed easily as long as you are much larger than them and can afford to lose money in the area they are active in: just offer lower prices until they are bankrupt, then raise them again. This model has proven to work in places like retail. Literally how Amazon got big. I think they are trying to use this strategy for AI as well, but at least right now there is so much competition and investment that it's tough to pull off.
UltraSane 2y ago
Training data is very much a moat. Especially training data that can be reliably filtered for LLM output.
arresin 2y ago
Didn’t Apple just announce a 500b investment plan?

by rvz 2y ago

devsda 2y ago

Isn't the training data a moat by itself? Due to sneaky website scraping by OpenAI many sites now have their content locked behind login, have their APIs severely restricted and watchout for scraping. Unless you want to build a specialized AI built on specific open or closed training corpus, huge training data is required to build something similar to OpenAI's models. Google with it's crawlers and associated relaxations should not be affected much by those restrictions. Miscrosoft can easily update its browser/OS ToS and say "we'll use all your browsing data for training, anonymously ofcourse" and there's not much users will do except shrug. Is there anyone else who is better positioned to collect the content for training or am I placing too much importance on training data?

a_bonobo 2y ago
I agree, it's far more the 'training data' than the model architecture. Given that Windows 11 is such a surveillance/advertisement machine I'm sure they sit on years of user movements MS can use to build a OS-running AI, similar to what Anthropic has been trying with 'computer use'. That's where I see Microsoft's niche in the future, and it's an enormous niche: how many people do 'data entry' as a full-time job who'd be unemployed once an LLM can move data from random sources into an Excel sheet by operating a simulated computer?
Davidzheng 2y ago
Disagree, now that there are great open models are available I think there's less need of huge training data--can just post train

est31 2y ago

Companies like Uber live off solely from the "we have a large war chest of money to kill competition" moat model where any competition in some area can be killed easily as long as you are much larger than them and can afford to lose money in the area they are active in: just offer lower prices until they are bankrupt, then raise them again. This model has proven to work in places like retail. Literally how Amazon got big. I think they are trying to use this strategy for AI as well, but at least right now there is so much competition and investment that it's tough to pull off.

UltraSane 2y ago

Training data is very much a moat. Especially training data that can be reliably filtered for LLM output.

arresin 2y ago

Didn’t Apple just announce a 500b investment plan?