3 ms·
I think OpenAI's moat is a large corpus of well-cleaned training data. There is no brilliant insight behind the model architecture for GPT-4, but rather 100s of
by paddw 3y ago
I think OpenAI's moat is a large corpus of well-cleaned training data. There is no brilliant insight behind the model architecture for GPT-4, but rather 100s of small incremental improvements upon previous model iterations. Eventually, large companies will be able to catch up. Open source should also be able to, in theory, all though it will be hard for things to come together.
- smartbettor 3y agoThat sounds like a moat.
- bluepod4 3y agoYes, the first sentence says so. The rest explains why the water in that moat isn’t very deep, that there’s no alligators in it, and that the competitors are carrying portable bridges.
- deleted 3y ago[deleted]
- philwelch 3y agoThe longer OpenAI’s models are around, the more the internet will be polluted with GPT-generated trash, which means any follow-on competitors will have a harder time building their corpus.
- mejutoco 3y agoThey can use OpenAI's output (like a lot of models already do), or train on a curated subset (books) or other media (movies).