5 ms·
What do you mean by hard to create a competitive moat around the tech? These LLMs are super expensive to train so isn’t that a moat, or am I understanding it di
by alexchantavy 4y ago
What do you mean by hard to create a competitive moat around the tech? These LLMs are super expensive to train so isn’t that a moat, or am I understanding it differently?
- sebzim4500 4y agoKind of? $5M to train a new LLM from scratch is not much of a moat even now we've left the 0 interest period, the question is will people start spending much more on networks in the future. (I'm sure they will spend more, but will there ever be a billion dollar network? idk) Certainly there are many industries that are impossible to enter with this than 100x that amount of money, either because of regulations or engineering challenges (or both).
- hedora 4y agoI’d assume the each member of the fortune 500 will train their own once they have a plausible use case. For instance, Domino’s Pizza has been in this space for at least five years. (The idea is to mount a camera on the egress size of conveyor oven, and have the AI grade each pizza in real time.)
- sroussey 4y agoRight now, sure. But we are in the exponential upturn right now that will moderate into an S curve where the incremental improvements will slow and you will have a convergence of the large language models. Using mostly the same data will lead to mostly the same models. At which point, once they leak, what difference does it make how expensive it is to create?
- alexchantavy 4y agoI think there’s a lot of value in having a tailored model trained on data specific to your domain, so self training will be needed for a lot of scenarios. Millions of dollars to train is definitely a higher barrier of entry than say someone writing an app on nights and weekends who just got a bit of seed funding