4 ms·
> The nodes used for training are rented, so that’s opex, right? It’s capex. They are putting money in, and getting an asset out (the weights). > The models a
by JimDabell 1y ago
> The nodes used for training are rented, so that’s opex, right?
It’s capex. They are putting money in, and getting an asset out (the weights).
> The models are in some sense consumable?
Assets depreciate.
- bee_rider 1y agoObsolete software don’t depreciate like obsolete hardware. If an LLM company has trained a truly better model, they can simply make as many copies of their own model as they want. Thus, if the new model is truly better in every way, the old one is completely valueless to them (of course there might be some tradeoffs which mean older models can stick around because they are, say, smaller… but, ultimately they will be valueless after some time). Because models are still being obsoleted every couple years, old models aren’t an asset. They are an R&D byproduct.
- qeternity 1y ago> the old one is completely valueless to them This is of course untrue for the same reason that people are still running Windows 2000.
- bee_rider 1y ago> This is of course untrue for the same reason that people are still running Windows 2000. What is the reason?
- dcre 1y agoThey’ve built processes around it and don’t feel like/can’t afford to/ don’t know to how change them.
- bee_rider 1y agoI guess we’ll see how that shakes out. Because models are getting much better every couple months, I wonder if getting too attached to a process built around one in particular is a bad idea.
- stavros 1y agoI would agree if Windows 2000 had the exact same APIs as the next version, but it doesn't. LLMs are text in -> text out, and you can drop in a new LLM and replace them without changing anything else. If anything, newer LLMs will just have more capabilities.
- qeternity 1y ago> LLMs are text in -> text out, and you can drop in a new LLM and replace them without changing anything else. If anything, newer LLMs will just have more capabilities. I don't mean to be too pointed here, but it doesn't sound like you have built anything at scale with LLMs. They are absolutely not plug n play from a behavior perspective. Yes, there is API compatibility (text in, text out) but that is not what matters. Even frontier SOTA models have their own quirks and specialties.
- msgodel 1y agoWhen I've built things with them I've mostly considered the quirks defects. Kind of like how httpds will have quirks but those aren't really a good thing and they're kind of plug and play.
- stavros 1y agoWhat kind of quirks have you seen that the next model wasn't better at?
- dcre 1y agoA simple example would be when models get better at following instructions, the frantic and somewhat insane-sounding exhortations required to get the crappier model to do what you want can cause the stronger model to be a bit too literal and inflexible.