3 ms·

> LLMs in the hot path. LLMs are expensive, a hell of a lot more expensive than executing some Javascript locally 1. The costs are not that high 2. They are g

> LLMs in the hot path. LLMs are expensive, a hell of a lot more expensive than executing some Javascript locally 1. The costs are not that high 2. They are going down constantly 3. The ease of using LLM's and the personalisation will result in more interactions which means companies don't mind subsidising This is like saying Amazon e-commerce can't work because logistics are expensive and its hard to pay for a delivery person to come to your doorstep.

danpalmer 7mo ago
Look I'm sure it'll happen in some places, it's just not going to be a major shift in the way this post describes. If the table stakes for using my API is a local GPU to build the interface, my competitors who used their GPUs once to create the interface for their customers will win. If the API getting started guide involves going to a user-contributed list of prompts to put together a set of things I want in the interface, the competitor who doesn't have that step and provides a default interface will win. Default interfaces being provided is not going to change, and the universal truth of defaults is that most people stick with them. Power users modifying their interfaces has always been a thing and is easier with LLMs, but it's going to remain niche, as in, something that power users/hobbyists do, or companies might provide their own internal UI to some external API, but again that already happens extensively. Amazon is the wrong analogy I think, because delivery is in some ways cheaper than every individual going to the store themselves, storing in warehouses is cheaper than storing in stores. In fact in some ways the Amazon analogy better fits the other way around. Not a perfect fit.