5 ms·
We just had a realization during a demo call the other day: The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go
by keiferski 6mo ago
We just had a realization during a demo call the other day:
The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. Not being dependent on LLMs for your fundamental product’s value will be a major advantage, at least in pricing.
- andersmurphy 6mo agoYup. Also regardless of price they need to spend more and more as the project collapses under the inevitable incidental complexity of 30k lines of code a day. It's similar to how if you know what you're doing you can manage a simple VPS and scale a lot more cost effectively than something like vercel. In a saturated market margins are everything. You can't necessarily afford to be giving all your margins to anthropic and vercel.
- prox 6mo agoI also can’t wait for the time when few know how to code. Just like how many folks don’t know html from css when the homebrew website went away. Their might always be llms, but the dependence is an interesting topic.
- Cthulhu_ 6mo agoLook no further to be honest; look at older generation programming languages like COBOL and how sought-after good developers for that language are. But I'm also afraid / certain that LLMs are able to figure out legacy code (as long as enough fits in their context window), so it's tenuous at best. Also, funny you mentioned HTML / CSS because for a while (...in the 90's / 2000's) it looked like nobody needed to actually learn those because of tools like Dreamweaver / Frontpage.
- prox 6mo agoWell, on the 2nd paragraph, I have no illusion they’ll figure out more as they are being trained. I am more thinking of the custodians (as coders turn into that) Say you are a good coder now, but you are becoming a custodian, checking the llm work will slowly erode your skills. Maybe if you got a good memory or an amazing skillset it might be some time, but if you don’t use it, you lose it.
- raw_anon_1111 6mo agoThe whole “you can make a lot of money programming in COBOL” is one of those myths that needs to die. Even the briefest of Google searches show they make around the same as any other enterprise dev if not slightly less.
- ipaddr 6mo agoCOBOL developers are sought after but still paid less than a grad doing crud. Is that the future?
- Max-q 6mo agoThe issue with COBOL code is that it’s hidden. It’s mostly internal systems so little code available for training. HTML, TypeScript, JavaScript, C, etc, are readily available, billions of code lines.
- solenoid0937 6mo agoHow are COBOL developers "sought after"? That's an oft-repeated but woefully incorrect meme. FAANG new grads make more. If the COBOL devs had upskilled throughout their career they'd be Senior Staff/Principal+ and making 5-10x more than they do today.
- BobbyTables2 6mo agoA time when few know how to code? I think that was about 10 years ago…
- finaard 6mo agoHow is that surprising? We've been taking that into account for any LLM related tooling for over a year now that we either can drop it, or have it designed in a way that we can switch to a selfhosted model when throwing money at hardware would pay for itself quickly. It's just another instance of cloud dependency, and people should've learned something from that over the last two decades.
- keiferski 6mo agoNot so much that it was surprising, rather that we looked at a competitor’s site and noticed that a) their prices went way up and b) their branding changed to be heavily AI-first. So we thought, hmm, “wonder if they are increasing prices to deal with AI costs,” and then projected that into a future where costs go up. We don’t have this dependence ourselves, so this seems to be a competitive advantage for us on pricing.
- michaelbuckbee 6mo agoWhat's weird though is the bifurcation in pricing in the market: aka if your app can function on a non-frontier level AI you can use last years model at a fraction of the cost.
- michaelje 6mo agoAbsolutely. Pricing exposure is the quiet story under all the waves of AI hype. Build for convenience → subsidise for dependence → meter for margin is a well-worn playbook, and AI-dependent companies are about to find out what phase three feels like. Hyperscalers are spending a fortune so we think AI = API, but renting intelligence is a business model, not a technical inevitability. Shameless link to my post on this: https://mjeggleton.com/blog/AIs-mainframe-moment https://mjeggleton.com/blog/AIs-mainframe-moment
- muppetman 6mo agoNo shit. People are just figuring this out now? This is the “Building my entire livelihood on Facebook, oh no what?” all over again. Oh no sorry I forgot, your laptops LLM can draw a potato, let me invest in you.
- lioeters 6mo agoIndeed, it was clear from the beginning, "AI" companies want to become infrastructure and a critical dependency for businesses, so they can capture the market and charge whatever they want. They will have all the capital and data needed to eventually swallow those businesses too, or more likely sell it to anyone who wants the competitive advantage.
- rybosworld 6mo agoSeriously. > We just had a realization during a demo call the other day These tools have been around for years now. As they've improved, dependency on them has grown. How is any organization only just realizing this? That's like only noticing the rising water level once it starts flooding the second floor of the house.
- keiferski 6mo agohttps://news.ycombinator.com/item?id=47804804 https://news.ycombinator.com/item?id=47804804
- strife25 6mo agoMarginal costs matter in this world.
- zozbot234 6mo ago> The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up. It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago. For the very largest models, I think the latter effect dominates quite easily.
- bcjdjsndon 6mo agoThere's only so far engineers can optimise the underlying transformer technique, which is and always has been doing all the heavy lifting in the recent ai boom. It's going to take another genius to move this forward. We might see improvements here and there but the magnitudes of the data and vram requirements I don't think will change significantly
- zozbot234 6mo agoState space models are already being combined with transformers to form new hybrid models. The state-space part of the architecture is weaker in retrieving information from context (can't find a needle in the haystack as context gets longer, the details effectively get compressed away as everything has to fit in a fixed size) but computationally it's quite strong, O(N) not O(N^2).
- aerhardt 6mo agoI’ve read and heard from Semi Analysis and other best-in-class analysts that the amount of software optimizations possible up and down the stack is staggering… How do you explain that capabilities being equal, the cost per token is going down dramatically?
- bcjdjsndon 6mo agoOptimizations, like I said. They'll never hack away the massive memory requirements however, or the pre training... Imagine the memory requirements without the pre training step....this is just part and parcel of the transformer architecture.
- anonyfox 6mo agoin fact I am betting opposite. frontier models are getting not THAT much better anymore at all, for common business needs at least. but the OSS models keep closing the gap. which means if trajectories hold there will be a near future moment probably where the big provider costs suddenly drop shaerply once the first viable local models consistently can take over tasks normally on reasonable hardware. Right now probably frontier providers rush for as much money as they possible can before LLMs become a true commodity for the 80% usecases outside of deep expert areas they will have an edge over as specialist juggernauts (iE a cybersecurity premium model). So its all a house of cards now, and the moment the bubble bursts is when local open inference has closed the gap. looks like chinese and smaller players already go hard into this direction.
- zozbot234 6mo agoLocal open inference can address hardware scarcity by repurposing the existing hardware that users need anyway for their other purposes. But since that hardware is a lot weaker than a proper datacenter setup, it will mostly be useful for running non-time-critical inference as a batch task. Many users will also seek to go local as insurance against rug pulls from the proprietary models side (We're not quite sure if the third-party inference market will grow enough to provide robust competition), but ultimately if you want to make good utilization of your hardware as a single user you'll also be pushed towards mostly running long batch tasks, not realtime chat (except tiny models) or human-assisted coding.
- sevenzero 6mo agoThis was as clear as the sky when the first llm based businesses popped up. How did you realize this only now?
- keiferski 6mo agoReplied here: https://news.ycombinator.com/item?id=47804804 https://news.ycombinator.com/item?id=47804804 And I don't really mean new businesses that are entirely built around LLMs, rather existing ones that pivoted to be LLM-dependent – yet still have non-LLM-dependent competitors.
- sevenzero 6mo agoYea that would've been extremely short sighted from your competitors. Thanks for linking the response!
- bdangubic 6mo agosame as Uber… in the beginning everyone pretty much new that the cost of rides cannot possibly be that cheap and that it is subsudized. once you corner the market etc people just got used to “real” prices to the poibt that now there are often cheaper alternatives than Uber but people still Uber…
- sevenzero 6mo agoIts also quite interesting to read about Uber exploits their drivers and discriminating algorithms. Cory Doctorow mentioned it in his latest book, sadly cant link the direct sources.
- sidewndr46 6mo agoNot really, the next move is to establish standards groups requiring the use of AI in product development. A mix of industry and governmental mandates. What you view are viewing as COGS instead becomes instead a barrier to entry.
- onion2k 6mo agoThe companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up Or they'll price the true cost in from the start, and make massive profits until the VC subsidies end... I know which one I'd do.
- andersmurphy 6mo agoWe don't know what anthropic's true costs are. So pricing that in is at best a guess.
- accrual 6mo ago> Not being dependent on LLMs for your fundamental product’s value I think more specifically not being dependent on someone else's LLM hardware. IMO having OSS models on dedicated hardware could still be plenty viable for many businesses, granted it'll be some time before future OSS reaches today's SOTA models in performance.
- Cthulhu_ 6mo agoThat'll be (part of) the big market correction, but also speaking broadly; as investor money dries up and said investors want to see results, many new businesses or products will realise they're not financially viable. On a small scale that's a tragedy, but there's plenty of analysts that predict an economic crash and recession because there's trillions invested in this technology.
- bjornroberg 6mo agoI wonder if it could be that they won't because the real mechanism is that AI wrapper pricing power is weak (switching costs near zero) but state of the art models makes it difficult to lower prices due to higher cost.
- thih9 6mo agoAlso: AI dependance could be explicit AI API usage by the product itself, but also anything else, like: AI assisted coding, AI used by humans in other surrounding workflows, etc.
- keiferski 6mo agoYeah that's actually what I initially meant: just dependence on AI as a technology, not purely the usage costs. I didn't spell that out well enough.
- lowsong 6mo agoAny company that has become dependant on AI will struggle to survive from here on. By the time many teams realise it'll be too late.