4 ms·
How Much Will LLM Token Costs Eat into Startups' Margins?
I’ve been speaking with founders about their token costs for using LLMs, and there’s a noticeable anxiety about this issue. While we’re all currently enjoying free credits, once those run out, how much will these costs eat into our margins?
Many founders are hopeful that costs will come down over time, but few are preparing for the inevitable expense. How are you planning for this? Are you considering alternative strategies, such as optimizing usage, exploring open-source models, or investing in your own infrastructure?
Would love to hear your thoughts and experiences. How are you managing or planning to manage LLM token costs? What steps are you taking to ensure these expenses don’t significantly impact your bottom line?
- eschneider 2y agoWhat on earth do 99% of startups even need LLMs for? Seriously.
- conso 2y agoMoney. Besides the obvious, do you think most startups will abandon LLMs or that most "AI startups" are vaporware?
- r2_pilot 2y agoIt's a pretty useful tool, just for rubber ducky-ing, and if you know what you're doing you can get customers just by showing them what capabilities exist(I used Claude 3 just yesterday to demo a proof of concept that got my startup a new customer on the spot).
- PaulHoule 2y agoLately I have been thinking about cost models for software development based on comparable companies: usually the main cost of development is labor and computing costs could be a fraction of that. I mean, a developer who gets paid about $100k might get a MacBook that costs $3k twice a year, I spent about my own yearly salary once to get a production server to host products being developed by 5 or so FTE, for instance. AI threatens to change that, people training their own AI can easily spend many times their pay in cloud costs and similarly you can pay through the nose for ChatGPT.