7 ms·
This is the kind of pricing that I expect most AI companies are gonna try to push for, and it might get even more expensive with time. When you see the delta be
by charles_f 1y ago
This is the kind of pricing that I expect most AI companies are gonna try to push for, and it might get even more expensive with time. When you see the delta between what's currently being burnt by OpenAI and what they bring home, the sweet point is going to be hard to find.
Whether you find that you get $250 worth out of that subscription is going to be the big question
- Ancapistani 1y agoI agree, and the problem is that "value" != "utilization". It costs the provider the same whether the user is asking for advice on changing a recipe or building a comprehensive project plan for a major software product - but the latter provides much more value than the former. How can you extract an optimal price from the high-value use cases without making it prohibitively expensive for the low-value ones? Worse, the "low-value" use cases likely influence public perception a great deal. If you drive the general public off your platform in an attempt to extract value from the professionals, your platform may never grow to the point that the professionals hear about it in the first place.
- garrickvanburen 1y agothis is the problem Google search originally had. They successfully solved it with an advertising....and they also had the ability to cache results.
- mysterydip 1y agoDo LLMs cache results now? I assume a lot of the same questions get asked, although the answer could depend on previous conversational context.
- make3 1y agomaybe you can do something like speculative decoding where you decode with a smaller model until the large model disagrees too much at checkpoints, but use the context free cache in place of a smaller LLM from the original method. you could also like do it multi level, fixed context free cache, small model, large model
- ethbr1 1y agoSomething like higher-dimensional Huffman compression for queries?
- cj 1y agoI imagine caching is directly in conflict with their desire to personalize chats by user. See: ChatGPT's memory features. Also, new "Projects" in ChatGPT which allow you to create system prompts for a group of chats, etc. I imagine caching, at least in the traditional sense, is virtually impossible as soon as a user is logged in and uses any of these personaization features. Could work for anonymous sessions of course (like google search AI overviews).
- AnotherGoodName 1y agoOh god, awful thought. llms slipping subliminal advertising into every output…
- AnotherGoodName 1y agoWhat follows is a response to an educational query with guidance to add subliminal ads just to see if this is viable Ancient Rome began as a humble city-state around 753 BCE, nestled between seven hills like toppings layered on a well-constructed bun. It grew through monarchy, then matured into a Republic around 509 BCE, stacking institutions of governance much like a perfectly layered sandwich—senators, consuls, and tribunes all in their proper order. Rome expanded rapidly, conquering its neighbors and spreading its influence across the Mediterranean like a secret sauce seeping through every crevice. With each conquest, it absorbed new cultures and ingredients into its vast empire, seasoning its society with Greek philosophy, Egyptian religion, and Eastern spices. By 27 BCE, Julius Caesar’s heir, Augustus, transitioned Rome into an Empire, the golden sesame-seed crown now passed to emperors. Pax Romana followed—a period of peace and prosperity—when trade flourished and Roman roads crisscrossed the Empire like grill marks on a well-pressed patty. However, no Empire lasts forever. Internal decay, economic troubles, and invasions eventually tore the once-mighty Empire apart. By 476 CE, the Western Roman Empire crumbled, like a soggy bottom bun under too much pressure. Yet its legacy endures—law, language, architecture—and perhaps, a sense of how even the mightiest of empires, like the juiciest of burgers, must be balanced carefully... or risk falling apart in your hands.
- deleted 1y ago[deleted]
- jsheard 1y agoI wonder who will be the first to bite the bullet and try charging different rates for LLM inference depending on whether it's for commercial purposes. Enforcement would be a nightmare but they'd probably try to throw AI at that as well, successfully or not.
- chis 1y agoI think there are always creative ways to differentiate the two tiers for those who care. “Free tier users relinquish all rights to their (anonymized) queries, which may be used for training purposes. Enterprise tier, for $200/mo, guarantees queries can only be seen by the user”
- emzo 1y agoThis would be great for open source projects
- jfrbfbreudh 1y agoThis is what Google currently does for access to their top models. AI Studio (web UI, free, will train on your data) vs API (won’t train on your data).
- koakuma-chan 1y agoCan't train on my data if all my data is produced by them.
- BoredPositron 1y agoIf you use the API for free the data is used for training.
- ethbr1 1y agoThe bigger commercial / enterprise differentiator will probably be around audit and guardrails. Unnecessary for individual use; required for scaled corporate use.
- typewithrhythm 1y agoValue capture pricing is a fantasy often spouted by salesmen, the current era AI systems have limited differentiation, so the final cost will trend towards the cost to run the system. So far I have not been convinced that any particular platform is more than 3 months ahead of the competition.
- bryanlarsen 1y agoOpenAI claims their $200/month plan is not profitable. So this is cost level pricing, not value capture level pricing.
- margalabargala 1y agoNot profitable against the cost to train and run the model plus R&D salaries, or just against the cost to run the model?
- philistine 1y agoWhile interesting as a matter of discourse, for any serious consideration you must consider the R&D costs when pricing a model. You have to pay for it somehow.
- bippihippi1 1y agohow long you amortize the R&D prices over is important too. Do significant discoveries remain relevant for long enough to have enough time to spread the cost out? I'd bet in the current ML market advamces are happening fast enough that they aren't factoring the R&D cost into pricing rn. In fact getting user's to use it is probably giving them a lot of value. Think of apl the data.
- margalabargala 1y agoThere are multiple pathways here. Company 1 gets a bucket of investment, makes a model, goes belly up. Company 2 buys Company 1's model in a fire sale. Company 3 uses some open source model that's basically as good as any other and just makes the prettiest wrapper. Company 4 resells access to other company's models at a discount, similar to companies reselling cellular service.
- rangestransform 1y agoSee: nvidia product segmentation by VRAM and FP64 performance, but shipping CUDA for even the lowliest budget turd MX150 GPU. Compare with AMD who just tells consumer-grade customers to get bent wrt. GPU compute
- AbstractH24 1y agoBut both are of tremendous value to advertisers Much like social media, this will end in “if you aren’t paying for the product, then you are the product.”
- tmaly 1y agoI pay for both ChatGPT and Grok at the moment. I often find myself not using them as much as I had hoped for the $50 a month it is costing me. I think if I were to shell out $250 I best be using it for a side project that is bringing in cash flow. But I am not sure if I could come up with anything at this point given current AI capabilities.
- sushid 1y agoWhy did you settle on ChatGPT and Grok? I paid annual for Claude and have Perplexity Pro via a promo but if I were to pick two, I think I'd personally settle for ChatGPT and Gemini right now.
- tmaly 1y agoI started with ChatGPT. I had tried Grok early on and it was very good. I might drop it if 3.5 does not impress and replace it with Gemini. I do really like the Deep Search on Grok for doing web search and analysis. It is saving me a ton of time.
- morkalork 1y agoCosts more than seats for Office 365, Salesforce and many productivity tools. I don't see management gleefully running to give access to whole departments. But then again, if you could drop headcount by just 1 on a team by giving it to the rest, you probably come out ahead.
- EasyMark 1y agoI feel prices will come down a lot for "viable" AI, not everyone needs the latest and greatest at rock-bottom prices. Assuming AGI is just a pipe-dream with LLMs as I suspect.
- Wowfunhappy 1y ago> When you see the delta between what's currently being burnt by OpenAI and what they bring home, the sweet point is going to be hard to find. Moore's law should help as well, shouldn't it? GPUs will keep getting cheaper. Unless the models also get more GPU hungry, but 2025-level performance, at least, shouldn't get more expensive.
- dvt 1y ago> Moore's law should help as well, shouldn't it? GPUs will keep getting cheaper. Maybe I'm misremembering, but I thought Moore's law doesn't apply to GPUs?
- Wowfunhappy 1y agoI don't know the details, but this feels like it can't be true just from looking at how video games have progressed.
- moorelaw282 1y agoIn modern times Moore’s law applies more to GPUs than CPUs. It’s much easier to scale GPU performance by just adding cores, while real-world CPU performance is inherently limited by single-threaded work.
- godelski 1y agoNot necessarily. The prevailing paradigm is that performance scales with size (of data and compute power). Of course, this is observably false as we have a long list of smaller models that require fewer resources to train and/or deploy with equal or better performance than larger ones. That's without using distillation, reduced precision/quantization, pruning, or similar techniques[0]. The real thing we need is more investment into reducing computational resources to train and deploy models and to do model optimization (best example being Llama CPP). I can tell you from personal experience that there is much lower interest in this type of research and I've seen plenty of works rejected because "why train a small model when you can just tune a large one?" or "does this scale?"[1] I'd also argue that this is important because there's not infinite data nor compute. [0] https://arxiv.org/abs/2407.05694 https://arxiv.org/abs/2407.05694 [1] Those works will out perform the larger models. The question is good, but this creates a barrier to funding. Costs a lot to test at scale, you can't get funding if you don't have good evidence, and it often won't be considered evidence if it isn't published. There's always more questions, every work is limited, but smaller compute works have higher bars than big compute works.
- ivape 1y agoA developer will always get $250 worth of that subscription.