5 ms·

One thing I like about AI tech is that the developing world is really eager to adopt it and many use cases are free or affordable to them. My assistant in the p

by cm2012 10mo ago

One thing I like about AI tech is that the developing world is really eager to adopt it and many use cases are free or affordable to them. My assistant in the phillipines has used it to substantially improve her communications, for instance. Its much less popular in the USA and EU, but thats nice since it gives the developing world a chance to catch up.

embedding-shape 10mo ago
> many use cases are free or affordable to them Because the technology is so fast, efficient and easy to run locally themselves? Or because currently there are remote APIs/UIs that are heavily subsidized by VC money yet the companies behind them are yet to be profitable? I agree that giving the developing world any ladders for catching up is a great thing, but I'm not sure this is that, it just happens to be that companies don't care about profit (yet) so things appear "free or affordable" to them, and when its gonna be time to make things realistic, we'll see how accessible it'll still be.
- ls612 10mo ago
  Inference is probably profitable in a unit economics sense today, there have been multiple back of the envelope calculations this year talking about this. And with multiple high quality open weights models out there I see no reason why competition between hosting providers won't drive price towards marginal cost of inference.
  - embedding-shape 10mo ago
    Sounds sensible, and I agree. But even with you and me making those assumptions/guesses, I still wouldn't claim that current AI tech is making it "free or affordable to them", it's subsidized, cannot really make claims about how affordable or not it is, at least not yet.
    - lukeschlather 10mo ago
      We can be pretty confident that these services are not subsidized. There are dozens of companies offering these services. Pretty much every single company has published open-weights models that you can run yourself. These open models, you could make money selling them for the same prices Google Gemini costs, while renting on-demand GPU instances from Google Cloud. It actually seems very implausible that Google is losing money on their proprietary models hosted on their own infrastructure. And OpenAI knows they have to compete with Google, which owns its own chips, OpenAI isn't going to be selling things at a loss. They cannot win that fight no matter how much Saudi money they get.
      - embedding-shape 10mo ago
        Again, I agree that it sounds plausible, but it doesn't guarantee anything, and the lack of hard data usually indicates things aren't as confidently profitable as you believe. Otherwise the companies would be bragging about it. Probably in the end it'll be profitable for the companies somehow, but exactly how or what the exact prices will be, I don't think anyone know at this point. That's why I'm reserving my "Developing countries can now affordably use AI too" for when that's reality, not based on guesses and assumptions.
        lukeschlather 10mo ago
        Google publishes their profits quarterly, but they only do that because they are required to by law. They would prefer people assume they're offering these services at a loss so nobody attempts to compete with them. But again, it's not a guess or assumption - you can run the latest DeepSeek model renting GPUs from a cloud provider, and it works, and it's affordable.
      - Imustaskforhelp 10mo ago
        I thought about it and here's my opinion: There are two (three technically) ways that AI can be used. > 1. renting gpu instances per minute from (you mention Google cloud) but I feel like some other providers can be cheaper too since new companies are usually cheaper, We get the lowendhosting of AI nowadays is usually via a marketplace-like thing (vast,runpod,tensordock) Now vast offers serverless per minute AI models so checking it for something like https://vast.ai/model/deepseek-v3.2-exp https://vast.ai/model/deepseek-v3.2-exp or even glm 3.6 basically every of these turns out to be $.30 cents/minute or 18$ per hour As an example GLM 4.6/ (now 4.7) have a YEARLY pricing of around 30 bucks iirc so now compare the immense difference in pricing 2. Using something like openrouter-based pricing :- Then we are basically on the same model of pricing similar to Google Cloud. Of course AI models are reaching frontier and I am cheering for them but I feel like long term/even short term, these are still pretty expensive (even something like openrouter imo) Someone please do genuine maths about this and I can be wrong, I usually am but I expect a 2-3x price (conservative side of things) increase if things arent subsidized These are probably 10s of billions of dollars worth of gpu's so I assume that they would be barely profitable on the current rate but they get around 100s of billions in some cases worth of tokens generations so they can probably work via the 3rd use case I mention Now coming to the third point which I assume is related to the 2nd/1st is that usually, the companies providing these GPU computes provide such compute, usually they can make money via providing by large term contracts. Even huggingface provides consulting services which I think is the biggest profit to them and Another big contender can probably be European GPU compute providers who can provide a layer of safety or privacy for EU companies. Now, looks like I had to go to reddit to find some more info but (https://www.reddit.com/r/LocalLLaMA/comments/1msqr0y/basically_are_the_companies_that_expose_llm_as_an/ https://www.reddit.com/r/LocalLLaMA/comments/1msqr0y/basical...), checking appenz's comment which I might add here (the relevant parts) The large labs (OpenAI, Anthropic) and Hyperscalers (Google, Meta) currently are not trying to be profitable with AI as they are trying to capture market share. They may not even try to have positive gross margins, although the massive scale limits how much they can use per inference operation. Pure inference hosters (Together, Fireworks etc.) have less capital and are probably close to zero gross margins. There are a few things that make all of this more complicated to account for. How do you depreciate GPUs (I have seen 3 years to 8 years), how do you allocate cost if you do inference during the day and train at night etc. The challenge with doing this yourself is that the market is extremely competitive. You need massive scale (as parallelism massively reduces cost), you need to be very good in negotiating cheap compute capacity and you need to be cost-effective in your G2M. Opinions are my own, and none of this is based on non-public information. So basically all of these are probably running in zero/net negative turns and they require billions of $'s to be spent and virtually there isn't any moat/lock-in (and neither there has to be) TLDR: no company right now is sustainable The only use case I can see is probably consulting but that will go as https://www.investopedia.com/why-ai-companies-struggle-financially-11795162 https://www.investopedia.com/why-ai-companies-struggle-finan... So I guess the only reasonable business feels to me is private AI for large businesses who genuinely need it for their business (once again the MIT study applies) but that usually wouldn't apply to us normal grade consumers anyway and would be actually really expensive but still private and would be so far off from us normal people. TLDR: The only ones making money are/ are gonna be B2B but even those are gonna dwindle if the AI bubble bursts because imagine an large business trying to explain why its gonna use AI if 1) the MIT study shows its unprofitable and 2) the fear around using AI etc. and all the financial consequences that the bubble's explosion might cause So that all being said, I doubt it. I think that these prices are only till the bubble lasts which is only as strong as its weakest link which is openAI right now with trillions promised and a net lose making company whose CEO said that AI market is in a bubble and whose CFO openly floats the idea that OpenAI should be bailed out by the US govt if need be So yeah..... Honestly Even local grade gpu's are expensive but with the innovations of open weights models, I feel like they would be the way to go for 90% of basic use cases being run inside them and probably there are very few cases of moat (and I doubt the moat existing in the first place)
  - Imustaskforhelp 10mo ago
    You are forgetting about how with the multiple high quality open weights models, we are gonna quickly/(already have?) reached the point where using completely local models will make sense. If the writer of the (grandparent comment?)/ (the person who wrote about the philipines secretary is reading this), I would love it if you can do me a simple task and instead of having them use the SOTA models for the stuff for which they are using AI right now, they use an open source model (even an tiny to mid model) and see what happens. > "My assistant in the phillipines has used it to substantially improve her communications, for instance." So if they are using it for communications, Honestly even a small-mid model would be good for them. Please let me know how this experiment goes. I might write about it and its just plain curiosity to me but I would honestly be 99% certain that the differences would be so negligible that using SOTA or even remotely hosted AI datacenter models wont make much sense but of course we can say nothing without empirical evidence which is why I also asked you to conduct my hypothesis.
    - embedding-shape 10mo ago
      > You are forgetting about how with the multiple high quality open weights models, we are gonna quickly/(already have?) reached the point where using completely local models will make sense. I'm not, since I'm a heavy user of local models myself, and even with the beast of a card I work with locally daily (RTX Pro 6000), the LLMs you can run locally are basically toy models compared to the hosted ones. I think, if you haven't already, you need to give it a try yourself to see the difference. I didn't mention or address it, as it's basically irrelevant because of the context. And besides that, how affordable how GPUs today in the developing world? Electricity costs? How to deal with thermals? Frequent black outs? And so on, many variables you seemingly haven't considered yet. Best way of making the difference between hosted models and local modals is to run your own private benchmarks against both of them and compare. Been doing this for years, and still local models are nowhere near the hosted ones, sadly. I'm eager for the day to come, but it will still take a while.
      - seanmcdirmid 10mo ago
        I’ve got a Max M3 with 64 GB ram and can run more than just toy models, even if they are obviously less than hosted ones. Honestly, I think local LLMs are the future and we are just going to be doing hosted until hardware catches up (and now they have something to catch up to!).
- cm2012 10mo ago
  You can get 90% of the value with old LLM models that cost the companies very little to run.
- skeledrew 10mo ago
  I don't think there's much room for providers to make the tech substantially less affordable. In order to do that, the one(s) starting the cost increase will need to provide a distinctly better service than those that remain decently affordable. And somehow put a moat around that offering, which is essentially impossible with the amount of players in the field.
  - intended 10mo ago
    The number of players in the field is sufficient to coordinate prices effectively. You can also introduce ads to the model. Amazon eventually started selling Amazon versions of popular cheap items, I can see the GenAI platforms doing the same. Meme bots for example.

by cm2012 10mo ago

embedding-shape 10mo ago

> many use cases are free or affordable to them Because the technology is so fast, efficient and easy to run locally themselves? Or because currently there are remote APIs/UIs that are heavily subsidized by VC money yet the companies behind them are yet to be profitable? I agree that giving the developing world any ladders for catching up is a great thing, but I'm not sure this is that, it just happens to be that companies don't care about profit (yet) so things appear "free or affordable" to them, and when its gonna be time to make things realistic, we'll see how accessible it'll still be.

ls612 10mo ago
Inference is probably profitable in a unit economics sense today, there have been multiple back of the envelope calculations this year talking about this. And with multiple high quality open weights models out there I see no reason why competition between hosting providers won't drive price towards marginal cost of inference.
- embedding-shape 10mo ago
  Sounds sensible, and I agree. But even with you and me making those assumptions/guesses, I still wouldn't claim that current AI tech is making it "free or affordable to them", it's subsidized, cannot really make claims about how affordable or not it is, at least not yet.
  - lukeschlather 10mo ago
    We can be pretty confident that these services are not subsidized. There are dozens of companies offering these services. Pretty much every single company has published open-weights models that you can run yourself. These open models, you could make money selling them for the same prices Google Gemini costs, while renting on-demand GPU instances from Google Cloud. It actually seems very implausible that Google is losing money on their proprietary models hosted on their own infrastructure. And OpenAI knows they have to compete with Google, which owns its own chips, OpenAI isn't going to be selling things at a loss. They cannot win that fight no matter how much Saudi money they get.
    - embedding-shape 10mo ago
      Again, I agree that it sounds plausible, but it doesn't guarantee anything, and the lack of hard data usually indicates things aren't as confidently profitable as you believe. Otherwise the companies would be bragging about it. Probably in the end it'll be profitable for the companies somehow, but exactly how or what the exact prices will be, I don't think anyone know at this point. That's why I'm reserving my "Developing countries can now affordably use AI too" for when that's reality, not based on guesses and assumptions.
      - lukeschlather 10mo ago
        Google publishes their profits quarterly, but they only do that because they are required to by law. They would prefer people assume they're offering these services at a loss so nobody attempts to compete with them. But again, it's not a guess or assumption - you can run the latest DeepSeek model renting GPUs from a cloud provider, and it works, and it's affordable.
    - Imustaskforhelp 10mo ago
      I thought about it and here's my opinion: There are two (three technically) ways that AI can be used. > 1. renting gpu instances per minute from (you mention Google cloud) but I feel like some other providers can be cheaper too since new companies are usually cheaper, We get the lowendhosting of AI nowadays is usually via a marketplace-like thing (vast,runpod,tensordock) Now vast offers serverless per minute AI models so checking it for something like https://vast.ai/model/deepseek-v3.2-exp https://vast.ai/model/deepseek-v3.2-exp or even glm 3.6 basically every of these turns out to be $.30 cents/minute or 18$ per hour As an example GLM 4.6/ (now 4.7) have a YEARLY pricing of around 30 bucks iirc so now compare the immense difference in pricing 2. Using something like openrouter-based pricing :- Then we are basically on the same model of pricing similar to Google Cloud. Of course AI models are reaching frontier and I am cheering for them but I feel like long term/even short term, these are still pretty expensive (even something like openrouter imo) Someone please do genuine maths about this and I can be wrong, I usually am but I expect a 2-3x price (conservative side of things) increase if things arent subsidized These are probably 10s of billions of dollars worth of gpu's so I assume that they would be barely profitable on the current rate but they get around 100s of billions in some cases worth of tokens generations so they can probably work via the 3rd use case I mention Now coming to the third point which I assume is related to the 2nd/1st is that usually, the companies providing these GPU computes provide such compute, usually they can make money via providing by large term contracts. Even huggingface provides consulting services which I think is the biggest profit to them and Another big contender can probably be European GPU compute providers who can provide a layer of safety or privacy for EU companies. Now, looks like I had to go to reddit to find some more info but (https://www.reddit.com/r/LocalLLaMA/comments/1msqr0y/basically_are_the_companies_that_expose_llm_as_an/ https://www.reddit.com/r/LocalLLaMA/comments/1msqr0y/basical...), checking appenz's comment which I might add here (the relevant parts) The large labs (OpenAI, Anthropic) and Hyperscalers (Google, Meta) currently are not trying to be profitable with AI as they are trying to capture market share. They may not even try to have positive gross margins, although the massive scale limits how much they can use per inference operation. Pure inference hosters (Together, Fireworks etc.) have less capital and are probably close to zero gross margins. There are a few things that make all of this more complicated to account for. How do you depreciate GPUs (I have seen 3 years to 8 years), how do you allocate cost if you do inference during the day and train at night etc. The challenge with doing this yourself is that the market is extremely competitive. You need massive scale (as parallelism massively reduces cost), you need to be very good in negotiating cheap compute capacity and you need to be cost-effective in your G2M. Opinions are my own, and none of this is based on non-public information. So basically all of these are probably running in zero/net negative turns and they require billions of $'s to be spent and virtually there isn't any moat/lock-in (and neither there has to be) TLDR: no company right now is sustainable The only use case I can see is probably consulting but that will go as https://www.investopedia.com/why-ai-companies-struggle-financially-11795162 https://www.investopedia.com/why-ai-companies-struggle-finan... So I guess the only reasonable business feels to me is private AI for large businesses who genuinely need it for their business (once again the MIT study applies) but that usually wouldn't apply to us normal grade consumers anyway and would be actually really expensive but still private and would be so far off from us normal people. TLDR: The only ones making money are/ are gonna be B2B but even those are gonna dwindle if the AI bubble bursts because imagine an large business trying to explain why its gonna use AI if 1) the MIT study shows its unprofitable and 2) the fear around using AI etc. and all the financial consequences that the bubble's explosion might cause So that all being said, I doubt it. I think that these prices are only till the bubble lasts which is only as strong as its weakest link which is openAI right now with trillions promised and a net lose making company whose CEO said that AI market is in a bubble and whose CFO openly floats the idea that OpenAI should be bailed out by the US govt if need be So yeah..... Honestly Even local grade gpu's are expensive but with the innovations of open weights models, I feel like they would be the way to go for 90% of basic use cases being run inside them and probably there are very few cases of moat (and I doubt the moat existing in the first place)
- Imustaskforhelp 10mo ago
  You are forgetting about how with the multiple high quality open weights models, we are gonna quickly/(already have?) reached the point where using completely local models will make sense. If the writer of the (grandparent comment?)/ (the person who wrote about the philipines secretary is reading this), I would love it if you can do me a simple task and instead of having them use the SOTA models for the stuff for which they are using AI right now, they use an open source model (even an tiny to mid model) and see what happens. > "My assistant in the phillipines has used it to substantially improve her communications, for instance." So if they are using it for communications, Honestly even a small-mid model would be good for them. Please let me know how this experiment goes. I might write about it and its just plain curiosity to me but I would honestly be 99% certain that the differences would be so negligible that using SOTA or even remotely hosted AI datacenter models wont make much sense but of course we can say nothing without empirical evidence which is why I also asked you to conduct my hypothesis.
  - embedding-shape 10mo ago
    > You are forgetting about how with the multiple high quality open weights models, we are gonna quickly/(already have?) reached the point where using completely local models will make sense. I'm not, since I'm a heavy user of local models myself, and even with the beast of a card I work with locally daily (RTX Pro 6000), the LLMs you can run locally are basically toy models compared to the hosted ones. I think, if you haven't already, you need to give it a try yourself to see the difference. I didn't mention or address it, as it's basically irrelevant because of the context. And besides that, how affordable how GPUs today in the developing world? Electricity costs? How to deal with thermals? Frequent black outs? And so on, many variables you seemingly haven't considered yet. Best way of making the difference between hosted models and local modals is to run your own private benchmarks against both of them and compare. Been doing this for years, and still local models are nowhere near the hosted ones, sadly. I'm eager for the day to come, but it will still take a while.
    - seanmcdirmid 10mo ago
      I’ve got a Max M3 with 64 GB ram and can run more than just toy models, even if they are obviously less than hosted ones. Honestly, I think local LLMs are the future and we are just going to be doing hosted until hardware catches up (and now they have something to catch up to!).
cm2012 10mo ago
You can get 90% of the value with old LLM models that cost the companies very little to run.
skeledrew 10mo ago
I don't think there's much room for providers to make the tech substantially less affordable. In order to do that, the one(s) starting the cost increase will need to provide a distinctly better service than those that remain decently affordable. And somehow put a moat around that offering, which is essentially impossible with the amount of players in the field.
- intended 10mo ago
  The number of players in the field is sufficient to coordinate prices effectively. You can also introduce ads to the model. Amazon eventually started selling Amazon versions of popular cheap items, I can see the GenAI platforms doing the same. Meme bots for example.