4 ms·

> The cost to build is so low now. The cost seems deceivingly low right now because those AI companies are fighting for monopoly, but in reality the cost is hu

> The cost to build is so low now. The cost seems deceivingly low right now because those AI companies are fighting for monopoly, but in reality the cost is huge – not only capital, but also trust, privacy, and environmental.

priyanmuthu 1y ago
If the concern is about the inference cost -- we do have open-weight models that are getting more powerful, and hardware to run small-ish models cheap. I run agents using small local models in my MacBook.
- manoDev 1y ago
  Yes – but when people say "AI", most often they are talking about the latest OpenAI/Claude development.
  - victorbjorklund 1y ago
    Todays open models are the latest OpenAI/Claude X months ago.
- nine_k 1y ago
  Can you use something like Cursor with your local models? Is the quality comparable? Is the speed acceptable?
  - jvanderbot 1y ago
    Aider!
    - derwiki 1y ago
      +1 Aider, first time I used it I knew I was looking into the future
  - brabel 1y ago
    For quality to be comparable, you need to use a relatively big model, which will only work if you have around 64GB of RAM or more. The latest OpenAI local models (https://openai.com/index/introducing-gpt-oss/ https://openai.com/index/introducing-gpt-oss/), for example, are really good, but you probably want the 120b to have results at least near what you get with their best cloud models, and that requires I think 80GB+. If you don't have that much, you can try stuff like the DeepSeek models, which are known for being ultra-efficient and runnable with "normal" computers, if you don't mind the politics of using that (and there are many models now that are similar!) but I haven't tried too many more to be able to comment. On my Macbook M1 Pro I can run the gpt-oss-20b model without issues and quite fast.
    - bossyTeacher 1y ago
      >if you don't mind the politics of using that what exactly are the "politics" of using DeepSeek? Feels weird to single out DeepSeek like that
      - jakelazaroff 1y ago
        Not sure why parent is being downvoted here. Even without getting into whether it's possible for technology to be apolitical, many AI companies have explicitly political goals. For example, OpenAI's charter is "to ensure that artificial general intelligence benefits all of humanity". They go on to list more specific political goals downstream from that: https://openai.com/charter/ https://openai.com/charter/
        layla5alive 1y ago
        And you believe them?
        jakelazaroff 1y ago
        That's neither here nor there. The point is that it's expressly political, so using ChatGPT is every bit as political as using DeepSeek.
  - overfeed 1y ago
    Aider (cli) and continue.dev (VS Code plugin) can both run with a local(net) Ollama. The qwen-coder models are pretty good and getting better; qwen3-coder is in the ballpark of Sonnet 3.5 for code-synthesis, albeit slower on my hardware.
conradev 1y ago
I care far more about the noise and air pollution that x.ai is causing in Memphis (ruining lives) than the environmental impact of the industry as a whole. 6% YoY growth in domestic electricity demand is frankly nothing compared to the capacity that developing economies are building out for things other than AI.

priyanmuthu 1y ago

If the concern is about the inference cost -- we do have open-weight models that are getting more powerful, and hardware to run small-ish models cheap. I run agents using small local models in my MacBook.

manoDev 1y ago
Yes – but when people say "AI", most often they are talking about the latest OpenAI/Claude development.
- victorbjorklund 1y ago
  Todays open models are the latest OpenAI/Claude X months ago.
nine_k 1y ago
Can you use something like Cursor with your local models? Is the quality comparable? Is the speed acceptable?
- jvanderbot 1y ago
  Aider!
  - derwiki 1y ago
    +1 Aider, first time I used it I knew I was looking into the future
- brabel 1y ago
  For quality to be comparable, you need to use a relatively big model, which will only work if you have around 64GB of RAM or more. The latest OpenAI local models (https://openai.com/index/introducing-gpt-oss/ https://openai.com/index/introducing-gpt-oss/), for example, are really good, but you probably want the 120b to have results at least near what you get with their best cloud models, and that requires I think 80GB+. If you don't have that much, you can try stuff like the DeepSeek models, which are known for being ultra-efficient and runnable with "normal" computers, if you don't mind the politics of using that (and there are many models now that are similar!) but I haven't tried too many more to be able to comment. On my Macbook M1 Pro I can run the gpt-oss-20b model without issues and quite fast.
  - bossyTeacher 1y ago
    >if you don't mind the politics of using that what exactly are the "politics" of using DeepSeek? Feels weird to single out DeepSeek like that
    - jakelazaroff 1y ago
      Not sure why parent is being downvoted here. Even without getting into whether it's possible for technology to be apolitical, many AI companies have explicitly political goals. For example, OpenAI's charter is "to ensure that artificial general intelligence benefits all of humanity". They go on to list more specific political goals downstream from that: https://openai.com/charter/ https://openai.com/charter/
      - layla5alive 1y ago
        And you believe them?
        jakelazaroff 1y ago
        That's neither here nor there. The point is that it's expressly political, so using ChatGPT is every bit as political as using DeepSeek.
- overfeed 1y ago
  Aider (cli) and continue.dev (VS Code plugin) can both run with a local(net) Ollama. The qwen-coder models are pretty good and getting better; qwen3-coder is in the ballpark of Sonnet 3.5 for code-synthesis, albeit slower on my hardware.

conradev 1y ago

I care far more about the noise and air pollution that x.ai is causing in Memphis (ruining lives) than the environmental impact of the industry as a whole. 6% YoY growth in domestic electricity demand is frankly nothing compared to the capacity that developing economies are building out for things other than AI.