3 ms·

Hopefully this will put pressure on the market to produce much more efficient AI models. As opposed to bigger, then bigger, and then even BIGGER models (which

by riskable 11mo ago

Hopefully this will put pressure on the market to produce much more efficient AI models. As opposed to bigger, then bigger, and then even BIGGER models (which is the current trend). FYI: gpt-oss:120b is better at coding (in benchmarks and my own anecdotal testing) than gpt5-mini. More importantly, it's so much faster too. We need more of this kind of optimization. Note that gpt5-mini is estimated to be around ~150 billion parameters.

KronisLV 11mo ago
For what it’s worth, even the Qwen 30B model has its use cases. And as far as some of the better open models go, by now the GLM 4.6 355B model is largely better than the Qwen3 Coder 480B variant, so it seems that the models are getting more efficient across the board.
estimator7292 11mo ago
Energy and GPU costs haven't pushed the needle any, so I don't see any reason to expect that RAM costs will.
muldvarp 11mo ago
> We need more of this kind of optimization. Who is the "we" in this sentence? The ultra-rich that don't want to pay white collar workers to build software? The advantages of LLMs are tiny for software engineers (you might be more productive, you don't get paid more) and the downsides are bad to life-altering (you get to review AI slop all day, you lose your job).
- bigbadfeline 11mo ago
  > The ultra-rich that don't want to pay white collar workers to build software? This is already a fact and it's set in stone - making AI cheaper won't change anything in that regard. However, a cheaper AI will allow the laid-off software engineers to use models independently of those firing them, and even compete on an equal footing.
  - muldvarp 11mo ago
    Compete for what exactly? Under the assumption that AI agents will make human software engineering obsolete there won't be a market for you to compete in. Everyone that wants a piece of software will ask their AI agent to create it. My ability to create software is only really useful to me because other people pay me for it. If AI agents take that from me, it won't matter that I can now create awesome software in minutes instead of weeks. The software was never really the thing that was useful for me.
    - bigbadfeline 11mo ago
      > Under the assumption that AI agents will make human software engineering obsolete That assumption is wrong, humans will always have to guide the bots to achieve human-worthy goals, be it as supervising engineers, small business owners or a combination of the two - that helps competition and avoids lock-in.
KurSix 11mo ago
You're right. And it's not just about parameter count. Efficiency is a full-stack problem: from the architecture (like MoE instead of dense transformers), to the inference techniques (speculative decoding), to the data formats (quantization down to INT4/FP4). The shortage will force everyone to optimize every step of the process, not just "add more layers"

KronisLV 11mo ago

For what it’s worth, even the Qwen 30B model has its use cases. And as far as some of the better open models go, by now the GLM 4.6 355B model is largely better than the Qwen3 Coder 480B variant, so it seems that the models are getting more efficient across the board.

estimator7292 11mo ago

Energy and GPU costs haven't pushed the needle any, so I don't see any reason to expect that RAM costs will.

muldvarp 11mo ago

> We need more of this kind of optimization. Who is the "we" in this sentence? The ultra-rich that don't want to pay white collar workers to build software? The advantages of LLMs are tiny for software engineers (you might be more productive, you don't get paid more) and the downsides are bad to life-altering (you get to review AI slop all day, you lose your job).

bigbadfeline 11mo ago
> The ultra-rich that don't want to pay white collar workers to build software? This is already a fact and it's set in stone - making AI cheaper won't change anything in that regard. However, a cheaper AI will allow the laid-off software engineers to use models independently of those firing them, and even compete on an equal footing.
- muldvarp 11mo ago
  Compete for what exactly? Under the assumption that AI agents will make human software engineering obsolete there won't be a market for you to compete in. Everyone that wants a piece of software will ask their AI agent to create it. My ability to create software is only really useful to me because other people pay me for it. If AI agents take that from me, it won't matter that I can now create awesome software in minutes instead of weeks. The software was never really the thing that was useful for me.
  - bigbadfeline 11mo ago
    > Under the assumption that AI agents will make human software engineering obsolete That assumption is wrong, humans will always have to guide the bots to achieve human-worthy goals, be it as supervising engineers, small business owners or a combination of the two - that helps competition and avoids lock-in.

KurSix 11mo ago

You're right. And it's not just about parameter count. Efficiency is a full-stack problem: from the architecture (like MoE instead of dense transformers), to the inference techniques (speculative decoding), to the data formats (quantization down to INT4/FP4). The shortage will force everyone to optimize every step of the process, not just "add more layers"