4 ms·

Is anyone else just exhausted by the pace of all this. The models change constantly and relentlessly and so does the pricing, basically weekly at this point bet

by jdprgm 1mo ago

Is anyone else just exhausted by the pace of all this. The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. It feels nearly impossible to have any rigorous approach when choosing a particular model and price point for a task and more like blindly picking one. The time period needed to actually get familiar with various models to a degree you can intuitively choose appropriate ones for a task is moot when it will likely be superseded faster than the needed time. I guess if companies are footing the bills most employees just opt for whatever the most expensive model they can get away with. Even then choosing between the various leading models is the same kind of frustrating task. Every release every company has the same random collection of graphs and charts claiming the best performance on X, Y, and Z.

fantasizr 1mo ago
I stopped caring about the latest and greatest but because there's so much, the 'obsolete' free models do what I need and are worth the price.
dominotw 1mo ago
maybe thats why opnrouter sold big
Pikamander2 1mo ago
That's how cutting edge tech has always worked. Imagine buying a shiny new PC in the 90s only to see it become practically obsolete within a year.
- phainopepla2 1mo ago
  That's not the experience of owning a PC I remember from the 90s at all.
  - bananaflag 1mo ago
    It is how I remember it.
  - embedding-shape 1mo ago
    I remember CPUs moving relatively fast back then, some years in the 90s had relatively big jumps, much bigger than we saw today. The classic graph, : https://i.extremetech.com/imagery/content-types/03zc6ghfKswe41smvPXi8Zh/images-6.jpg https://i.extremetech.com/imagery/content-types/03zc6ghfKswe...
  - computomatic 1mo ago
    It was both. 90% of people never needed nor purchased a bleeding-edge computer. The mid-tier was "good enough" and far closer to affordable for most people; though, that bar also moved upward every year. If you bought a mid-tier computer that was good enough for what you needed, then you probably didn't shop/compare for the next few years and didn't notice. But if you shelled out $7-10k for a top-of-the-line system and paid attention to progress, you'd easily see that become the mid-tier $1000 option within two years or less. This is how it was in the 90's PC boom, at least. Likely the same for the decades before, not sure how it went in the 2000's.
    - senordevnyc 1mo ago
      if you shelled out $7-10k for a top-of-the-line system and paid attention to progress, you'd easily see that become the mid-tier $1000 option within two years or less This is not how I remember that period at all. Do you have any examples?
      - benjiro29 1mo ago
        My first PC 386 was in todays money easily $5000+ (basic 2d GPU + screen)... A lot of hardware in our family was handed down to my folks, because you lost so much on selling, that it was better to keep using them as they had less demands. 386 to 486 to the first Pentium (with the bug!)... You did not upgrade in place, it was often a new system. Sure, you maybe kept your screen, keyboard etc but ... The only upgrade we had on the same MB, was a coprocessor upgrade. Remember those? Each new generation of CPU was a new motherboard. Upgrading CPUs in the same MB really became a thing only later on. GPUs had a shelf life of barely a year. Its been 35 year but i remember TNT to TNT2 having like 9 month in between. Moving from 2D to 3D involved a constant cost as GPUs evolved fast and the latest games required latest hardware. We have not talked about the ISA, AGP, and PCI fun ... The “bus wars”. DOS to Windows 3.1 (and OS/2 somewhere in between) to 95 ... with software being pushing hardware, just like games did. This is why people are spoiled with cheap PC hardware where its cheap, and easily lasts 4+ years. Even with the bad memory price and more expensive GPUs, your can stil buy a $1500 system that will last you years (with maybe some lower game settings later on ... or the catalog of 10.000s games that will easily run on a mid tier GPU). PC hardware has become boring but extreme stable. You can run GPUs for year, switch MBs without issues while keeping large amounts of old hardware. That was NOT the 80s and 90s that i remember.
  - Nition 1mo ago
    "Within a year" is a bit of an exaggeration but it's true that the pace of PC tech during the 90s was much, much faster than it is now. CPU power was doubling every two years, and today we're at roughly eight years. Add onto that the rise of video cards in the late 90s.
  - fooker 1mo ago
    I remember memory size going up by a factor of 8 at every PC upgrade for the same price.
- upupupandaway 1mo ago
  Or you could buy a PC with a Celeron CPU, which was obsolete way before launch.
  - bananaflag 1mo ago
    When my dad bought one he told me outright "this is for poor people".
    - lackoftactics 1mo ago
      I have some fond memories from my celeron days :) But I was upgrading from AMD K5 100 MHz
      - exe34 1mo ago
        I had a 600MHz/64MB/9GB laptop that came with Windows mistake edition. I managed to survive first year of uni on it by switching to Vector Linux, which was really fast compared to Windows. (Of course, it had issues playing sound from more than one source, this was oss days). Then one day the hard drive appeared to die. I eventually realised the issue was located around the 1.5gb mark, so I recreated my Linux partitions after 2gb and it worked fine for the rest of the year.
        lackoftactics 1mo ago
        that's what I call true creativity AI can't replace
        exe34 1mo ago
        Nah, AIs will cheat their way through anything you let them do. If skipping half the drive will let it get that sweet RL reward, it will do it. Chat gpt came up with it: https://chatgpt.com/share/6a9ae2c9-1910-83ed-9591-1b30f8834cb9 https://chatgpt.com/share/6a9ae2c9-1910-83ed-9591-1b30f8834c... I assume it hasn't had time to read my post yet.
- re-thc 1mo ago
  Hardware definitely has longer lifecycle than AI model releases at this point. You don't see Nvidia and AMD fighting every other month over the latest cards.
- unreal37 1mo ago
  The 486 chip came out in 1989. The 586 came out in 1993. The pace of change ("practically obsolete") is different then and now.
  - dcl 1mo ago
    That's kind of wild. Our first PC was a an IBM PS/2 486SX 33Mhz, 4MB RAM, that was purchased in 1993.
upupupandaway 1mo ago
> The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. A dev in my team saw a new model and changed one application to use said model (essentially changing the contents of a url). One week later I received an escalation from the CTO of the company that our pace of weekly usage was in the millions of dollars (rather than low hundred thousands). Turns out that the new model was 5x more expensive but no one noticed.
- arjie 1mo ago
  Okay, well, that seems like a natural problem. I could understand if he went from one of the Gemini Flashes to the next (when they rebranded Flash to Flash Lite and came up with a new much more expensive Flash). Now that would be a mess.
gavinray 1mo ago
> Is anyone else just exhausted by the pace of all this. This is only the beginning. We are in the infancy of AI, progress will continue to accelerate until some filtering event or energy limitation happens.
tonyedgecombe 1mo ago
Fire and motion, Joel Spolsky blogged about this: https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ https://www.joelonsoftware.com/2002/01/06/fire-and-motion/
smcleod 1mo ago
The new releases and breakthroughs do the opposite for me - I feel energised by them. I felt like nothing truly that interesting had happened in tech for quite some time, now it's like the space race (except there is no one moon to reach). I appreciate boring tech as much as the next well worn engineer and I'm not saying this is all positive but it's so sure as hell thrilling and you don't have to be an astronaut to immediately benefit (or suffer I guess) from it.
matheusmoreira 1mo ago
Yeah I'm a bit exhausted at this point. I just finished benchmarking GPT 5.6 Sol and Fable 5.0 like two days ago. My data became obsolete literally one day after.
brokencode 1mo ago
You really don’t need to watch it that closely. If the model you’re using today is working well, just stick with it. If one day you open up Claude Code and it’s Opus 5.1 now instead of Opus 5, no big deal. It probably will work about the same as it did before. Maybe a little better. Or if you’re on Codex and some new cool Claude model comes out, no worries. There will probably be a similar new model for Codex within a few weeks. Maybe even within a few days.
- shostack 1mo ago
  One suggestion is to make a list or make a skill to have your agent keep a list of things you do not feel work well with today's models. And then, when new models come out, periodically, revisit items on that list to see if you get better results.
  - KnightHawk3 1mo ago
    Okay serious question, why not just write this down in your notes or something? Why use a language model for it.
    - shostack 1mo ago
      You can. The point of having your agent keep track of it is that it will likely notice things you won't, and it can automate cataloging it with relevant metadata (prompts, environment, examples, etc.) that make it trivial to automate rerunning those tests when new models launch.
Aurornis 1mo ago
I could see how this might feel frustrating to someone who doesn't enjoy experimenting with new things all the time. In practice, you can get away without keeping up with everything all the time. For personal use, pick a provider and get on their ~$20/month plan. Learn their high/medium/low model hierarchy. Start with their highest or second-highest model (GPT-5.6, Opus, etc) and observe your quota usage. If you're doing a lot of manual code review and analysis, the $20/month plan goes very far even on the highest models. If you're trying to vibecode everything as fast as possible it's a different story. If you keep running into quota limits, experiment with the next model down for easier tasks or adjusting the effort level. If the results are good enough, you've found your fit. If they're not, you might need the next plan up. For API/business use, you have to be checking your token spend as you go to calibrate to how much each task costs and where you fall in your budget. There are a lot of different tools that make this easy to visualize. For data tasks, you should have an eval with a golden dataset that you can run against new models for a nominal amount of token expenditure. It should be as simple as pointing the eval script at a new API or model and checking the score versus price.
- danenania 1mo ago
  Another suggestion to get the most bang for your buck: use the best model you have access to with max reasoning for planning, implement with a smaller model/lower reasoning, then review with the big model. Repeat as needed. Input tokens are much cheaper than output tokens. Not only because of baseline price—caching makes a huge difference too. There are many ways to take advantage of this asymmetry to get similar quality for a fraction of the cost!
teaearlgraycold 1mo ago
I just use Claude Opus and the GLM series. Nothing’s really changed for my workflow in the last 6 months.
epolanski 1mo ago
If model X fits your need, you don't need to upgrade. I have released applications on Gemini 3.5 flash that make real money and I don't see any particular reason to upgrade.
Zizizizz 1mo ago
It feels like this every day https://youtube.com/shorts/vGKC9LpGnOQ?is=iCG7qvAIL9oI5-_d https://youtube.com/shorts/vGKC9LpGnOQ?is=iCG7qvAIL9oI5-_d
flockonus 1mo ago
It is exhausting to keep up with model releases yes, much like it was for a while during the Cambrian explosion of FE frameworks, eventually tech seems to work out to consolidation. But more so it seems there is Fear of missing out (FOMO) in our behaviours. The reality is, if whatever model you are using are good for your purpose, well, keep on it.
mfkhalil 1mo ago
Hey, I'm on the team at LiteLLM that's building the auto-router and our goal right now is to abstract that decision making away from the end user. The biggest thing we're trying to figure out right now is how do we do that without frustrating the end user - as a developer myself I would hate for my agent to be dumbed down below the threshold needed to complete a task. In theory though, there is a minimum viable model for any given task, and we think that is a problem that the big labs will avoid because they profit from charging more per task. We're trying heuristic and LLM-based approaches but it's still a work in progress, so if this is something you'd be interested in trying would highly recommend trying ours out -- any and all feedback at this point is extremely valuable to us. https://docs.litellm.ai/docs/proxy/auto_routing https://docs.litellm.ai/docs/proxy/auto_routing
ghthor 1mo ago
I want the cheapest fastest model personally and at work. Stay in flow, edit like the wind.

by jdprgm 1mo ago

fantasizr 1mo ago

I stopped caring about the latest and greatest but because there's so much, the 'obsolete' free models do what I need and are worth the price.

dominotw 1mo ago

maybe thats why opnrouter sold big

Pikamander2 1mo ago

That's how cutting edge tech has always worked. Imagine buying a shiny new PC in the 90s only to see it become practically obsolete within a year.

phainopepla2 1mo ago
That's not the experience of owning a PC I remember from the 90s at all.
- bananaflag 1mo ago
  It is how I remember it.
- embedding-shape 1mo ago
  I remember CPUs moving relatively fast back then, some years in the 90s had relatively big jumps, much bigger than we saw today. The classic graph, : https://i.extremetech.com/imagery/content-types/03zc6ghfKswe41smvPXi8Zh/images-6.jpg https://i.extremetech.com/imagery/content-types/03zc6ghfKswe...
- computomatic 1mo ago
  It was both. 90% of people never needed nor purchased a bleeding-edge computer. The mid-tier was "good enough" and far closer to affordable for most people; though, that bar also moved upward every year. If you bought a mid-tier computer that was good enough for what you needed, then you probably didn't shop/compare for the next few years and didn't notice. But if you shelled out $7-10k for a top-of-the-line system and paid attention to progress, you'd easily see that become the mid-tier $1000 option within two years or less. This is how it was in the 90's PC boom, at least. Likely the same for the decades before, not sure how it went in the 2000's.
  - senordevnyc 1mo ago
    if you shelled out $7-10k for a top-of-the-line system and paid attention to progress, you'd easily see that become the mid-tier $1000 option within two years or less This is not how I remember that period at all. Do you have any examples?
    - benjiro29 1mo ago
      My first PC 386 was in todays money easily $5000+ (basic 2d GPU + screen)... A lot of hardware in our family was handed down to my folks, because you lost so much on selling, that it was better to keep using them as they had less demands. 386 to 486 to the first Pentium (with the bug!)... You did not upgrade in place, it was often a new system. Sure, you maybe kept your screen, keyboard etc but ... The only upgrade we had on the same MB, was a coprocessor upgrade. Remember those? Each new generation of CPU was a new motherboard. Upgrading CPUs in the same MB really became a thing only later on. GPUs had a shelf life of barely a year. Its been 35 year but i remember TNT to TNT2 having like 9 month in between. Moving from 2D to 3D involved a constant cost as GPUs evolved fast and the latest games required latest hardware. We have not talked about the ISA, AGP, and PCI fun ... The “bus wars”. DOS to Windows 3.1 (and OS/2 somewhere in between) to 95 ... with software being pushing hardware, just like games did. This is why people are spoiled with cheap PC hardware where its cheap, and easily lasts 4+ years. Even with the bad memory price and more expensive GPUs, your can stil buy a $1500 system that will last you years (with maybe some lower game settings later on ... or the catalog of 10.000s games that will easily run on a mid tier GPU). PC hardware has become boring but extreme stable. You can run GPUs for year, switch MBs without issues while keeping large amounts of old hardware. That was NOT the 80s and 90s that i remember.
- Nition 1mo ago
  "Within a year" is a bit of an exaggeration but it's true that the pace of PC tech during the 90s was much, much faster than it is now. CPU power was doubling every two years, and today we're at roughly eight years. Add onto that the rise of video cards in the late 90s.
- fooker 1mo ago
  I remember memory size going up by a factor of 8 at every PC upgrade for the same price.
upupupandaway 1mo ago
Or you could buy a PC with a Celeron CPU, which was obsolete way before launch.
- bananaflag 1mo ago
  When my dad bought one he told me outright "this is for poor people".
  - lackoftactics 1mo ago
    I have some fond memories from my celeron days :) But I was upgrading from AMD K5 100 MHz
    - exe34 1mo ago
      I had a 600MHz/64MB/9GB laptop that came with Windows mistake edition. I managed to survive first year of uni on it by switching to Vector Linux, which was really fast compared to Windows. (Of course, it had issues playing sound from more than one source, this was oss days). Then one day the hard drive appeared to die. I eventually realised the issue was located around the 1.5gb mark, so I recreated my Linux partitions after 2gb and it worked fine for the rest of the year.
      - lackoftactics 1mo ago
        that's what I call true creativity AI can't replace
        exe34 1mo ago
        Nah, AIs will cheat their way through anything you let them do. If skipping half the drive will let it get that sweet RL reward, it will do it. Chat gpt came up with it: https://chatgpt.com/share/6a9ae2c9-1910-83ed-9591-1b30f8834cb9 https://chatgpt.com/share/6a9ae2c9-1910-83ed-9591-1b30f8834c... I assume it hasn't had time to read my post yet.
re-thc 1mo ago
Hardware definitely has longer lifecycle than AI model releases at this point. You don't see Nvidia and AMD fighting every other month over the latest cards.
unreal37 1mo ago
The 486 chip came out in 1989. The 586 came out in 1993. The pace of change ("practically obsolete") is different then and now.
- dcl 1mo ago
  That's kind of wild. Our first PC was a an IBM PS/2 486SX 33Mhz, 4MB RAM, that was purchased in 1993.

upupupandaway 1mo ago

> The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. A dev in my team saw a new model and changed one application to use said model (essentially changing the contents of a url). One week later I received an escalation from the CTO of the company that our pace of weekly usage was in the millions of dollars (rather than low hundred thousands). Turns out that the new model was 5x more expensive but no one noticed.

arjie 1mo ago
Okay, well, that seems like a natural problem. I could understand if he went from one of the Gemini Flashes to the next (when they rebranded Flash to Flash Lite and came up with a new much more expensive Flash). Now that would be a mess.

gavinray 1mo ago

> Is anyone else just exhausted by the pace of all this. This is only the beginning. We are in the infancy of AI, progress will continue to accelerate until some filtering event or energy limitation happens.

tonyedgecombe 1mo ago

Fire and motion, Joel Spolsky blogged about this: https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ https://www.joelonsoftware.com/2002/01/06/fire-and-motion/

smcleod 1mo ago

The new releases and breakthroughs do the opposite for me - I feel energised by them. I felt like nothing truly that interesting had happened in tech for quite some time, now it's like the space race (except there is no one moon to reach). I appreciate boring tech as much as the next well worn engineer and I'm not saying this is all positive but it's so sure as hell thrilling and you don't have to be an astronaut to immediately benefit (or suffer I guess) from it.

matheusmoreira 1mo ago

Yeah I'm a bit exhausted at this point. I just finished benchmarking GPT 5.6 Sol and Fable 5.0 like two days ago. My data became obsolete literally one day after.

brokencode 1mo ago

You really don’t need to watch it that closely. If the model you’re using today is working well, just stick with it. If one day you open up Claude Code and it’s Opus 5.1 now instead of Opus 5, no big deal. It probably will work about the same as it did before. Maybe a little better. Or if you’re on Codex and some new cool Claude model comes out, no worries. There will probably be a similar new model for Codex within a few weeks. Maybe even within a few days.

shostack 1mo ago
One suggestion is to make a list or make a skill to have your agent keep a list of things you do not feel work well with today's models. And then, when new models come out, periodically, revisit items on that list to see if you get better results.
- KnightHawk3 1mo ago
  Okay serious question, why not just write this down in your notes or something? Why use a language model for it.
  - shostack 1mo ago
    You can. The point of having your agent keep track of it is that it will likely notice things you won't, and it can automate cataloging it with relevant metadata (prompts, environment, examples, etc.) that make it trivial to automate rerunning those tests when new models launch.

Aurornis 1mo ago

I could see how this might feel frustrating to someone who doesn't enjoy experimenting with new things all the time. In practice, you can get away without keeping up with everything all the time. For personal use, pick a provider and get on their ~$20/month plan. Learn their high/medium/low model hierarchy. Start with their highest or second-highest model (GPT-5.6, Opus, etc) and observe your quota usage. If you're doing a lot of manual code review and analysis, the $20/month plan goes very far even on the highest models. If you're trying to vibecode everything as fast as possible it's a different story. If you keep running into quota limits, experiment with the next model down for easier tasks or adjusting the effort level. If the results are good enough, you've found your fit. If they're not, you might need the next plan up. For API/business use, you have to be checking your token spend as you go to calibrate to how much each task costs and where you fall in your budget. There are a lot of different tools that make this easy to visualize. For data tasks, you should have an eval with a golden dataset that you can run against new models for a nominal amount of token expenditure. It should be as simple as pointing the eval script at a new API or model and checking the score versus price.

danenania 1mo ago
Another suggestion to get the most bang for your buck: use the best model you have access to with max reasoning for planning, implement with a smaller model/lower reasoning, then review with the big model. Repeat as needed. Input tokens are much cheaper than output tokens. Not only because of baseline price—caching makes a huge difference too. There are many ways to take advantage of this asymmetry to get similar quality for a fraction of the cost!

teaearlgraycold 1mo ago

I just use Claude Opus and the GLM series. Nothing’s really changed for my workflow in the last 6 months.

epolanski 1mo ago

If model X fits your need, you don't need to upgrade. I have released applications on Gemini 3.5 flash that make real money and I don't see any particular reason to upgrade.

Zizizizz 1mo ago

It feels like this every day https://youtube.com/shorts/vGKC9LpGnOQ?is=iCG7qvAIL9oI5-_d https://youtube.com/shorts/vGKC9LpGnOQ?is=iCG7qvAIL9oI5-_d

flockonus 1mo ago

It is exhausting to keep up with model releases yes, much like it was for a while during the Cambrian explosion of FE frameworks, eventually tech seems to work out to consolidation. But more so it seems there is Fear of missing out (FOMO) in our behaviours. The reality is, if whatever model you are using are good for your purpose, well, keep on it.

mfkhalil 1mo ago

Hey, I'm on the team at LiteLLM that's building the auto-router and our goal right now is to abstract that decision making away from the end user. The biggest thing we're trying to figure out right now is how do we do that without frustrating the end user - as a developer myself I would hate for my agent to be dumbed down below the threshold needed to complete a task. In theory though, there is a minimum viable model for any given task, and we think that is a problem that the big labs will avoid because they profit from charging more per task. We're trying heuristic and LLM-based approaches but it's still a work in progress, so if this is something you'd be interested in trying would highly recommend trying ours out -- any and all feedback at this point is extremely valuable to us. https://docs.litellm.ai/docs/proxy/auto_routing https://docs.litellm.ai/docs/proxy/auto_routing

ghthor 1mo ago

I want the cheapest fastest model personally and at work. Stay in flow, edit like the wind.