7 ms·
I’m struggling with MRF. Model Release Fatigue. It’s a syndrome of constantly context switching new large models. Claude 4, gpt, llama, Gemini 2.5, pro-mini, mi
by tdhz77 1y ago
I’m struggling with MRF. Model Release Fatigue. It’s a syndrome of constantly context switching new large models. Claude 4, gpt, llama, Gemini 2.5, pro-mini, mistrial.
I fire off the ide switch the model and think oh great this is better. I switch to something that worked before and man, this sucks now.
Context switching llm, Model Release Fatigue
- vpShane 1y agoNow just make a chatbot for each model and then compete them all against each other in the ultimate showdown. Winner gets your attention for a week.
- didibear 1y agoI believe perfs of previous versions are worse because providers reallocate resources to newer versions. Also because of training data cut-off to previous years. This is what happened between claude sonnet 3.5 and 3.7. Personally I only use Claude/Anthropic and ignore other providers because I understand it the more. It's smart enough, I rarely need the latest greatest.
- emilsedgh 1y agoWhy do you even follow? Just stick to one that works well for you?
- barbazoo 1y agoTotally, I feel like though you do have to pay some attention for example in the context I'm working on, for the last while, Gemini was our gold standard for code generation whereas today, Claude subjectively produces the better results. Sure you can stick to what worked abut then you're missing the opportunity to be more productive or less busy, whichever one you choose.
- tartoran 1y agoFOMO may be one of the reasons amongst others.
- mrcwinn 1y agoWhat a luxury! One way to avoid this: stick with one LLM and bet on the company behind it (meaning, over time, they’ll always have the best offering). I’ve bet on OpenAI. Others can make different conclusions.
- tenuousemphasis 1y agoWhen the medicine is worse than the disease...
- criemen 1y agoI totally get it. Due to my work, I mostly keep up with new model releases, but the pace is not sustainable for individuals, or the industry. I'm hoping that model releases (and the entire development speed of the field) will slow down over time, as LLMs mature and most low-hanging fruits in model training have been picked. Are we there yet? Surely not.
- bee_rider 1y agoA major reason I haven’t really tried any of these things (despite thinking they are vaguely neat). I think I will wait until… 2026, second half, most likely. At least I’ll check if we have local models and hardware that can run them nicely, by then. Hats off to the folks who have decided to deal with the nascent versions though.
- nosianu 1y agoI have a modified tiered approach, that I adopted without consciously thinking hard about it. I use AI mostly for problems on my fringes. Things like manipulating some Excel table somebody sent me with invoice data from one of our suppliers and some moderately complex question that they (pure business) don't know how to handle, where simple formulas would not be sufficient and I would have to start learning Power Query. I can tell the AI exactly what I want in human language and don't have to learn a system that I only use because people here use it to fill holes not yet served by "real" software (databases, automated EDI data exchange, and code that automates the business processes). It works great, and it saves me hours on fringe tasks that people outsource to me, but that I too don't really want to deal with too much. For example, I also don't check various vendors and models against one another. I still stick to whatever the default is from the first vendor I signed up with, and so far it worked well enough. If I were to spend time checking vendors and models, the knowledge would be outdated far too quickly for my taste. On the other hand, I don't use it for my core tasks yet. Too much movement in this space, I would have to invest many hours in how to integrate this new stuff when the "old" software approach is more than sufficient, still more reliable, and vastly more economical (once implemented). Same for coding. I ask AI on the fringes where I don't know enough, but in the core that I'm sufficiently proficient with I wait for a more stable AI world. I don't solve complex sciency problems, I move business data around. Many suppliers, many customers, different countries, various EDI formats, everybody has slightly different data and naming and procedures. For example, I have to deal with one vendor wanting some share of pre-payment early in the year, which I have to apply to thousands of invoices over the year and track when we have to pay a number of hundreds or thousands of invoices all with different payment conditions and timings. If I were to ask the AI I would have to be so super specific I may as well write the code. But I love AI on the not-yet-automated edges. I'm starting to show others how they can ask some AI, and many are surprised how easy it is - when you have thee right task and know exactly hat you have and what you want. My last colleague-convert was someone already past retirement age (still working on the business side). I think this is a good time to gradually teach regular employees some small use cases to get them interested, rather than some big top-down approach that mostly creates more work and many people then rightly question what the point is. About politically-touched questions like whether I should rather use an EU-made AI like the one this topic is about, or use one from the already much of the software-world dominating US vendor, I don't care at this point, because I'm not yet creating any significant dependencies. I am glad to see it happening though (as an EU country citizen).
- reilly3000 1y agoNot to invalidate your feelings of fatigue, but I’m sure glad that there are a lot of choices in the marketplace, and that they are innovating at a decent clip. If you’re committed to always be using the best of all options you’re in for a wild ride, but it beats stagnation and monopoly.
- ivape 1y agoWe’re also headed into a world where there will be very few open weight models coming out (Meta going closed source, not releasing Behemoth). This era of constant model releases may be over before it even started. Gratitude definitely needs to be echoed.
- echelon 1y agoIf China stays open, then the rest of the world will build on open. I'm frankly shocked that a domestic player isn't doing this. Fine tuning will work for niche business use cases better than promises of AGI.
- seszett 1y ago> If China stays open, then the rest of the world will build on open I was listening to a Taiwanese news channel earlier today and although I wasn't paying much attention, I remember hearing about how Chinese AIs are biased towards Chinese political ideas and that some programme to create a more Taiwanese-aligned AI was being put in place. I wouldn't be surprised if just for this reason, at least a few different open models kept being released, because even if they don't directly bring in money, several actors care more about spreading or defending their ideas and IAs are perfect for that.
- outworlder 1y agoIt makes sense that they would be trojan horses.
- 1y ago
- sunaookami 1y agoYou only need Claude and GPT. Everything else is not worth your time.
- vouaobrasil 1y agoAn alternative: don't use LLMs. Focus on the enjoyment of coding, not on becoming more efficient. Because the lion's share of the gains from increased efficiency are mainly going to the CEOs.
- wahnfrieden 1y agoThis is HN, we are not all wage workers here For wage workers, not learning the latest productivity tools will result in job loss. By the time it is expected of your role, if you have not learned already, you won't be given the leniency to catch up on company time. There is no impactful resistance to this through individual protest, only by organizing your peers in industry
- wahnfrieden 1y agoI would like downvotes to explain what’s wrong
- gitremote 1y agoWhat does wage versus salary have to do with anything?
- wahnfrieden 1y agoSalary is a specific type of wage
- deleted 1y ago[deleted]
- freedomben 1y agoThis might be good short term advice, but in the medium and long term I think devs who don't use any AI will start to be much slower at delivery than devs who do. I'm alreay seeing it IRL (and I'm not a fan of AI coding, so this sucks for me)
- ivape 1y ago
- sva_ 1y agoAll the competition is great to me. I'm using premium models all the time and barely spent a few euro on them, as there's always some offers that are almost free if you look around.
- zamadatix 1y agoMuch like with new computer hardware, announcements are constant but they rarely entice me to drop one thing and switch to another. If an average user picked a top 3 option last year and stuck with them through now you didn't really miss out on all that much, even if your particular choice wasn't the absolute latest and greatest the entire time.
- wahnfrieden 1y agoSticking with one year old models would mean no o3 which is a huge loss for dev work
- Ey7NFZ3P0nzAe 1y agoUsing litellm and/or openrouter.ai really makes it painless