15 ms·

AI is stifling new tech adoption?

• jgalt212 2y ago
  Herein lies the key for IP protection. Never use cloud hosted coding tools as the world will soon be able to copy your homework at zero cost.
  • falcor84 2y ago
    I for one love it that we can copy each other's homework. The open source mindset is what made me fall in love with this industry, and I love the fact that sharing code got easier. If you really want to continue reinventing wheels, go ahead.
    • jgalt212 2y ago
      Yes, open is overall a great thing. But what if I want my work not to be open source, and the LLMs make it so without my consent. As we've seen, this and related matters of fair use, are working their way through the courts.
• ZaoLahma 2y ago
  Seems plausible, especially in combination with the AI-coma that occurs when you tab-complete your way through problems at full speed.
• _as_text 2y ago
  know what this will be about without reading Python 3.12-style type annnotations are a good example imo, no one uses the type statement because dataset inertia
  • deleted 2y ago
    [deleted]
  • bigfishrunning 2y ago
    Usually, i remember that type annotations exist when I'm debugging after things aren't working. If you look at the python code that I've written, type annotations are a sure sign that "there was a problem here". it's like a scar on repaired code
  • DrFalkyn 2y ago
    Type annotations by themselves, are little more than a comment
    • OutOfHere 2y ago
      They're more than that as they allow for easy consistency checking of types across calls. This makes all the difference.
• jimnotgym 2y ago
  Is this such a bad result? Do we need office CRUD apps to use bleeding edge technologies?
  • Manfred 2y ago
    It's also a problem when adopting new functionality in existing frameworks (eg. upgrading an app to new Android release), dropping the use of deprecated functionality, taking advantage of more readable syntax in programming languages, etc.
    • 9rx 2y ago
      > taking advantage of more readable syntax in programming languages "AI" is the programming language here. The readability of any lower level language(s) that may exist as a compiler target of the "AI" is about as important as how readable assembly language is to someone writing software in Rust.
  • jtbayly 2y ago
    If they don’t, then will the bleeding edge technologies ever be used by more serious apps? Will it even be ready if we don’t test it with easier use-cases?
  • deleted 2y ago
    [deleted]
  • benrutter 2y ago
    Probably not, but I guess the worry would be, if nobody adopts and uses them, bleeding edge technologies don't become the new normal. Unless you think we've reached perfection, it's almost guaranteed that future developers will look at React/Python/etc as we look at developing in Assembly or COBOL.
• jgalt212 2y ago
  Along similar lines, I found Google auto complete to constrict my search space. I would only search the terms that auto complete.
• physicsguy 2y ago
  If AI stifles the relentless churn in frontend frameworks then perhaps it's a good thing.
  • ZaoLahma 2y ago
    It isn't only frontend frameworks. I currently AI-coma / tab-complete C++17 with decent results for stuff ridiculously far away from frontend, but I do wonder who is providing the training data for C++23 and onwards as there isn't wide adaptation yet.
    • kannanvijayan 2y ago
      I can chime in with a similar anecdote. I use co-pilot extensively in "fancy tab completion" mode. I don't use the conversational features - just the auto-complete to help my coding along. I specifically found it very useful when dealing with a bunch of boilerplate C++ shim code that used some advanced template magic to declare families of signatures for typed thunks that wrapped and augmented some underlying function call with record/replay log. It was arcane, bespoke stuff. Highly unlikely to be imitated in training data. The underlying code was written by a colleague (the CTO) but it was hard to grok just because of all the noise the template magic added to the syntax, and carefully keeping track of what was getting substituted where. The changes I was making were mostly to add new shims to this collection of shims, and co-pilot happily used the pattern of shims in previous lines of code to extrapolate what my new shims should look like and offer me tab-completions that were sensible. That included some bad guesses (like inferred names for other templated forms that referred to different argument types), but overall it was structurally close enough to use as a reference point and fix up as needed. It was really great. Took what would have been about day's worth of work carefully figuring out from first principles how the template system was structured.. and made it into about half an hour of "get it to generate the next shim, verify that the shim does the right thing".
      • GrinningFool 2y ago
        > It was really great. Took what would have been about day's worth of work carefully figuring out from first principles how the template system was structured.. and made it into about half an hour of "get it to generate the next shim, verify that the shim does the right thing". That also seems to highlight the disadvantage too - if you'd taken a day, you would have come away with a deeper understanding of the template system.
• CharlieDigital 2y ago
  As the saying goes: while (React.isPopular) { React.isPopular = true } It's actually quite sad because there are objectively better models both for performance and memory including Preact, Svelte, Vue, and of course vanilla.
  • chipgap98 2y ago
    Does that really matter to most companies/developers? I’d much rather have a good enough solution with a large ecosystem built around it. It also takes a lot of investment for companies to change their tech stack
    • CharlieDigital 2y ago
      > Does that really matter to most companies/developers? If you're asking about performance and memory, then yes, it does. This is especially true in e-commerce where many studies have shown that overall page performance has a correlation to conversion. Add to that the fact that a lot of e-commerce has moved to mobile web, there's a strong case for picking the best performing technologies versus developer preference -- especially if AI is generating it. But even outside of e-comm, consider government websites where you may have low income users that are using cheaper, older, lower capability devices to access information and services using lower speed networks to boot. I do my day-to-day work on an M3 Max with 64GB RAM and fiber to the home; it's easy for developers to forget that many times, their end users can be on older devices, on low performing networks, and other factors that affect performance and usability of web applications. > ...with a large ecosystem built around it When you can generate whatever you want nearly instantly, what does "ecosystem" mean? Your possibilities are endless and infinite. Your mindset is in a world where it's necessary to depend on the work of others because it's too expensive for you to write your own component library. Yes, that was true 1 year ago; why would you waste time and energy to create your own calendar component? But if an LLM can generate any bespoke component that you need in < 3 seconds, do you still need a third party component library? In fact, you may be better off for not creating the added external dependency.
      • 2y ago
• tiahura 2y ago
  Perhaps reasoning will help?
  • johnecheck 2y ago
    Yes! Reasoning is the answer! It will solve all of our problems! General AI is just around the corner!!!!!!!!!!!!!!!!!!!
• VMG 2y ago
  Guess I figured out my niche as a SWE: have a later knowledge cutoff date than LLMs
  • weeniehutjr 2y ago
    Niche: know anything about anything.
• spiderfarmer 2y ago
  >With Claude 3.5 Sonnet, which is generally my AI offering of choice given its superior coding ability, my “What personal preferences should Claude consider in responses?” profile setting includes the line “When writing code, use vanilla HTML/CSS/JS unless otherwise noted by me”. Despite this, Claude will frequently opt to generate new code with React, and in some occurrences even rewrite my existing code into React against my intent and without my consultation. I noticed this too. Anyone found out how to make Claude work better?
  • stuffoverflow 2y ago
    Since the system prompt tied to the artifacts feature seems to be the reason for it having a preference for react, seems like the solution would be to use the API instead. Plenty of front ends available nowadays that let you use your own API key. I've been using typingmind since I paid for it over a year ago but I'd be interested to know if some good open source alternatives have popped up more recently.
    • gordonhart 2y ago
      The main blocker to using the API with an alternative frontend is the cost. Daily API usage can easily go above the $20/month subscription cost since output tokens are expensive and each new message reuses the whole message chain as input tokens. Especially true if you often upload images or documents.
      • freehorse 2y ago
        Third party providers work through the APIs and thus do not have these issues. I use Kagi but there are many options out there for a price similar to the normal subscription, and plus you get a wider choice of models.
  • deleted 2y ago
    [deleted]
  • gordonhart 2y ago
    Claude is particularly bad about this, almost makes it unusable for my frontend use cases. I specify the exact tech stack in my prompt and it responds with a solution using whichever packages are available in its environment (Tailwind, shadcn/ui, etc.). My request to model providers: the strength of your offering is its generality. Please let it be general-purpose and resist adding features (alignment, system prompts, custom stuff like Claude artifacts) that limit this.
• orbital-decay 2y ago
  So... it slows down adoption by providing easier alternatives for beginners? I guess you could look at it that way too. Eventually it will go either of the two ways, though: - models will have enough generalization ability to be trained on new stuff that has passed the basic usefulness test in the hands of enthusiasts and shows promise - models will become smart enough to be useful even for obscure things
• PaulRobinson 2y ago
  I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation. LLMs should not have hard-wired preferences through providers' prompt structure. And while LLMs are stochastic parrots, and are likely to infer React if a lot of the training corpus mentions React, work should be done to actively prevent biases like this. If we can't get this right with JS frameworks, how are we going to solve it for more nuanced structural biases around ethnicity, gender, religion or political perspective? What I'm most concerned about here is that Anthropic is taking investment from tech firms who vendor dev tooling - it would not take much for them to "prefer" one of those proprietary toolchains. We might not have much of a problem with React today, but what if your choice of LLM started to determine if you could or couldn't get recommendations on AWS vs Azure vs GCP vs bare metal/roll your own? Or if it suggested only commercial tools instead of F/LOSS? And to take that to its logical conclusion, if that's happening, how do I know that the history assignment a kid is asking for help with isn't sneaking in an extreme viewpoint - and I don't care if it's extreme left or right, just warped by a political philosophy to be disconnected from truth - that the kid just accepts as truth?
  • avbanks 2y ago
    This is actually a very interesting insight, not only do you have to worry about sponsored results but people could game the system by spamming their library/language in a places which will be included in the training set of models. This will also present a significant challenge for security, because I can have a malicious library/package spam it in paths that will be picked up in the training set and have that package be referenced by the LLM.
  • deleted 2y ago
    [deleted]
  • BoxFour 2y ago
    The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners. As others have pointed out, it's a flywheel: Popular library gains traction → LLMs are trained to produce the “most likely response”, which naturally aligns with what’s already popular → people stop seeking alternative solutions and instead double down on the existing mainstream choices. (Hypothetically) It's not that OpenAI decides to push AWS, it's just that at the time it was trained AWS was the only real option so it just regurgitates a common view from a point in time. To extend your analogy, the more realistic scenario isn’t that kids slip in extreme view points and take them as ground truth in their history assignments, it’s that they don’t take a stance on anything at all: Their essays become like CSPAN transcripts, perfectly regurgitating what happened without taking any position or applying any critical thinking one way or the other. Imagine kids writing about civil rights, but all their reference material was stuck in time at 1953: That's what's more likely to happen.
• Eridrus 2y ago
  This will be solved eventually on the AI model side. It isn't some law of nature that it takes a million tokens for an AI to learn something; just the fact that we can prompt these models should convince you of that.
  • trescenzi 2y ago
    Maybe, but why would they bother? If 80% of the demand is met by generating really good Python and generating really good X is a lot more work but only 2% of the demand it’s likely there isn’t going to be a reason to solve that problem well.
  • orbital-decay 2y ago
    That's assuming it's a novel problem to deal with, see e.g. C++, JavaScript or every standard in networking ever. The barrier between better tech and worse tech that accidentally made it into production and became legacy cruft has always been high, without any AI.
• avbanks 2y ago
  LLM based AI tools are the new No/Low Code.
  • ebiester 2y ago
    It is, but it's an order of magnitude better than the last set of no/low code tools for anyone who has the basics of programming already.
• tajd 2y ago
  Yeah maybe. But I think the thing I like is that is takes me a much shorter amount of time to create solutions for my users and myself. Then I can worry about “tech adoption” once I’ve achieved a relevant solution to my users. If performance is an issue then sure let’s look at options. But I don’t think it’s appropriate to expect that sort of level of insight into an optimised solution from llms - but maybe that’s just because I’ve used them a lot. They’re just a function of their training data at the end of the day. If you want to use new technology you might have to generate your own training data as it were.
• jwblackwell 2y ago
  Larger context windows are helping solve this, though. I use ALpineJS which is not as well known as React etc, but I just added a bunch of examples and instructions to the new cursor project rules, and it's now close to perfect. Gemini models have up to 2M context windows, meaning you can probably fit your whole codebase and a ton of examples in a single request. Furthermore, the agenetic way Cursor is now behaving, automatically building up context before taking action, seems to be another way around this problem
  • causal 2y ago
    I also suspect reasoning models will start contributing genuine innovations to public repositories in the near future.
• conradfr 2y ago
  I was thinking the other day how coding assistants would hinder new languages adoption.
• killjoywashere 2y ago
  Pathologists as a specialty has been grousing about this for several years, at least since 2021 when the College of American Pathologists established the AI Committee. As a trivial example: any trained model deployed will necessarily be behind any new classification of tumors. This makes it harder to push the science and clinical diagnosis of cancer forward. The entire music community has been complaining about how old music gets more recommendations on streaming platforms, necessarily making it harder for new music to break out. It's absolutely fascinating watching software developers come to grips with what they have wrought.
  • Schiendelman 2y ago
    The healthcare diagnosis one may be wrong. For existing known diagnoses, (or at least the sliver of diagnoses in this one study), AI can beat doctors - and doctors don't like listening when it challenges them, so it will disrupt them badly as people learn they can provide data from tests directly to AI agents. Sure, this doesn't replace new diagnoses, but the vaaaast majority of failures to diagnose are for existing well classified diagnoses. https://www.advisory.com/daily-briefing/2024/12/03/ai-diagnosis-ec https://www.advisory.com/daily-briefing/2024/12/03/ai-diagno... Edit: yeah, people don't like this.
    • killjoywashere 2y ago
      I'm familiar with the linked study, which presents legitimately challenging analytic problems. There's a difference between challenging analytic problems and new analytic problems. A new platform poses new analytic problems. A new edition of the WHO's classification of skin tumors (1), for example, presents new analytic problems. (1) https://tumourclassification.iarc.who.int/chapters/64 https://tumourclassification.iarc.who.int/chapters/64
      • Schiendelman 2y ago
        Right, but the vast majority of patient issues today are missing existing diagnoses, not new ones.
    • 2y ago
• delichon 2y ago
  Working in Zed I'm full of joy when I see how well Claude can help me code. But when I ask Claude about how to use Zed it's worse than useless, because it's training data is old compared to Zed, and it freely hallucinates answers. So for that I switch over to Perplexity calling OpenAI and get far better answers. I don't know if it's more recent training or RAG, but OpenAI knows about recent Zed github issues where Claude doesn't. As long as the AI is pulling in the most recent changes it wouldn't seem to be stiflling.
  • soared 2y ago
    I tried uses chatgpt 4o to write a simple website that used the chagpt api. It always generated code for their deprecated API. I’d paste the error about using old calls, and it would recognize its error and.. generate old calls again. Couldn’t ever use it owns api.
• chrisco255 2y ago
  This makes me fear less for web development jobs being lost to AI, to be honest. Look, we can create new frameworks faster than they can train new models. If we all agree to churn as much as possible the AIs will never be able to keep up.
  • kdmtctl 2y ago
    Until the AI groks the principle of deviation and starts generating frameworks faster than we do.
• anarticle 2y ago
  Sadly, as a person who used write AVX in C for real time imaging systems: don't care shipped. I love dingling around with Cursor/Claude/qwen to get a 300 line prototype going in about 3-5 minutes with a framework I don't know. It's an amazing time to be small, I would hate to be working at a megacorp where you have to wait two months to get approval to use only GitHub copilot (terrible), in a time of so many interesting tools and more powerful models every month. For new people, you still have to put the work in and learn if you want to transcend. That's always been there in this industry and I say that as a 20y vet, C, perl, java, rails, python, R, all the bash bits, every part matters just keep at it. I feel like a lot of this is the js frontend committee running headlong into their first sea change in the industry.
• mtkd 2y ago
  Sonnet + Tailwind is something of a force multiplier though -- backend engineers now have a fast/reliable way of making frontend changes that are understandable and without relying on someone else -- you can even give 4o a whiteboard drawing of a layout and get the tailwind back in seconds On the wider points, I do think it is reducing time coders are thinking about strategic situation as they're too busy advancing smaller tactical areas which AI is great at assisting -- and agree there is a recency issue looming, once these models have heavy weightings baked in, how does new knowledge get to the front quickly -- where is that new knowledge now people don't use Stackoverflow? Maybe Grok becomes important purely because it has access to developers and researchers talking in realtime even if they are not posting code there I worry the speed that this is happening results in younger developers not spending weeks or months thinking about something -- so they get some kind of code ADHD and never develop the skills to take on the big picture stuff later which could be quite a way off AI taking on
  • the__alchemist 2y ago
    > backend engineers now have a fast/reliable way of making frontend changes that are understandable and without relying on someone else backend engineers in this context could learn JS.
• hiAndrewQuinn 2y ago
  >Consider a developer working with a cutting-edge JavaScript framework released just months ago. When they turn to AI coding assistants for help, they find these tools unable to provide meaningful guidance because their training data predates the framework’s release. [... This] incentivises them to use something [older]. That sounds great to me, actually. A world where e.g. Django and React are considered as obvious choices for backend and frontend as git is for version control sounds like a world where high quality web apps become much cheaper to build.
  • matsemann 2y ago
    What if it happened just before React, and you therefore got stuck with angular? Should we now be stuck with React forever just because it's okay-ish, never allowing future better framework to emerge?
    • hiAndrewQuinn 2y ago
      >What if it happened just before React, and you therefore got stuck with angular? Still a good thing. :) The massive bump in developer market liquidity is far more valuable in my eyes than any inherent DevEx advantages. You'd still have much cheaper high quality web apps, although, if React truly has a technical advantage over Angular (doubtful), maybe not as much cheaper, but still much cheaper than pre-LLM. If you truly want to figure out where I think the equation sign flips, it's probably like, pre-Smalltalk somewhere.
    • tobyhinloopen 2y ago
      Or we ended up writing jqueryscript forever! $(function() { // yay })
  • klysm 2y ago
    I’m all for boring technologies but can we please at least use compiled languages with types
    • lasagnagram 2y ago
• moyix 2y ago
  One thing that is interesting is that this was anticipated by the OpenAI Codex paper (which led to GitHub Copilot) all the way back in 2021: > Users might be more inclined to accept the Codex answer under the assumption that the package it suggests is the one with which Codex will be more helpful. As a result, certain players might become more entrenched in the package market and Codex might not be aware of new packages developed after the training data was originally gathered. Further, for already existing packages, the model may make suggestions for deprecated methods. This could increase open-source developers’ incentive to maintain backward compatibility, which could pose challenges given that open-source projects are often under-resourced (Eghbal, 2020; Trinkenreich et al., 2021). https://arxiv.org/pdf/2107.03374 https://arxiv.org/pdf/2107.03374 (Appendix H.4)
  • MattGaiser 2y ago
    ChatGPT and Gemini default to create-react-app, which has been considered poor practice for 2 years at least.
    • normie3000 2y ago
      What's considered better practice? NextJS?!
      • tylerjaywood 2y ago
        vite
      • ZeWaka 2y ago
        Sure, or something like create-vite. Dealers choice.
      • TheRealPomax 2y ago
        Literally yes? https://react.dev/learn/creating-a-react-app https://react.dev/learn/creating-a-react-app lists several options because different people will click with different solutions. Or find one of the many other ones yourself of course, they're not hard to find, but cutting your teeth on an official recommendation before moving on to greener pastures is always a good idea. The first one is always to learn from and then throw away.
• jleask 2y ago
  The underlying tech choice only matters at the moment because as software developers we are used to that choice being important. We see it as important because we currently are the ones that have to use it. As more and more software is generated and the prompt becomes how we define software rather than code i.e. we shift up an abstraction level, how it is implemented will become less and less interesting to people. In the same way that product owners now do not care about technology, they just want a working solution that meets their requirements. Similarly I don't care how the assembly language produced by a compiler looks most of the time.
• booleandilemma 2y ago
  Seems like a short-term problem. We're going to get to the point (maybe we're already there?) where we'll be able to point an AI at a codebase and say "refactor that codebase to use the latest language features" and it'll be done instantly. Sure, there might be a lag of a few months or a year, but who cares?
• at_ 2y ago
  Anecdotally, working on an old Vue 2 app I found Claude would almost always return "refactors" as React + Tailwind the first time, and need nudging back into using Vue 2.
  • AznHisoka 2y ago
    I have a similar experience when I tell ChatgPT I need Ruby 2.x code. It always gives me a Ruby 3 version even after I tell it I need code that works with version 2. I need to scream and curse at it before it fixes it so it works
• pmuk 2y ago
  I have noticed this. I think it also applies to the popularity of the projects in general and the number of training examples it has seen. I was testing Github copilot's new "Agent" feature last weekend and rapidly built a working app with Vue.js + Vite + InstantSearch + Typesense + Tailwind CSS + DaisyUI Today I tried to build another app with Rust and Dioxus and it could barely get the dev environment to load, kept getting stuck on circular errors.
• evanjrowley 2y ago
  Neovim author TJ DeVries Express similar concerns in a video earlier this year: https://youtu.be/pmtuMJDjh5A?si=PfpIDcnjuLI1BB0L https://youtu.be/pmtuMJDjh5A?si=PfpIDcnjuLI1BB0L
• benrutter 2y ago
  I think annecdotally this is true, I've definitely seen worse, but older technologies be chosen on the basis of LLM's knowing more about them. That said, I also think it's a bad choice, and here's some good news on that front- you can make good choices which will put you and your project/company ahead of many projects/companies making bad choices! I don't think the issue is that specific to LLMs- people have been choosing React and similar technologies "because it's easy to find developers" for ages. It's definitely a shame to see people make poor design decisions for new reasons, but I think poor design decisions for dumb reasons are gonna outlive LLMs by some way.
• trescenzi 2y ago
  Generative AI is fundamentally a tool that enables acceleration. Everything mentioned in this already true without Gen AI. Docs of new versions aren’t as easy to find till they aren’t as new. This is even true for things in the zeitgeist. Anyone around for the Python 2 to 3 or React class to hooks transitions knows how annoying that can be. Yes new programmers will land on Python and React for most things. But they already do. And Gen AI will do what it does best and accelerate. It remains to be seen what’ll come of that trend acceleration.
• dataviz1000 2y ago
  I'm on the fence with this. I've been using Copilot with vscode constantly and it has greatly increased my productivity. Most important it helps me maintain momentum without getting stuck. Ten years ago I would face a problem with no solution, write a detailed question on Stack Exchange, and most likely solve it in a day or two with a lot of tinkering. Today I ask Claude. If it doesn't give me a good answer, I can get the information I need to solve the problem. I've been thinking a lot of T.S. Eliot lately. He wrote and essay, "Tradition and the Individual Talent," which I think is pertinent to this issue. [0] (I should reread it.) [0] https://www.poetryfoundation.org/articles/69400/tradition-and-the-individual-talent https://www.poetryfoundation.org/articles/69400/tradition-an...
• lherron 2y ago
  I don't know how you solve the "training data and tooling prompts bias LLM responses towards old frameworks" part of this, but once a new (post-cutoff) framework has been surfaced, LLMs seem quite capable of adapting using in-context learning. New framework developers need to make sure their documentation is adequate for a model to use it when the docs are injected into the context.
• memhole 2y ago
  I've wondered this myself. There was a post about gumroad a few months ago where the CEO explained the decision to migrate to typescript and react. The decision was in part because of how well AI generated those, iirc.
• lackoftactics 2y ago
  > OpenAI’s latest models have cutoffs of late 2023. The first paragraph is factually incorrect; the cutoff is June 2024 for 4o. Awww, no more new JavaScript frameworks and waiting only for established technologies to cut through the noise. I don't see that as a bad thing. Technologies need to mature, and maintaining API backward compatibility is another advantage.
  • mtkd 2y ago
    The problem is acute with APIs that move fast and deprecate (Shopify and some of the Google ones)
  • smashed 2y ago
    It does not really matter as even though the models get updated, the new data was produced with the help of the older models, it is feeding on itself. Just imagine how hard it would be to push a new programming language. No AI models would be able to generate code in that new language, or they would be extremely limited. This would make adoption much more difficult in a world where all developers use AI tooling extensively. I believe this trend could create new opportunities also: as everyone uses AI tools to generate statistically average quality code, only those not using AI tools will be able to create true innovation.
    • Workaccount2 2y ago
      In some sense I am hopeful that AI will be able to just write everything directly in binary. Everything written ideally, with no abstraction, fluff or bumpers for human brains. Computers don't talk in any high level programming language, they talk in binary. If anything we should probably be focusing LLMs on getting good at that. I can only imagine that the amount of energy wasted on CPU cycles from layers of bloated programming languages makes stuff like bitcoin mining look like a rounding error.
      • atypeoferror 2y ago
        Not sure that’s always a good thing - see the occasionally erratic behavior of Tesla’s autopilot. It directly speaks the language of the systems it connects with, and also occasionally steers into dividers, killing people - and nobody knows why. We need to be able to verify correctness of what the models generate.
• matsemann 2y ago
  I actually asked this a while back, but got little response: https://news.ycombinator.com/item?id=40263033 https://news.ycombinator.com/item?id=40263033 > Ask HN: Will LLMs hurt adoption of new frameworks and technology? > If I ask some LLM/GPT a react question I get good responses. If I ask it about a framework released after the training data was obtained, it will either not know or hallucinate. Or if it's a lesser known framework the quality will be worse than for a known framework. Same with other things like hardware manuals not being trained on yet etc. > As more and more devs rely on AI tools in their work flows, will emerging tech have a bigger hurdle than before to be adopted? Will we regress to the mean?
  • spamizbad 2y ago
    It seems self-evident it will, and it's largely self-reinforcing. Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... I do wonder if this problem could get solved by basically providing documentation explicitly written for LLMs to consume and produce more detailed "synthetic" documentation/examples from. No idea if that's possible or even wise, but probably a problem space worth exploring. Or if these LLMs develop some sort of standardized way to rapidly apply new bodies of work that avoids costly retraining - like kernel modules, but for LLMs.
    • cbg0 2y ago
      Since the current chatbots have the ability to tap into Google Search, it's not unlikely they could gather their own up to date documentation on-the-fly. This would create a slew of new attack vectors where malicious actors will try to add backdoors into the documentation, which the LLM would reproduce. A seasoned software engineer will easily pick it up, but the large amount of folks that are just copy pasting chatbot output to make their own apps will certainly miss it.
      • com2kid 2y ago
        With people using LLMs instead of googling, there isn't an incentive for people to write tutorial blog posts.
• photochemsyn 2y ago
  The central issue is high cost of training the models, it seems: > "Once it has finally released, it usually remains stagnant in terms of having its knowledge updated. This creates an AI knowledge gap. A period between the present and AI’s training cutoff... The cutoff means that models are strictly limited in knowledge up to a certain point. For instance, Anthropic’s latest models have a cutoff of April 2024, and OpenAI’s latest models have cutoffs of late 2023." Hasn't DeepSeek's novel training methodology changed all that? If the energy and financial cost for training a model really has drastically dropped, then frequent retraining including new data should become the norm.
  • lrae 2y ago
    > Hasn't DeepSeek's novel training methodology changed all that? If the energy and financial cost for training a model really has drastically dropped, then frequent retraining including new data should become the norm. Even if training gets way cheaper or even if it stays as expensive but more money gets thrown at it, you'll still run into the issue of having no/less data to train on?
    • photochemsyn 2y ago
      True. One effective test for AGI might be the ability to first create a new language, then also write performant code in that language.
• armchairhacker 2y ago
  AI may be exaggerating this issue, but it's always existed. New tech has an inherent disadvantage vs legacy tech, because there's more built-up knowledge. If you choose React, you have better online resources (official docs, tutorials, answers to common pitfalls), more trust (it won't ship bugs or be abandoned), great third-party helper libraries, built-in IDE integration, and a large pool of employees with experience. If you choose some niche frontend framework, you have none of those. Also, popular frameworks usually have better code, because they have years of bug-fixes from being tested on many production servers, and the API has been tailored from real-world experience. In fact, I think the impact of AI generating better outputs for React is far less than that of the above. AI still works on novel programming languages and libraries, just at worse quality, whereas IDE integrations, helper libraries, online resources, etc. are useless (unless the novel language/library bridges to the popular one). And many people today still write code with zero AI, but nobody writes code without the internet.
  • kerblang 2y ago
    I've been looking all through here for someone to finally make this most obvious point. Even for those of us who use mostly stack overflow/google, it's much cheaper to wait on someone else to run into your problem and document the solution than to be first into the fire. We've relied on this strategy for a couple of decades now. I don't think the OP has demonstrated that adoption rates for new tech have changed in any way since AI. > Also, popular frameworks usually have better code, because they have years of bug-fixes from being tested on many production servers, and the API has been tailored from real-world experience. Overall I am very resistant to the idea that popular==good. I'd say popular==more popular. Also I think there's often a point where feeping creaturism results in tools that are overcomplicated, prone to security bugs and no longer easy to use.
• thecleaner 2y ago
  Shove the docs as context. Gemini has 2m context length.
• highfrequency 2y ago
  I have definitely noticed that ChatGPT is atrocious at writing Polars code (which was written recently and has a changing API) while being good at Pandas. I figure this will mostly resolve when the standard reasoning models incorporate web search through API documentation + trial and error code compilation into their chain of thought.
• d_watt 2y ago
  It's always been a thing with modes of encapsulating knowledge. The printing press caused the freezing of language, sometimes in a weird place* Where great documentation was make or break for a open source project for the last 10 years, I think creating new projects with AI in mind will be required in the future. Maybe that means creating a large amount of examples, maybe it means providing fine tunes, maybe it means publishing a MCP server. Maybe sad because it's another barrier to overcome, but the fact that AI coding is so powerful so quickly probably means it's worth the tradeoff, at least for now. *https://www.dictionary.com/e/printing-press-frozen-spelling/ https://www.dictionary.com/e/printing-press-frozen-spelling/
  • gradientsrneat 2y ago
    That article is so good that it deserves its own top-level Hacker News post. Regarding documentation: isn't the whole point of these LLMs to hoover up information in whatever form it's currently in and try to produce intelligible results? For SEO, things are going to get interesting, but software projects don't necessarily have such perverse incentives.
• draw_down 2y ago
  [dead]
• spenvo 2y ago
  Like several other commenters in this thread, I also wrote[0] something recently on a related topic: Google's AI Overviews and ChatGPT harm the discovery of long tail information - from a product builder's perspective. Basically, users are having a tougher time finding accurate info about your product (even if the correct answer to their query is in Google's own search results). And I also found the basic tier of ChatGPT hallucinated my app's purpose in a way that was borderline slanderous. AI can make it tougher (at scale) for creators trying to break through. [0] - https://keydiscussions.com/2025/02/05/when-google-ai-overviews-and-chatgpt-get-it-very-wrong-about-your-product/ https://keydiscussions.com/2025/02/05/when-google-ai-overvie...
  • diggan 2y ago
    > And I also found the basic tier of ChatGPT hallucinated my app's purpose in a way that was borderline slanderous. I'm curious about this, what exactly did ChatGPT write and how was it borderline slanderous? Sounds like a big danger.
    • spenvo 2y ago
      So ChatGPT seemingly guessed its purpose just from its name. Its name is CurrentKey Stats, and it inaccurately described it as an app that kept stats on the current keyboard keys you were pressing, so essentially a key logger, which is again completely wrong. I was actually somewhat hesitant to make this comment out of the fear that the next AI models will train on it and reinforce the false idea that that's what my app is. Sad times
      • ben_w 2y ago
        FWIW, one thing they're pretty competent at is sentiment analysis, so if they read your comment, even in isolation, what they'll probably learn that reading that kind of thing into a name is really bad in general. It's not like the bad old days where sentiment analysis was a bag of words model, add up all the "positive" words and subtract from that total the number of "negative" words — back then, they would mis-identify e.g. "Product was not as described, it did not come with batteries, and the surface wasn't even painted" as "this review favours this product" because they couldn't handle "not" or "wasn't".
• mring33621 2y ago
  I don't know how this is surprising. LLM-provided solutions will reinforce existing network effects. Things that are popular will have more related content...
• __MatrixMan__ 2y ago
  Can confirm, I recently gave up on learning anything new re: data visualization and have just been using matplotlib instead. Training data for it has been piling up since 2008. The AI's are so good at it that you hardly ever have to look at the code, just ask for changes to the graph and iterate. Honestly it's been kind of fun, but I do feel like the door is closing on certain categories of new thing. Local maxima are getting stickier, because even a marginal competence is enough to keep you there--since the AI will amplify that competence in well-trained domains by so much. Emacs lisp is another one. I'd kind of like to build a map of these.
  • hobs 2y ago
    Even trivial requirements they fail at when my boss says something and I have no idea if MPL supports it - visualization and visual mediums with complex interconnected bits are actually one of the harder things to do with a text/programmer basis... we literally fail to write tests to validate visual behavior all the time without tools like selenium.
• __MatrixMan__ 2y ago
  The Arrows of Time by Greg Egan (Orthogonal, Book 3) deals with something analogous to this: Our characters must break themselves out of a cycle which is impeding innovation. If you like your scifi hard, the Orthogonal series is a lot of fun.
• stevemadere 2y ago
  This is truly terrible. What happened to a new JS front end library every week? If this keeps up, we won't get to completely throw away all of our old code and retool every two years (the way we've been operating for the last 20 years) How will we ever spend 85% of our time spinning up on new js front end libraries? And don't even get me started on the back end. If AI had been around in 2010, we probably still have some people writing apps in Rails. OMG what a disaster that would be. It's a good thing we just completely threw away all of the work that went into all of those gems. If people had continued using them, we wouldn't have had the chance to completely rewrite all of them in node and python from scratch.
  • beej71 2y ago
    PHP everywhere?
  • dehrmann 2y ago
    New JS frameworks every week stopped around the time React became popular.
  • wruza 2y ago
    As if web/css wasn't a new gui library no one asked for. Peak programming was VB6 & Delphi 6 (maybe 7). Everything after that was just treading water in increasingly degenerate ways.
    • mouse_ 2y ago
      Eh. Web is the best write-once-run-everywhere we've achieved so far, especially with the proliferation of WASM. I'd be lying if I said it was perfect, but it's better than Java.
      • mrguyorama 2y ago
        >Web is the best write-once-run-everywhere we've achieved so far, Web for a decade or more now has been "Rewrite a hundred times, run only in chrome"
  • 2y ago
• deleted 2y ago
  [deleted]
• carlosdp 2y ago
  I don't think this is a bad thing. Pretty much all of the author's examples of "new and potentially superior technologies" are really just different flavors of developer UX for doing the same things you could do with the "old" libraries/technologies. In a world where AI is writing the code, who cares what libraries it is using? I don't really have to touch the code that much, I just need it to work. That's the future we're headed for, at lightning speed.
  • 3D30497420 2y ago
    The problem is if that code hasn't already been written in some form or another, then the LLM is much less effective at giving recommendations. I've been playing around with embedded systems, specifically LoRa libraries on ESP32s. Code from LLMs is next to useless for a lot of what I'm trying to do since it is relatively niche.
  • pphysch 2y ago
    > In a world where AI is writing the code, who cares what libraries it is using? I don't really have to touch the code that much, I just need it to work. That's the future we're headed for, at lightning speed. This attitude works for write-and-forget workflows where the only thing that matters is whether it returns the answer you want (AKA "hacking it"). Once you add in other concerns: security, performance, maintainability, it can fall apart.
    • hypothesis 2y ago
      > Once you add in other concerns: security, performance, maintainability, it can fall apart. Does anyone care about that? E.g. CRWD is all time high, after all. There is zero need to change anything according to market.
      • pphysch 2y ago
        > Does anyone care about that? Yes
        • 2y ago
• tobyhinloopen 2y ago
  I noticed this as I experimented with alternatives for React and all of them I tried were terrible on OpenAI/ChatGPT. Either it doesn't know them, or it makes weird mistakes, or uses very outdated (no longer working) versions of the code. It is also annoying that most modern JS things have 4 versions to do the same thing: With TS, With TS + Decorators, With plain JS, with JSX, etc. so code generation picks one that isn't compatible with the "mode" you use.
  • thewebguyd 2y ago
    I've noticed ChatGPT/GH Copilot is also particularly bad at PowerShell (I do sysadminy things), especially anything to do with the MS Graph API. It just makes up Cmdlets a lot of the time. If you prod it enough though it will eventually get it right, which strikes me as odd, it's like the training data was just full of really bad code. By contrast, anything I've asked it to do in Python has been more or less spot on. I fear that in the future the choice of tech stack is going to be less on the merits of the stack itself and more centered around "Which language and framework does ChatGPT (or other AI) produce the best output for"
• lcfcjs6 2y ago
  There is an enormous fear from mainstream media of AI, but the thing that excites me the most about this is in health care. AI will find the cure to Alzeimers and countless other diseases, there's no doubt about it. This simple fact is enough to make it acceptable.
• ripped_britches 2y ago
  This should not be relevant with cursor being able to include docs in every query. For those who don’t use this I feel for ya.
• pphysch 2y ago
  I don't think this is unique to AI. There are categories of knowledge that are infested with bad practices (webdev, enterprise software), and even a direct web search will lead you to those results. AI definitely regurgitates many of these bad practices, I've seen it, but it's not obvious to everyone. I think it's unrealistic to expect a general purpose LLM would be an practical expert in a new field where there are potentially 0 human practical experts.
• tolerance 2y ago
  So what. …if society continues to delegate more of their work to AI then we are going to fall back into the grips that inform us that some people are better at things than other people are and some are worse at things than other people are and this is what lies beneath the bridge of relying or not relying on AI to leverage your capacity to think and act on what you feel. I think that People who will be willing to put in effort for their crafts without AI will be the ones who will be willing to try out new things and seek opportunities for ingenuity in the future. I think that the problem people have with this idea is that it runs counter to notions related to—ahem— diversity, equity and inclusion… On one hand and on it’s little finger is the legitimate concern that if companies who develop LLMs are not transparent with the technologies they make available to users when generating code, then they’ll hide all the scary and dangerous things that they make available to the people who’ll think, act and feel corrupt regardless of the tools they wield to impose disadvantages onto others. But I don’t think that will make a difference. The only way out is hard work in a world bent on making the work easy after it makes you weak.
  • yapyap 2y ago
    I think it’s very easy to say people dislike the notion you said cause it goes against DEI (the e stands for equality btw), like it’s such an easy scapegoat. People just don’t wanna put the work in, or aren’t able to put the work in cause they are busy surviving day to day, y’know, putting food on the table. Cause that is not a given for everyone.
    • michaelcampbell 2y ago
      > DEI (the e stands for equality btw), According to whom? https://en.wikipedia.org/wiki/Diversity%2C_equity%2C_and_inclusion https://en.wikipedia.org/wiki/Diversity%2C_equity%2C_and_inc...
    • tolerance 2y ago
      I know I didn’t make it easy to catch, but I think you may have misread me. I wasn’t referring to “DEI” as in the corpo-state initiative but the concepts themselves as they’re received independent of how they’re packaged in the acronym; in a political context. In this way, I think to call it “scapegoating” would do a disservice to a legitimate social conflict. I agree with your final observation in general, but what’s your point?
• Rehanzo 2y ago
  Does anyone know what font is used here?
  • esafak 2y ago
    https://fonts.google.com/specimen/Lexend https://fonts.google.com/specimen/Lexend
    • Rehanzo 2y ago
      Thanks!
• anal_reactor 2y ago
  Not a problem. I'm sure that being able to work well with new information is the next goal most researchers are working towards, so the entire post feels like a boomer complaining "computers are bad because they're big and bulky" thirty years ago, not being able to imagine the smartphone revolution.
• benve 2y ago
  I think this is true because I myself said to myself: "it is useless for me to create a library or abstraction for the developers of my project, much better to use everything verbose using the most popular libraries on the web". Until yesterday having an abstraction (or a better library/framework) could be very convenient to save time in writing a lot of code. Today if the code is mostly generated there is no need to create an abstraction. AI understands 1000 lines of code in python pandas much better than 10 lines of code using my library (which rationalises the use of pandas). The result will not only be a disincentive to use new technologies, but a disincentive to build products with an efficient architecture in terms of lines of code, and in particular a disincentive to abstraction. Maybe some product will become a hell with millions of lines of code that no one knows how to evolve and manage.
  • gavmor 2y ago
    Wow, and this posture doesn't apply to junior developers, ie a good abstraction is needed to avoid overwhelming the human "context window." But it is a shame--and possibly an existential risk--that we then begin to write code that can only be understood via LLM.
  • hobs 2y ago
    This is completely wrong and assumes that an LLM is just much better at its job than it is - an LLM doesn't do better with a chaotic code base, nobody does - a deeply nonsensical system that sort of works is by far the hardest to reason about if you want to fix or change anything, especially for a thing that has subhuman intelligence.
    • baq 2y ago
      LLMs work best matching patterns. If 1k loc matches patterns and the 10 loc doesn’t, it’s a problem. The only thing the OP is missing which combines the best of both worlds is to always put source of and/or docs for his abstractions into the context window of the LLM.
      • xmprt 2y ago
• JimboOmega 2y ago
  Has there been any progress or effort on solving the underlying problem? I'm not entirely sure why AI knowledge must be close to a year old, and clearly this is a problem developers are aware of. Is there are a technical reason they can't be, for instance, a month behind rather than close to a year?
• OutOfHere 2y ago
  Always get the response with and without a web search. The web search may yield a newer solution. Also, each package should ideally provide an LLM ingestible document. Upload this for the LLM, and have it answer questions specific to the new package.
• jayd16 2y ago
  It's pretty interesting and mildly shocking that everyone is just making the same 'who needs a new JS library' joke. What about closed source tooling? How do you expect an AI to ever help you with something it doesn't have a license to know about? Not everything in the world can be anonymously scraped into the yearly revision. If AI is going to stay we'll have to solve the problem of knowledge segmentation. If we solve that, keeping it up to date shouldn't be too bad.
  • impure-aqua 2y ago
    >What about closed source tooling? How do you expect an AI to ever help you with something it doesn't have a license to know about? Not everything in the world can be anonymously scraped into the yearly revision. This is not a novel problem. Proprietary toolchains already suffer from decreased resources on public forums like Stack Overflow; AI did not create this knowledge segmentation, it is scraping this public information after all. >It's pretty interesting and mildly shocking that everyone is just making the same 'who needs a new JS library' joke. Surely the proprietary toolchain is itself the 'new JS library'? Most developers I know don't enjoy working with esoteric commercial solutions that suffer from poor documentation, when there exists an open-source solution that is widely understood and freely documented. I do not see why AI code generation further incentivizing the use of the open-source solution is a problem.
    • jayd16 2y ago
      > This is not a novel problem. AI did not create this knowledge segmentation, it is scraping this public information after all. I think you misunderstand the situation. You as a person can be privy to private knowledge. Relying on AI enough that you can't use that private knowledge is the novel situation. > I do not see why AI code generation further incentivizing the use of the open-source solution is a problem. Maybe you don't get it and have never experienced it but there's a missive amount of development done against unreleased APIs or hardware. Game engines, firmware, etc. I doubt Apple is going to publish new SDKs for their new widgets long before any devs use them.
      • 2y ago
• montjoy 2y ago
  The lack of new training data also makes it bad at projects that are still maturing because it will suggest outdated code - or worse it will mix/match old and new syntax and generate something completely broken. I worry that the lack of new examples for it to train on will self-reinforce running old syntax that has bad patterns. If the "AI" could actually store its mistakes and corrections from interactive sessions long-term I think it would greatly alleviate this problem, but that opens up another whole set of problems.
• NiloCK 2y ago
  I, too, wrote a shittier version of this a little while back: https://www.paritybits.me/stack-ossification/ https://www.paritybits.me/stack-ossification/ Another observation since then: good documentation for newer tech stacks will not save the LLM's capabilities with that tech. I think the reason, in short, is that there's no shortcut for experience. Docs are book learning for tech stacks - millions (billions) of lines of source code among the training data are something else entirely.
• richardw 2y ago
  I tried a new agent library with a model a few weeks ago. Just pasted the relevant api docs in and it worked fine. However, while I’m proud of the outcomes, I’m not proud of the code. I’m not releasing anything open source until I feel it’s mine, which is another step. I’d be a bit embarrassed bringing another dev on. “I’m Richard and I’m using AI to code” Support Group: “Hi Richard”
• slevis 2y ago
  Looks like I might be the minority, but I disagree with this prediction. Better models will also be better at abstracting and we have seen several examples (e.g. the paper LIMO: Less is More for Reasoning) that with a small amount of training data, models can outperform larger models.
• g9yuayon 2y ago
  > Once it has finally released, it usually remains stagnant in terms of having its knowledge updated....meaning that models will not be able to service users requesting assistance with new technologies, thus disincentivising their use. I find such argument weak. We can say the same thing about a book, like "Once The Art of Computer Program is finally published, it usually remains stagnant in terms of having its knowledge updated, thus disincentivizing people to learn new algorithms".
• casey2 2y ago
  Truly and honestly 99% of developers haven't even heard of chatgpt or copilot let alone the general public. It's a self-emposed problem on the orgs that choose to use such tools. More to the point, recency bias is so much stronger I'd rather have a system that points people to the current correct solution than a slightly better solution that is somehow harder to understand despite it's claimed simplicity by fanatics.
• ilrwbwrkhv 2y ago
  > However, a leaked system prompt for Claude’s artifacts feature shows that both React and Tailwind are specifically mentioned. Damn.
• cushychicken 2y ago
  …Isn’t this the website that constantly encourages people to “choose boring technology” for their web tech startups? Aren’t a reasonable portion of the readers here people who bemoan the constant learning curve hellscape of frontend development? And now we’re going to be upset that tools that help us work faster, which are trained on data freely available on the internet and thus affected by the volume of training material, decide to (gasp) choose solutions with a greater body of examples? Just can’t satisfy all the people all the time, I guess! SMH.
• crazygringo 2y ago
  No, AI isn't. Any new tech, or version upgrade, or whatever, takes time for people to become familiar with it. You might as well say "Stack Overflow is stifling new tech adoption" because brand-new stuff doesn't have many Q's and A's yet. But that would be a silly thing to say. I'm not going to adopt a brand-new database regardless of LLM training data cutoff, just because enough people haven't had enough experience with it. And LLM's have a commercial incentive to retrain every so often anyways. It's not like we're going to confront a situation where an LLM doesn't know anything about tech that come out 5 or 10 years ago. Early adopters will be early adopters. And early adopters aren't the kind of people relying on an LLM to tell them what to try out.
  • deleted 2y ago
    [deleted]
  • TheTaytay 2y ago
    I feel myself taking AI’s base knowledge of a tech stack into account when I work. Otherwise it feels like I am swimming upstream. I can’t be the only one, and this article resonated for me.
    • crazygringo 2y ago
      Sure, but what I'm saying is that's not where the knowledge bottleneck is. The knowledge bottleneck is in the human population, which is then reflected in Stack Overflow and blogs, which is then reflected in LLM's. LLM's aren't doing anything special to stifle new tech adoption. New tech is harder to adopt because it's new tech, because people in general are less familiar with it. (And there's a little bit of a training delay in LLM's updating, but that's a minor aspect here compared to the years it takes for a new Python package to become popular, for example.)
      • hinkley 2y ago
        It’s generally true that the people who come up with the first interesting idioms in a new space get an outsized influence on the community. It takes a while for people to borrow or steal their ideas and run with them. On the plus side while the AI can’t write in that language there’s money to be made doing the work organically.
• datadrivenangel 2y ago
  This is the same problem as google/search engines: A new technology has less web presence, and thus ranks lower in the mechanisms for information distribution and retrieval until people put in the work to market it.
• skeeter2020 2y ago
  I don't agree, because the people using these tools for their wore work were never doing innovative tech in the first place.
• deleted 2y ago
  [deleted]
• nektro 2y ago
  developers using ai continue to find new and novel ways to make themselves worse
• janalsncm 2y ago
  I’ve been out of web dev for a while, but maybe the problem is there’s a new framework every 6 months and instead of delivering value to the end user, developers are rewriting their app in whatever the new framework is.
  • cluckindan 2y ago
    Nice stereotype. Does it hold water? I’ve been using the same backend framework professionally for over 10 years, and the same frontend framework for over 6 years. Clearly your thoughts on the matter are not reflective of reality.
    • loandbehold 2y ago
      Yes. If you have a working system built on 10 years old web framework, it's considered obsolete and in need of being upgraded/rewritten. Why? Imagine houses needing to be rebuild because its foundation is 10 years old.
      • cluckindan 2y ago
        Not talking about a single system here. The framework has gone through five major versions in that time and my projects usually last a couple months to a couple years.
  • bmurphy1976 2y ago
    That's only part of the problem. Complexity and expectations have exploded in the last decade, while investment is getting tighter and tighter (for the average corporation anyway, not necessarily the big boys). The constant frameworks churn is one attempt at solving the complexity problem. Unfortunately I don't think it's working.
  • throwup238 2y ago
    I don’t think that’s really the case anymore. The vast majority are on React, Vue, or Svelte (in order of my perception of their popularity). On the CSS side it seems like Tailwind and PostCSS has taken over. The heavier framework category is covered by Next.js. Other than Next, most of that popularity started solidifying in 2020. There are a bunch of newer frameworks like Astro and HTMX and so on, but it doesn’t seem like their adoption is eating into much of the bigger player’s “market.” There’s still the problem of many library authors playing fast and loose with semver breaking changes and absolutely gigantic dependency trees that exacerbate the issue. That’s where most of my churn comes from, not new frameworks.
• AlienRobot 2y ago
  >Consider a developer working with a cutting-edge JavaScript framework released just months ago. When they turn to AI coding assistants for help, they find these tools unable to provide meaningful guidance because their training data predates the framework’s release. This forces developers to rely solely on potentially limited official documentation and early adopter experiences, which, for better or worse, tends to be an ‘old’ way of doing things and incentivises them to use something else. I can't help but feel that a major problem these days is the lack of forums on the Internet, specially for programming. Forums foster and welcome new members, unlike StackOverflow. They're searchable, unlike Discord. Topics develop as people reply, unlike Reddit. You're talking to real people, unlike ChatGPT. You can post questions in them, unlike Github Issues. When I had an issue with a C++ library, I could often find a forum thread made by someone with a similar problem. Perhaps because there are so many Javascript libraries, creating a separate forum for each one of them didn't make sense, and this is the end result. I also feel that for documentation, LLMs are just not the answer. It's obvious that we need better tools. Or rather, that we need tools. I feel like before LLMs there simply weren't any universal tools for searching documentation and snippets other than Googling them, but Googling them never felt like the best method, so we jumped from one subpar method to another. No matter what tool we come up with, it will never have the flexibility and power of just asking another human about it.
• bilater 2y ago
  This is precisely why I have said that every new framework/library should have a markdown or text or whatever is the best format for LLM models endpoint that has all the docs and examples in one single page so you can easily copy it over to a models context. You want to make it as easy as possible for LLMs to be aware of how your software works. The fancy nested navigation guide walkthrough thing is cool for users but not optimized for this flow.
  • vrosas 2y ago
    This is literally just SEO 2.0
  • DangitBobby 2y ago
    This is something I'd like to have for pretty much any framework/library anyway.
• amelius 2y ago
  This is like saying in the 90s that Google Search would stifle tech adoption ... I don't buy it. AI can teach me in 5 minutes how to write a kernel module, even if I've never seen one. AI brings more tech to our fingertips, not less.
  • IshKebab 2y ago
    Did you read the article? It makes valid points that your comment doesn't address. It isn't a brainless "AI is making us stupider" post.
    • amelius 2y ago
      Yes, but do search engines not have a bias towards existing technologies?
      • IshKebab 2y ago
        Yes but they are much more up-to-date. And they don't have the react issue he mentioned.
        • amelius 2y ago
          I think it may hold for mediocre programmers who brainlessly do what AI tells them. Good programmers will always look for better ways, and for them AI is a great way to explore time-tested ideas in addition to a fast tab-completer. It will certainly not drive them away from new ideas (or at least the arguments in this piece didn't convince me).
• IshKebab 2y ago
  Huh how long until advertisers pay to get their product preferred by AI? If it isn't already happening...
• mxwsn 2y ago
  This ought to be called the qwerty effect, for how the qwerty keyboard layout can't be usurped at this point. It was at the right place at the right time, even though arguably its main design choices are no longer relevant, and there are arguably better layouts like dvorak. Python and React may similarly be enshrined for the future, for being at the right place at the right time. English as a language might be another example.
  • pinoy420 2y ago
    > arguably better layouts like dvorak I don’t think this has any truth outside of causing such argument. I am sure there is a paper on it that showed negligible difference for different layouts. I used to use Dvorak but then I stopped when I was around 17? Qwerty for life.
  • basch 2y ago
    It would be interesting if Apple and Google and Samsung agreed to make dvorak (or a mutually created better mobile/swipe variant) the default mobile keyboard for all NEW accounts going forward. Unlike hardware, software keyboards could be swapped in a generation or two.
  • jimmaswell 2y ago
    QWERTY is a poor example. The keyboard layout is not the bottleneck for anyone who does not participate in olympic typing competitions. DVORAK is just as arbitrary as QWERTY to everyone else including professionals, and there's value in backwards compatibility e.g. old keyboards don't become e-waste.
    • rhet0rica 2y ago
      It sounds like you're thinking purely about the speed of a skilled typist. Alternative keyboard layouts offer a tangible ergonomic benefit even at lower WPM counts, and can have a lower hunt-and-peck time for novices by clustering frequently-used letters together. (This last effect is particularly pronounced on small touch screens, where the seek time is non-trivial and the buttons are much too close together for any sort of real touch-typing.)
• zombiwoof 2y ago
  Yup, python pretty much wins due to training data
• hinkley 2y ago
  I’m working on a side project that actually probably could use AI later on and I’m doing everything I can not to “put a bird on it” which is the phase we are at with AI. I might be willing to use a SAT solver or linear algebra on it if I ever get to that point but there’s a lot else to do first. The problem space involves humans, so optimizing that can very quickly turn into “works in theory but not in practice”. It’d be the sort of thing where you use it but don’t brag about it.
• hinkley 2y ago
  I don’t like that this conclusion seems to be that if humans adopt every new technology before AI can train on it that their jobs will be more secure. That is its own kind of hell.
  • palmotea 2y ago
    > I don’t like that this conclusion seems to be that if humans adopt every new technology before AI can train on it that their jobs will be more secure. That is its own kind of hell. It's the hell we'll be forced into. The powers that be care not one whit for our well being or comfort. We have to duck and weave (or get smashed), while they "creatively" destroy.
    • lasagnagram 2y ago
      We have the power to destroy, too.
      • palmotea 2y ago
        No nearly as much. And when we do it, we tend to get punished.
• ausbah 2y ago
  i do wonder if this could be mitigated by sufficiently popular newer libraries submitting training data of their library or whatever in action
• ramoz 2y ago
  We could call this the hamster-wheel theory.
• catapulted 2y ago
  There is a counter example for this: MCP, a standard pushed by Anthropic, provides a long txt/MD optimized for Claude to be able to understand the protocol, which is very useful to bootstrap new plugins/servers that can be used as tools for LLMs. I found that fascinating and it works really well, and I was able to one-shot improve my CLInE extension (a coding agent similar to cursor.sh) to work with existing APIs/data. It's so easy to bootstrap that even though the standard is a couple of months old, already has a few hundred (albeit probably low quality) implementations to adapt to different services. - txt/markdown for LLMs: https://modelcontextprotocol.io/llms-full.txt https://modelcontextprotocol.io/llms-full.txt - server implementations: https://github.com/modelcontextprotocol/servers#-community-servers https://github.com/modelcontextprotocol/servers#-community-s...
  • Animats 2y ago
    Now we're going to see sites specifically optimized to promote something to AIs. It's the new search engine optimization.
    • delanyoyoko 2y ago
      Prompt Engine Optimization
• yieldcrv 2y ago
  Eh a cooldown period between the fanfare of a new thing and some battle testing before it gets added to the next AI’s training set is a good thing the delay is like 8 months for now, thats fine I think this is also great for some interview candidate assessments, you have new frameworks that AI can't answer questions about yet, and you can quiz a candidate on how well they are able to figure out how to use the new thing
• jgalt212 2y ago
  If you can build an app that an AI cannot, then you know some sort n-month head start on the competition.
• owenversteeg 2y ago
  I think as new data gets vacuumed up faster, this will be less of an issue. About a year ago here on HN I complained about how LLMs were useless for Svelte as they did not have it in their training data, and that they should update on a regular basis with fresh data. At the time my comment was considered ridiculous. One year later, that’s where we are, of course; the average cutoff of “LLM usefulness” with a new subject has dropped from multiple years to months and I see no reason that the trend will not continue.
• feoren 2y ago
  The answer to this seems obvious: continuous training of live models. No more "cutoff dates": have a process to continually ingest new information and update weights in existing models, to push out a new version every week. Note that I said "obvious", not "easy", because it certainly isn't. In fact it's basically an unsolved problem, and probably a fiendishly difficult one. It may involve more consensus-based approaches like mixture of experts where you cycle out older experts, things like that -- there are dozens of large problems to tackle with it. But if you want to solve this, that's where you should be looking.
  • thatguysaguy 2y ago
    Yeah I think every lab would love to do this and the field has been thinking about it forever. (Search lifelong learning or continual learning on Google Scholar). I don't think a technological solution is likely enough that we should pursue it instead of social solutions.
  • carlio 2y ago
    While you might be able to continuously update the model, are you able to continuously update the moderation of it? As the article says, it takes time to tune it and filter it; if you allow any content in without some filtering of outputs you might end up with another Tay. You'd have to think the liability would slow down the ability to simply update on the fly. Also, if the proportion of training data available is larger for more established frameworks, then the ability of the model to answer usefully are necessarily dictated by the volume of content which is biased towards older frameworks. It might be possible with live updating to get something about NewLibX but it probably would be a less useful answer compared to asking about 10YearOldLibY
    • zelphirkalt 2y ago
      Moderation is the real reason it will be difficult to have online learning models in production. I think the technical side of how to do it will not be the biggest issue. The biggest one will be liability for the output.
  • perrygeo 2y ago
• tomduncalf 2y ago
  I was talking about this the other day - to some extent it feels like React (and Tailwind) has won, because LLMs understand it so deeply due to the amount of content out there. Even if they do train on other technologies that come after, there maybe won’t be the volume of data for it to gain such a deep understanding. Also it doesn’t hurt that React has quite a stable/backwards compatible API, so outdated snippets probably still work… and in Tailwind’s case, I suspect the direct colocation of styles with the markup makes it a bit easier for AI to reason about.
• mncharity 2y ago
  In contrast, I suggest AI could accelerate new tech adoption. > if people are reluctant to adopt a new technology because of a lack of AI support, there will be fewer people [emphasis added] likely to produce material regarding said technology, which leads to an overall inverse feedback effect. Lack of AI support prevents a technology from gaining the required critical adoption mass, which in turn prevents a technology from entering use and having material made for it, At present. But what if this is a transient? It depends on the new technology's dev team being unable to generate synthetic material. What happens when they can create for themselves a fine tune that translates between versions of their tech, and between "the old thing everyone else is using" and their new tech? One that encapsulates their "idiomatic best practice" of the moment? "Please generate our rev n+1 doc set Hal"? "Take the new Joe's ten thousand FAQ questions about topic X list and generate answers"? "Update our entries in [1]"? "Translate the Introduction to Data Analysis using Python open-source textbook to our tech"? The quote illustrates a long-standing problem AI can help with - just reread it swapping "AI support" to "documentation". Once upon a time, releasing a new language was an ftp-able tar file with a non-portable compiler and a crappy text-or-PS file and a LISTSERV mailinglist. Now people want web sites, and spiffy docs, and Stack Overflow FAQs, and a community repo with lots and lots of batteries, and discuss, and a language server, and yes, now LLM support. But the effort delta between spiffy docs and big repo vs LLM support? Between SO and LLM latency? That depends on how much the dev team's own LLM can help with writing it all. If you want dystopian, think lots of weekend "I made my own X!" efforts easily training transliteration from an established X, and running a create-all-the-community-infrastructure-for-your-new-X hook. Which auto posts a Show HN. AI could at long last get us out of the glacial pace of stagnant progress which has characterized our field for decades. Love the ongoing learning of JS churn? Just wait for HaskellNext! ;P [1] https://learnxinyminutes.com/ https://learnxinyminutes.com/ https://rigaux.org/language-study/syntax-across-languages.html https://rigaux.org/language-study/syntax-across-languages.ht... https://rosettacode.org/wiki/Category:Programming_Languages https://rosettacode.org/wiki/Category:Programming_Languages ...
• j45 2y ago
  If people are skipping one shelf of tech, and jumping to the next shelf up with only ai trying to cover everything, and are let down, maybe there is an opportunity to share that there may be more realistic offers in the interim to offer across both.
• lasagnagram 2y ago
  No, new tech is just 100% extractive, wealth-generating garbage, and people are sick and tired of it. Come up with something new that isn't designed to vacuum up your data and your paycheck, and then maybe people will be more enthusiastic about it.
  • cpursley 2y ago
    Who's it extracting from?
  • blt 2y ago
    You didn't read the article
• kristianp 2y ago
  > Claude’s artifacts feature The article mentions that Claude’s artifacts feature is opinionated about using react and will even refuse, to code for Svelte Runes. It's hard to get it to use plain JavaScript because react is in the system prompt for artefacts. Poor prompt engineering in claude.
• spaceguillotine 2y ago
  If the only new feature is AI, its not worth the upgrade. Outside the lil tech bubble, people hate it.
• drbojingle 2y ago
  I think llms will be great for Lang's like elm personally. Especially with agents that can operate in an eval loop.
• jadbox 2y ago
  Upgrading to Tailwind v4 was horribly frustrating as every AI insisted on using v3 even though it technically knew the v4 api.
• 1970-01-01 2y ago
  Are we really at the point where we are concerned how abstraction levels are not being abandoned as quickly as they were yesterday?
• milesrout 2y ago
  Why would AI stifle tech adoption when ~nobody uses it? I think HN is in a bit of a bubble here. People on here seem to often think that everyone is using AI at work, it is really common and widely appreciated, etc. It really, really isn't. Most people in the software industry do not use it. Its use in other industries and in the professions is even lower. AI coding tools are bad enough at widely used things like Python and JS. They are DOGSHIT at generating C or C++. They are basically terrible at doing anything other than regurgitating things from Medium blogspam tutorials. The result is not people moving to only using technology that AI is "good" at (relatively, given it is terrible at coding anything at all). It is that the overwhelming majority don't use it at all. The thing is, nobody really talks about this because it isn't interesting _not_ to use something. You can't write many high-engagement blog posts to content-market your company by saying you still just use vim and ctags and documentation to write code, just like you did 10 years ago. That isn't noteworthy and nobody will read it or upvote it. HN is always biased by this towards the new, the noteworthy, changes to practices, etc. Just like browsing HN would lead you to believe people are rewriting their websites in new JS frameworks every 6 months. No, but posts about doing that obviously generate more engagement than 6-monthly "Update: Our website is still written in Ruby on Rails" posts would.
• zkmon 2y ago
  People used to live in villages and places that were not connected by roads. Now since we have roads, any place that is not connected by a road will be seen a rough place. The difficulty is caused by usage of roads and vehicles, because it was not perceived or felt back in those days. So technology and assistance create new perceived problems.
• ofirg 2y ago
  while it is true that there is a gap between what most LLMs "know" and current time that gap is getting smaller not larger with time, it is also possible to teach a model pass the knowledge cut-off with tools and an LLM might be encouraged to be aware of the gap and reach out for the latest information when it might have changed (pi is still pi but the country with the most people might have changed)
• razodactyl 2y ago
  Not entirely sure it's a hard fact but this is definitely an example of bias in an AI system. More reason to decouple and think for ourselves.
• nbuujocjut 2y ago
  Related: https://www.mjlivesey.co.uk/2025/02/01/llm-prog-lang.html https://www.mjlivesey.co.uk/2025/02/01/llm-prog-lang.html It’s doesn’t matter if a minority of passion techies will still be up for new tech, if the average developer just wanting to get the job done and relying on LLMs finds it harder, it will be a significant barrier.