4 ms·

I actually asked this a while back, but got little response: https://news.ycombinator.com/item?id=40263033 https://news.ycombinator.com/item?id=40263033 > Ask

I actually asked this a while back, but got little response: https://news.ycombinator.com/item?id=40263033 https://news.ycombinator.com/item?id=40263033 > Ask HN: Will LLMs hurt adoption of new frameworks and technology? > If I ask some LLM/GPT a react question I get good responses. If I ask it about a framework released after the training data was obtained, it will either not know or hallucinate. Or if it's a lesser known framework the quality will be worse than for a known framework. Same with other things like hardware manuals not being trained on yet etc. > As more and more devs rely on AI tools in their work flows, will emerging tech have a bigger hurdle than before to be adopted? Will we regress to the mean?

spamizbad 2y ago
It seems self-evident it will, and it's largely self-reinforcing. Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... I do wonder if this problem could get solved by basically providing documentation explicitly written for LLMs to consume and produce more detailed "synthetic" documentation/examples from. No idea if that's possible or even wise, but probably a problem space worth exploring. Or if these LLMs develop some sort of standardized way to rapidly apply new bodies of work that avoids costly retraining - like kernel modules, but for LLMs.
- cbg0 2y ago
  Since the current chatbots have the ability to tap into Google Search, it's not unlikely they could gather their own up to date documentation on-the-fly. This would create a slew of new attack vectors where malicious actors will try to add backdoors into the documentation, which the LLM would reproduce. A seasoned software engineer will easily pick it up, but the large amount of folks that are just copy pasting chatbot output to make their own apps will certainly miss it.
  - com2kid 2y ago
    With people using LLMs instead of googling, there isn't an incentive for people to write tutorial blog posts.
- logifail 2y ago
  > Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... Q: Could it be that those who aren't relying on ChatGPT (or similar) might have a significant competitive advantage?
  - kdmtctl 2y ago
    Just a head start. The LLM retraining cycle could probably become shorter and shorter over time.
efavdb 2y ago
Supporting anecdata: I was interested to see that chatgpt doesn’t know how to use one of my (small, not too popular) open source python packages — despite having blog posts and documentation on them, all from more than five years back.
curious_cat_163 2y ago
I like what FastHTML folks did by preparing what amounts to instructions for LMs. From their Github [1]: > This example is in a format based on recommendations from Anthropic for use with Claude Projects. This works so well that we’ve actually found that Claude can provide even better information than our own documentation! This is the file: https://docs.fastht.ml/llms-ctx.txt https://docs.fastht.ml/llms-ctx.txt [1] https://github.com/AnswerDotAI/fasthtml https://github.com/AnswerDotAI/fasthtml
- bckmn 2y ago
  This example is based on this proposed standard (making to robots.txt or security.txt): [The /llms.txt file – llms-txt](https://llmstxt.org/ https://llmstxt.org/)
- authorfly 2y ago
  I do not know how FastHTML works with AI, but for the recent Svelte 5, which has a similar llms.txt file, it's clear to me that actual usage patterns are required over explanatory llms.txt content with instructions. In Svelte 5 usage, it's consistent at certain things which the docs spell out for version 5 and would not be in the training data (runes) but not in their placement, reactivity, or usage changes implied by the system (e.g. changes of information flow outside of props/states) It seems similar to a cheat sheet which has formulas, vs a cheat sheet with worked through sample problems that align perfectly with the test. The latter exists for historic modules - the former is the best you can do for recent libraries/versions. I am not sure where React stands as I know they have changed their reactivity model and introduced patterns which reduce boilerplate code. Can anyone comment on the latest react version used with AI?
ra0x3 2y ago
+1 ChatGPT 4o still does not understand that React's ChakraUI is now on version 3 - which is incompatible with it's ChakraUI version 2 training data :')
- alphan0n 2y ago
  That hasn’t been my experience at all. Could you share some examples? > As of February 14, 2025, the latest version of Chakra UI for React is 3.8.0, released on February 9, 2025. This release introduced new hooks such as useElementRect, useForceUpdate, useLiveRef, usePrevious, and useSafeLayoutEffect, as well as a new FocusTrap component. Additionally, the Breadcrumb component received fixes for RTL support, and the Group component was updated to handle invalid children appropriately. https://chatgpt.com/share/67aff366-db20-8011-8549-861b1fc31c72 https://chatgpt.com/share/67aff366-db20-8011-8549-861b1fc31c...

spamizbad 2y ago

It seems self-evident it will, and it's largely self-reinforcing. Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... I do wonder if this problem could get solved by basically providing documentation explicitly written for LLMs to consume and produce more detailed "synthetic" documentation/examples from. No idea if that's possible or even wise, but probably a problem space worth exploring. Or if these LLMs develop some sort of standardized way to rapidly apply new bodies of work that avoids costly retraining - like kernel modules, but for LLMs.

cbg0 2y ago
Since the current chatbots have the ability to tap into Google Search, it's not unlikely they could gather their own up to date documentation on-the-fly. This would create a slew of new attack vectors where malicious actors will try to add backdoors into the documentation, which the LLM would reproduce. A seasoned software engineer will easily pick it up, but the large amount of folks that are just copy pasting chatbot output to make their own apps will certainly miss it.
- com2kid 2y ago
  With people using LLMs instead of googling, there isn't an incentive for people to write tutorial blog posts.
logifail 2y ago
> Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... Q: Could it be that those who aren't relying on ChatGPT (or similar) might have a significant competitive advantage?
- kdmtctl 2y ago
  Just a head start. The LLM retraining cycle could probably become shorter and shorter over time.

efavdb 2y ago

Supporting anecdata: I was interested to see that chatgpt doesn’t know how to use one of my (small, not too popular) open source python packages — despite having blog posts and documentation on them, all from more than five years back.

curious_cat_163 2y ago

I like what FastHTML folks did by preparing what amounts to instructions for LMs. From their Github [1]: > This example is in a format based on recommendations from Anthropic for use with Claude Projects. This works so well that we’ve actually found that Claude can provide even better information than our own documentation! This is the file: https://docs.fastht.ml/llms-ctx.txt https://docs.fastht.ml/llms-ctx.txt [1] https://github.com/AnswerDotAI/fasthtml https://github.com/AnswerDotAI/fasthtml

bckmn 2y ago
This example is based on this proposed standard (making to robots.txt or security.txt): [The /llms.txt file – llms-txt](https://llmstxt.org/ https://llmstxt.org/)
authorfly 2y ago
I do not know how FastHTML works with AI, but for the recent Svelte 5, which has a similar llms.txt file, it's clear to me that actual usage patterns are required over explanatory llms.txt content with instructions. In Svelte 5 usage, it's consistent at certain things which the docs spell out for version 5 and would not be in the training data (runes) but not in their placement, reactivity, or usage changes implied by the system (e.g. changes of information flow outside of props/states) It seems similar to a cheat sheet which has formulas, vs a cheat sheet with worked through sample problems that align perfectly with the test. The latter exists for historic modules - the former is the best you can do for recent libraries/versions. I am not sure where React stands as I know they have changed their reactivity model and introduced patterns which reduce boilerplate code. Can anyone comment on the latest react version used with AI?

ra0x3 2y ago

+1 ChatGPT 4o still does not understand that React's ChakraUI is now on version 3 - which is incompatible with it's ChakraUI version 2 training data :')

alphan0n 2y ago
That hasn’t been my experience at all. Could you share some examples? > As of February 14, 2025, the latest version of Chakra UI for React is 3.8.0, released on February 9, 2025. This release introduced new hooks such as useElementRect, useForceUpdate, useLiveRef, usePrevious, and useSafeLayoutEffect, as well as a new FocusTrap component. Additionally, the Breadcrumb component received fixes for RTL support, and the Group component was updated to handle invalid children appropriately. https://chatgpt.com/share/67aff366-db20-8011-8549-861b1fc31c72 https://chatgpt.com/share/67aff366-db20-8011-8549-861b1fc31c...