5 ms·

Maybe I’m slow and stupid, can someone explain to me how new things will be introduced to new forms of LLMs if it all slowly becomes replaced by LLMs? It seems

Maybe I’m slow and stupid, can someone explain to me how new things will be introduced to new forms of LLMs if it all slowly becomes replaced by LLMs? It seems a bit like an ouroboros of human generated content consumption. Also, SO is flawed that’s without a doubt, but it still even in the age of LLMs been really useful to answer questions that are somewhat tangential not pure code. I was having a problem with FairPlay DRM for live streaming with iOS 16 and SO had something to point me in the right direction. Chatgpt being trained on older data didn’t have the answer. What happens in the future???

stocknoob 3y ago
New things will still be documented, which LLMs can digest. The documentation may have been auto-generated by a different LLM, no problem. Incorporating recent info into an LLM (fine tuning) is a solved problem.
- Vanit 3y ago
  This, and I believe this is what Phind does. https://www.phind.com/ https://www.phind.com/
- NtochkaNzvanova 3y ago
  If documentation were adequate, there wouldn't have been a need for Stack Overflow in the first place. At least for now, it still takes (human) intelligence to consume documentation and synthesize knowledge through experience with new technologies, at which point it can be communicated to others. LLMs may get to that point, but they aren't there yet. Also, good documentation explains not only the what and the how of a codebase (which LLMs may be able to summarize from the code), but also the why and the when (e.g. when to use one method or another).
  - gwoolhurme 3y ago
    The “why” is another great example too. When you document something you try to do so with some way of showing intent as well. While LLMs are so amazing (really). There is still a lot that I get curious about how it can be effectively used.
- gwoolhurme 3y ago
  Maybe, however there seems to be a limit to that no? I’m not talking about errors that are fixed or factual information. More like discoveries, ex: the FairPlay issue wasn’t documented from Apple it was discovered by multiple users who found and solved the issue I had. How does something like that exist with an LLM? There wasn’t really an error message that told you what to look for. It was a community effort of trial and error. Stackoverflow is nice because of the collaboration that can go on.
- professoretc 3y ago
  Do LLMs "learn" how to code by reading documentation, or by reading code? I was under the impression that it was more the latter; a programming language or library which was documented but had few or no examples of real use in the wild for the LLM to consume would (I think) be hard for current LLMs to work with. Unfortunately, I can't think of a real-world example (library where ChatGPT has probably seen the documentation, but not any source code) to try out.
p-e-w 3y ago
LLMs can already browse the web with the help of auxiliary systems, consuming current content. AFAIK GPT-4 has a browser plugin for that, as well as several other plugins for retrieving specific types of (current) information. A well-trained language model only needs to be retrained when the language changes. New facts can be fed to the model by simply telling it about them, which is how those "plugins" work behind the scenes.
- gwoolhurme 3y ago
  That still doesn’t answer my question on human to human discovery. Say a site like SO disappears because it’s not viable economically to keep it running. Then having something that scrapes the internet doesn’t seem to help does it? Like a quirk in hardware that a human notices, documents, fixes. Something low level.
  - p-e-w 3y ago
    How is that situation different from how it was before AI? If a human documents something and then the document becomes inaccessible, the information is gone, regardless of whether the "researcher" is an LLM or a human.
    - gwoolhurme 3y ago
      The problem I think I’m not conveying correctly is human to human discovery is made easier with communication hubs like SO. With something like an LLM if that reduces the viability for economically running a site like SO. Is there a gap or period where we don’t see that easy human to human communication? Eventually new generated content from a human has to enter the loop until the AI is up to a level where it can replace humans, but if we see less and less human generated content being viable then we have a clear chicken or the egg problem for new training data
      - afarviral 3y ago
        Is it specifically human-human contact thats required to generate new insights or document things? I see no major distinction between that and human-LLM contact (aside from awe) or simply an intelligence pusuing a goal by itself for that matter.

stocknoob 3y ago

New things will still be documented, which LLMs can digest. The documentation may have been auto-generated by a different LLM, no problem. Incorporating recent info into an LLM (fine tuning) is a solved problem.

Vanit 3y ago
This, and I believe this is what Phind does. https://www.phind.com/ https://www.phind.com/
NtochkaNzvanova 3y ago
If documentation were adequate, there wouldn't have been a need for Stack Overflow in the first place. At least for now, it still takes (human) intelligence to consume documentation and synthesize knowledge through experience with new technologies, at which point it can be communicated to others. LLMs may get to that point, but they aren't there yet. Also, good documentation explains not only the what and the how of a codebase (which LLMs may be able to summarize from the code), but also the why and the when (e.g. when to use one method or another).
- gwoolhurme 3y ago
  The “why” is another great example too. When you document something you try to do so with some way of showing intent as well. While LLMs are so amazing (really). There is still a lot that I get curious about how it can be effectively used.
gwoolhurme 3y ago
Maybe, however there seems to be a limit to that no? I’m not talking about errors that are fixed or factual information. More like discoveries, ex: the FairPlay issue wasn’t documented from Apple it was discovered by multiple users who found and solved the issue I had. How does something like that exist with an LLM? There wasn’t really an error message that told you what to look for. It was a community effort of trial and error. Stackoverflow is nice because of the collaboration that can go on.
professoretc 3y ago
Do LLMs "learn" how to code by reading documentation, or by reading code? I was under the impression that it was more the latter; a programming language or library which was documented but had few or no examples of real use in the wild for the LLM to consume would (I think) be hard for current LLMs to work with. Unfortunately, I can't think of a real-world example (library where ChatGPT has probably seen the documentation, but not any source code) to try out.

p-e-w 3y ago

LLMs can already browse the web with the help of auxiliary systems, consuming current content. AFAIK GPT-4 has a browser plugin for that, as well as several other plugins for retrieving specific types of (current) information. A well-trained language model only needs to be retrained when the language changes. New facts can be fed to the model by simply telling it about them, which is how those "plugins" work behind the scenes.

gwoolhurme 3y ago
That still doesn’t answer my question on human to human discovery. Say a site like SO disappears because it’s not viable economically to keep it running. Then having something that scrapes the internet doesn’t seem to help does it? Like a quirk in hardware that a human notices, documents, fixes. Something low level.
- p-e-w 3y ago
  How is that situation different from how it was before AI? If a human documents something and then the document becomes inaccessible, the information is gone, regardless of whether the "researcher" is an LLM or a human.
  - gwoolhurme 3y ago
    The problem I think I’m not conveying correctly is human to human discovery is made easier with communication hubs like SO. With something like an LLM if that reduces the viability for economically running a site like SO. Is there a gap or period where we don’t see that easy human to human communication? Eventually new generated content from a human has to enter the loop until the AI is up to a level where it can replace humans, but if we see less and less human generated content being viable then we have a clear chicken or the egg problem for new training data
    - afarviral 3y ago
      Is it specifically human-human contact thats required to generate new insights or document things? I see no major distinction between that and human-LLM contact (aside from awe) or simply an intelligence pusuing a goal by itself for that matter.