5 ms·

Well, that may be true but because it's a 'better UI' while also being completely independent of Stack Overflow, it has been hammering SO's traffic recently.

n4r9 3y ago
The reason stack overflow is such a useful resource is that people actually ask questions on it and receive high quality answers. I wonder, how long until the website becomes obsolete because people are asking LLMs instead, and how long after that do LLMs themselves consequently become less useful?
- manojlds 3y ago
  But LLMs are getting the "upvote" as feedback...more the usage, more they improve as well?
  - n4r9 3y ago
    I don't know how LLM upvotes work, but I'm guessing it's typically only the direct user that will upvote or downvote. The value of a public forum voting system is that a pool of experienced users can verify whether an answer is of good quality. If an inexperienced engineer asks for code and the LLM produces something that has sub-par performance, is overly complex, and botches various edge-cases, the engineer may not realise until years later if at all that they could have produced something much better. Combined with the fact that there are plenty of things that mostly only junior engineers will ask about, and I think you have a problem.
- PoignardAzur 3y ago
  > that people actually ask questions on it and receive high quality answers That really hasn't been my experience with StackOverflow for the last 10 years.
  - n4r9 3y ago
    Fair enough. Regardless, as I understand it LLMs can only be as good as their training data. Therefore they are vulnerable to that data losing its freshness.
    - famahar 3y ago
      Wouldn't it seem reasonable to assume that any new technology will train an LLM with extensive documentation and tutorials before being released. GPT4 can browse the web now too and open source LLMa can be trained on custom data.
      - n4r9 3y ago
        That is reasonable and I believe it would greatly help. However, docs and tutorials rarely anticipate all the ways in which people need to use that technology.
- ilaksh 3y ago
  How long? It started months ago. That's the fundamental reason why there was such a falling out between Stack Overflow executives and the moderators. People will say it was just the way they handled the interactions, which was part of it, but I don't believe there was any way to do it in a way that the users wouldn't hate. Because the basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. Because now they are already competing with ChatGPT etc. doing exactly that. It's true that taking away human answers from Stack Overflow will make it impossible for AI to answer many of those types of questions. But it will still retain fundamental knowledge and skills from other sources. And can be trained on its own debugging experiences as it gets deployed for building software. So although that presents a challenge, it's not insurmountable at all.
  - n4r9 3y ago
    > The basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. By going in that direction, I feel like they put the nail in the coffin. Why would I use their website when I can just ask ChatGPT directly? I think they had one other option, which was to identify the fundamental limitations of LLMs, and find a way to demonstrate value over and above.
    - cm2187 3y ago
      Plus where will chatgpt take new stuff from?
      - famahar 3y ago
        Whatever is new will probably be designed for chatgpt to understand first. Maybe people won't bother using your tool if it can't be used without a AI chat interface helping. The same way I won't learn a programming language if the documentation is bad. Tool makers knowing this will make chatgpt specific technical documents that can be easily parsed by an LLM.
        cm2187 3y ago
- throwuwu 3y ago
  LLMs can read the docs too
  - n4r9 3y ago
    That helps but may not be not enough imo. You can follow the docs to the tee while still producing overly complex, badly performing code which botches many edge cases.

n4r9 3y ago

The reason stack overflow is such a useful resource is that people actually ask questions on it and receive high quality answers. I wonder, how long until the website becomes obsolete because people are asking LLMs instead, and how long after that do LLMs themselves consequently become less useful?

manojlds 3y ago
But LLMs are getting the "upvote" as feedback...more the usage, more they improve as well?
- n4r9 3y ago
  I don't know how LLM upvotes work, but I'm guessing it's typically only the direct user that will upvote or downvote. The value of a public forum voting system is that a pool of experienced users can verify whether an answer is of good quality. If an inexperienced engineer asks for code and the LLM produces something that has sub-par performance, is overly complex, and botches various edge-cases, the engineer may not realise until years later if at all that they could have produced something much better. Combined with the fact that there are plenty of things that mostly only junior engineers will ask about, and I think you have a problem.
PoignardAzur 3y ago
> that people actually ask questions on it and receive high quality answers That really hasn't been my experience with StackOverflow for the last 10 years.
- n4r9 3y ago
  Fair enough. Regardless, as I understand it LLMs can only be as good as their training data. Therefore they are vulnerable to that data losing its freshness.
  - famahar 3y ago
    Wouldn't it seem reasonable to assume that any new technology will train an LLM with extensive documentation and tutorials before being released. GPT4 can browse the web now too and open source LLMa can be trained on custom data.
    - n4r9 3y ago
      That is reasonable and I believe it would greatly help. However, docs and tutorials rarely anticipate all the ways in which people need to use that technology.
ilaksh 3y ago
How long? It started months ago. That's the fundamental reason why there was such a falling out between Stack Overflow executives and the moderators. People will say it was just the way they handled the interactions, which was part of it, but I don't believe there was any way to do it in a way that the users wouldn't hate. Because the basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. Because now they are already competing with ChatGPT etc. doing exactly that. It's true that taking away human answers from Stack Overflow will make it impossible for AI to answer many of those types of questions. But it will still retain fundamental knowledge and skills from other sources. And can be trained on its own debugging experiences as it gets deployed for building software. So although that presents a challenge, it's not insurmountable at all.
- n4r9 3y ago
  > The basic task for Stack Overflow IS to replace the users answering the questions with AI. They don't have a choice. By going in that direction, I feel like they put the nail in the coffin. Why would I use their website when I can just ask ChatGPT directly? I think they had one other option, which was to identify the fundamental limitations of LLMs, and find a way to demonstrate value over and above.
  - cm2187 3y ago
    Plus where will chatgpt take new stuff from?
    - famahar 3y ago
      Whatever is new will probably be designed for chatgpt to understand first. Maybe people won't bother using your tool if it can't be used without a AI chat interface helping. The same way I won't learn a programming language if the documentation is bad. Tool makers knowing this will make chatgpt specific technical documents that can be easily parsed by an LLM.
      - cm2187 3y ago
throwuwu 3y ago
LLMs can read the docs too
- n4r9 3y ago
  That helps but may not be not enough imo. You can follow the docs to the tee while still producing overly complex, badly performing code which botches many edge cases.