5 ms·

Data is valuable so for example; I don't understand why Reddit and Stack Overflow gave their valuable data to OpenAI and Google for pennies, when they could've

Data is valuable so for example; I don't understand why Reddit and Stack Overflow gave their valuable data to OpenAI and Google for pennies, when they could've made their own chatbots and beat OpenAI and Google at their own game.

pizza 2y ago
Stack Overflow did try this but imo their LLM wasn't that good
- mrkramer 2y ago
  So after few iterations they decided to give up and get the quick buck? So shortsighted from them.
  - solardev 2y ago
    Do they have much time left? Seems like once the answers are all scraped and trained, that site wouldn't be able to survive long anyway?
    - bandrami 2y ago
      The next version of libfoo released after 2023 will have a new set of options in /etc/foo.conf and at some point a human being who knows that will have to answer a question about it for an LLM to know that.
      - reducesuffering 2y ago
        No, future LLMs will ingest the codebase, any docs, and be able to answer the question anyway. That is, if the LLM didn’t generate the code base itself…
        KeplerBoy 2y ago
        There's a lot of wishful thinking going on here isn't it? All these models still regularly fail to provide me with correct code to use even the most common open source libraries (stuff like numpy and matplotlib), even though there were certainly trained on that code.
        Jensson 2y ago
        LLMs doesn't learn from reading manuals, LLMs learn by mimicking explanations it has seen made by humans who read the manuals. Until you solve that we will still need humans.
        bandrami 2y ago
        Except, no. Not unless docs and codebases are re-written in question and answer format
    - pizza 2y ago
      Something like 1/3rd of Google searches have never been asked before. Maybe something similar could be said of Stack Overflow questions? There could still be quite a need for human expertise. Tbh, without significant coaxing, LLMs tend to tailor their replies as if they were responding to the 'average' user - there could still be a lot of value for two 'non-average' users engaging in conversation.
Rucadi 2y ago
I assume they thought that OpenAI and Google would have used that data anyways without a clear way to prove otherwise.
marcinzm 2y ago
It's not their core business model and so far LLMs aren't very profitable. Investing a ton of money into a whole new business area with heavy competition that may be profitable eventually while not swimming in cash is often how companies die.
yreg 2y ago
StackOverflow is Creative Commons, so until courts/regulators decide otherwise, anyone can probably claim it's fair game to train on it, same as Wikipedia. Actually,
- yreg 2y ago
  The "Actually," is extra, but I can no longer edit it.
saintfire 2y ago
Apparently (according to AI enthusiasts) all publicly accessible data is free from copyright when used as training data. It doesn't really look like "owning" the data is worth very much money at all.
- astrange 2y ago
  Also according to eg German law, when done for research purposes.
asadotzler 2y ago
OpenAI would have stolen it if they didn't license it so it's "free money" for Reddit or Stack Overflow when OpenAI or any other comes along and offers money up front.
- barbariangrunge 2y ago
  It’s already stolen in a way from all the users who just wanted to be able to use a forum and had no serious alternatives

pizza 2y ago

Stack Overflow did try this but imo their LLM wasn't that good

mrkramer 2y ago
So after few iterations they decided to give up and get the quick buck? So shortsighted from them.
- solardev 2y ago
  Do they have much time left? Seems like once the answers are all scraped and trained, that site wouldn't be able to survive long anyway?
  - bandrami 2y ago
    The next version of libfoo released after 2023 will have a new set of options in /etc/foo.conf and at some point a human being who knows that will have to answer a question about it for an LLM to know that.
    - reducesuffering 2y ago
      No, future LLMs will ingest the codebase, any docs, and be able to answer the question anyway. That is, if the LLM didn’t generate the code base itself…
      - KeplerBoy 2y ago
        There's a lot of wishful thinking going on here isn't it? All these models still regularly fail to provide me with correct code to use even the most common open source libraries (stuff like numpy and matplotlib), even though there were certainly trained on that code.
      - Jensson 2y ago
        LLMs doesn't learn from reading manuals, LLMs learn by mimicking explanations it has seen made by humans who read the manuals. Until you solve that we will still need humans.
      - bandrami 2y ago
        Except, no. Not unless docs and codebases are re-written in question and answer format
  - pizza 2y ago
    Something like 1/3rd of Google searches have never been asked before. Maybe something similar could be said of Stack Overflow questions? There could still be quite a need for human expertise. Tbh, without significant coaxing, LLMs tend to tailor their replies as if they were responding to the 'average' user - there could still be a lot of value for two 'non-average' users engaging in conversation.

Rucadi 2y ago

I assume they thought that OpenAI and Google would have used that data anyways without a clear way to prove otherwise.

marcinzm 2y ago

It's not their core business model and so far LLMs aren't very profitable. Investing a ton of money into a whole new business area with heavy competition that may be profitable eventually while not swimming in cash is often how companies die.

yreg 2y ago

StackOverflow is Creative Commons, so until courts/regulators decide otherwise, anyone can probably claim it's fair game to train on it, same as Wikipedia. Actually,

yreg 2y ago
The "Actually," is extra, but I can no longer edit it.

saintfire 2y ago

Apparently (according to AI enthusiasts) all publicly accessible data is free from copyright when used as training data. It doesn't really look like "owning" the data is worth very much money at all.

astrange 2y ago
Also according to eg German law, when done for research purposes.

asadotzler 2y ago

OpenAI would have stolen it if they didn't license it so it's "free money" for Reddit or Stack Overflow when OpenAI or any other comes along and offers money up front.

barbariangrunge 2y ago
It’s already stolen in a way from all the users who just wanted to be able to use a forum and had no serious alternatives