3 ms·

What about the fact that OpenAI used Reddit as a dataset for training GPT? Seems like the data was valuable.

by BowBun 3y ago

capableweb 3y ago
What about the fact that reddit closing API access does in no way stop others from doing the same? There are multiple archives of reddit all over the place, and it hardly won't stop just because they make the API paid. Instead people will scrape the HTML and extract the data there, which will surely be more costly for reddit than the simple JSON API they have (for now).
- BowBun 3y ago
  What about it? You're arguing a point I haven't made. I was merely saying the data set is valuable or they wouldn't have trained on it. GP was saying the data was worthless.
journey_how 3y ago
This is a problem people have with macro, large scale data, the problem you are exhibiting in regards to "value". Ill use a simple example to illustrate. If you (yourself) were to write a song that sounded just like a Beatles song and was in effect a lift or "copy" of it and it went to #1 on the charts. You would expect a Beatles lawyer to contact you pretty quickly. Thats called "one to one" copyright infringement. Lawyers and court systems are setup to process these claims and do so from time to time. However, if you were to copy and make available with no royalty ALL THE SONG EVER WRITTEN AND RECORDED, the legal system, and lawyers have almost no way to get their head around it and the judicial system can barely assign a plaintiff to the defendant. This is more analagous to the situation Google created when it scanned "every book written in the english language" and made it available via google search. At the time the courts and the law had no concept of such a thing and couldnt process the mass copyright violation. Google didnt scan all the book s because they had "value". Google scanned all the books to "devalue" them to zero, to serve its purposes. Google as far as I know never paid to this day a cent. So ChatGPT didnt scan the entire internet (reddit was a rounding error), because it HAD value. It scanned it to form english grammer out of tokens and to get away with mass theft. Just like google. As an AI assistant, I hope I answered your question to your satisfaction.

What about the fact that OpenAI used Reddit as a dataset for training GPT? Seems like the data was valuable.

by BowBun 3y ago

What about the fact that OpenAI used Reddit as a dataset for training GPT? Seems like the data was valuable.

What about the fact that reddit closing API access does in no way stop others from doing the same? There are multiple archives of reddit all over the place, and it hardly won't stop just because they make the API paid. Instead people will scrape the HTML and extract the data there, which will surely be more costly for reddit than the simple JSON API they have (for now).

BowBun 3y ago
What about it? You're arguing a point I haven't made. I was merely saying the data set is valuable or they wouldn't have trained on it. GP was saying the data was worthless.

journey_how 3y ago

This is a problem people have with macro, large scale data, the problem you are exhibiting in regards to "value". Ill use a simple example to illustrate. If you (yourself) were to write a song that sounded just like a Beatles song and was in effect a lift or "copy" of it and it went to #1 on the charts. You would expect a Beatles lawyer to contact you pretty quickly. Thats called "one to one" copyright infringement. Lawyers and court systems are setup to process these claims and do so from time to time. However, if you were to copy and make available with no royalty ALL THE SONG EVER WRITTEN AND RECORDED, the legal system, and lawyers have almost no way to get their head around it and the judicial system can barely assign a plaintiff to the defendant. This is more analagous to the situation Google created when it scanned "every book written in the english language" and made it available via google search. At the time the courts and the law had no concept of such a thing and couldnt process the mass copyright violation. Google didnt scan all the book s because they had "value". Google scanned all the books to "devalue" them to zero, to serve its purposes. Google as far as I know never paid to this day a cent. So ChatGPT didnt scan the entire internet (reddit was a rounding error), because it HAD value. It scanned it to form english grammer out of tokens and to get away with mass theft. Just like google. As an AI assistant, I hope I answered your question to your satisfaction.