4 ms·

You usually expect people to cite sources. Granted, that very often doesn't happen, and the amount of citing expected depends on the context. But ChatGPT just d

by hannob 4y ago

You usually expect people to cite sources. Granted, that very often doesn't happen, and the amount of citing expected depends on the context. But ChatGPT just doesn't cite sources at all. I think there's a case to be made that they should.

pixl97 4y ago
Hello [Oxford Dictionary: 1827] Oh, wait, I'm not going to cite sources in a non-scientific work as this leads to madness. The following is a previous post of mine on HN "Your mind exists in a state where it is constantly 'scraping' copyrighted work. Now, in general limitations of the human mind keep you from accurately reproducing that work, but if I were able to look at your output as an omniscient being it is likely I could slam you with violation after violation where you took stylization ideas off of copyrighted work. RMS covers this rather well in 'The right to read'. Pretty much any model that puts hard ownership rules on ideas and styles leads to total ownership by a few large monied entities. It's much easier for Google to pay some artist for their data that goes into an AI model. Because the 'google ai' model is now more culturally complete than other models that cannot see this data Google entrenches a stronger monopoly in the market, hence generating more money in which to outright buy ideas to further monopolize the market."
ericd 4y ago
People don’t remember the sources that formed their opinions, it’s just baked into the structure of their brain after reading, same for the model.
- stevesearer 4y ago
  With search engines, it does feel like there was is a more clear trade of scraping access in exchange for web traffic. With ChatGPT the traffic benefit isn’t there, so it feels like it isn’t a fair trade. Google adding the context and data to their search results page also started blurring this trade making it unnecessary to click to the site the info was cleaned from.
  - williamcotton 4y ago
    How does someone site a source when they are using GPT to convert a box score into an entertaining paragraph about a baseball game? Or to convert a natural language command into a JSON format ready for downstream processing?
    - ericd 4y ago
      Right, it’s a gestalt from a huge set of sources. It’s not copying single text sources verbatim into your output.
rom-antics 4y ago
Humans have a pretty good sense of when you need to cite sources, and when you don't. For example, long ago I learned from some website how to write a for-loop in python, and now I write them all the time without giving credit. I'm okay with ChatGPT writing a for-loop without citing its source. I would say most knowledge about words/grammar/laws of nature can be taken for granted without a citation, but there are some important exceptions where things must be cited. I don't know how you'd reliably teach the difference to a computer though.
- candiodari 4y ago
  And yet, exactly in this example, I HATE that people don't put sources. Perhaps not for "for loops", but search anything simple in python. "Python JSON output", for example, and you will find a billion articles that describe a simple python library ... but DON'T link to python.org or the "javadoc". They're always dicussing the most blatantly obvious simple thing, never remotely complete, never link to where you can actually find more info (but jobs, courses, ads, ... those will be linked) It's getting me to the point of refusing to use Google, or only use Google with "site:...". I mean, the site varies, but without site limits Google's becoming useless.
christkv 4y ago
The bing leak seemed to mention sources.
lolinder 4y ago
ChatGPT doesn't have a concept of sources. It has weights that together define a function that allow it to guess the most likely next word from the context. As a neat side effect of this contextual next-word guessing, it often can share accurate information. If ChatGPT were to be required to share its sources, they would need a completely different approach. I'm not commenting on whether or not that would be a bad thing, but it would render the current iteration completely useless. You can't strap a source-crediting mechanism on top of a transformers-based model after the fact.
- anonymouskimmer 4y ago
  > You can't strap a source-crediting mechanism on top of a transformers-based model after the fact. I've read that ChatGPT is not connected to the net, but if it was: Couldn't you have it do a google search (or better yet corpus search) for the string it generated and then return the most significant matches (significance by string matching, not google rank)? It would be really crude, but wouldn't this just be a handful of lines of code that don't interfere with the "transformers-based model" code at all?
  - shagie 4y ago
    Why couldn't you, as a human do that to verify it? The other day I had GPT write a rap battle between Burger King and Ronald McDonald. One of the stanzas came back: Burger King: Your burgers are plain, your buns a bore. Your clown's been around since '63, I'm sure my flame-grilled taste will leave you impressed My burgers are fresh, my fries are the best It turns out that yes, Ronald McDonald was first introduced in 1963. https://en.wikipedia.org/wiki/File:McDonald%27s_commercial_(1963).webm https://en.wikipedia.org/wiki/File:McDonald%27s_commercial_(... (from https://en.wikipedia.org/wiki/Willard_Scott#Created_Ronald_McDonald,_the_Hamburger-Happy_Clown https://en.wikipedia.org/wiki/Willard_Scott#Created_Ronald_M... ) So here's the challenge for you - who do you compensate for that line? The complaint that people have isn't that GPT isn't citing its sources but rather that it isn't compensating the people who created the data that has that information. ... and now, if you're ever asked about historical clown trivia and pull out the "Ronald has been around since 1963", who should you give a royalty to? Me (for writing this), GPT (for making me aware of it), Wikipedia (for the source of my links in this post), the estate of Willard Scott for the Joy of Living (which Wikipedia cites), some random blog author that had some clown trivia on it that happened to have been part of the training set for GPT?

by hannob 4y ago

pixl97 4y ago

Hello [Oxford Dictionary: 1827] Oh, wait, I'm not going to cite sources in a non-scientific work as this leads to madness. The following is a previous post of mine on HN "Your mind exists in a state where it is constantly 'scraping' copyrighted work. Now, in general limitations of the human mind keep you from accurately reproducing that work, but if I were able to look at your output as an omniscient being it is likely I could slam you with violation after violation where you took stylization ideas off of copyrighted work. RMS covers this rather well in 'The right to read'. Pretty much any model that puts hard ownership rules on ideas and styles leads to total ownership by a few large monied entities. It's much easier for Google to pay some artist for their data that goes into an AI model. Because the 'google ai' model is now more culturally complete than other models that cannot see this data Google entrenches a stronger monopoly in the market, hence generating more money in which to outright buy ideas to further monopolize the market."

ericd 4y ago

People don’t remember the sources that formed their opinions, it’s just baked into the structure of their brain after reading, same for the model.

stevesearer 4y ago
With search engines, it does feel like there was is a more clear trade of scraping access in exchange for web traffic. With ChatGPT the traffic benefit isn’t there, so it feels like it isn’t a fair trade. Google adding the context and data to their search results page also started blurring this trade making it unnecessary to click to the site the info was cleaned from.
- williamcotton 4y ago
  How does someone site a source when they are using GPT to convert a box score into an entertaining paragraph about a baseball game? Or to convert a natural language command into a JSON format ready for downstream processing?
  - ericd 4y ago
    Right, it’s a gestalt from a huge set of sources. It’s not copying single text sources verbatim into your output.

rom-antics 4y ago

Humans have a pretty good sense of when you need to cite sources, and when you don't. For example, long ago I learned from some website how to write a for-loop in python, and now I write them all the time without giving credit. I'm okay with ChatGPT writing a for-loop without citing its source. I would say most knowledge about words/grammar/laws of nature can be taken for granted without a citation, but there are some important exceptions where things must be cited. I don't know how you'd reliably teach the difference to a computer though.

candiodari 4y ago
And yet, exactly in this example, I HATE that people don't put sources. Perhaps not for "for loops", but search anything simple in python. "Python JSON output", for example, and you will find a billion articles that describe a simple python library ... but DON'T link to python.org or the "javadoc". They're always dicussing the most blatantly obvious simple thing, never remotely complete, never link to where you can actually find more info (but jobs, courses, ads, ... those will be linked) It's getting me to the point of refusing to use Google, or only use Google with "site:...". I mean, the site varies, but without site limits Google's becoming useless.

christkv 4y ago

The bing leak seemed to mention sources.

lolinder 4y ago

ChatGPT doesn't have a concept of sources. It has weights that together define a function that allow it to guess the most likely next word from the context. As a neat side effect of this contextual next-word guessing, it often can share accurate information. If ChatGPT were to be required to share its sources, they would need a completely different approach. I'm not commenting on whether or not that would be a bad thing, but it would render the current iteration completely useless. You can't strap a source-crediting mechanism on top of a transformers-based model after the fact.

anonymouskimmer 4y ago
> You can't strap a source-crediting mechanism on top of a transformers-based model after the fact. I've read that ChatGPT is not connected to the net, but if it was: Couldn't you have it do a google search (or better yet corpus search) for the string it generated and then return the most significant matches (significance by string matching, not google rank)? It would be really crude, but wouldn't this just be a handful of lines of code that don't interfere with the "transformers-based model" code at all?
- shagie 4y ago
  Why couldn't you, as a human do that to verify it? The other day I had GPT write a rap battle between Burger King and Ronald McDonald. One of the stanzas came back: Burger King: Your burgers are plain, your buns a bore. Your clown's been around since '63, I'm sure my flame-grilled taste will leave you impressed My burgers are fresh, my fries are the best It turns out that yes, Ronald McDonald was first introduced in 1963. https://en.wikipedia.org/wiki/File:McDonald%27s_commercial_(1963).webm https://en.wikipedia.org/wiki/File:McDonald%27s_commercial_(... (from https://en.wikipedia.org/wiki/Willard_Scott#Created_Ronald_McDonald,_the_Hamburger-Happy_Clown https://en.wikipedia.org/wiki/Willard_Scott#Created_Ronald_M... ) So here's the challenge for you - who do you compensate for that line? The complaint that people have isn't that GPT isn't citing its sources but rather that it isn't compensating the people who created the data that has that information. ... and now, if you're ever asked about historical clown trivia and pull out the "Ronald has been around since 1963", who should you give a royalty to? Me (for writing this), GPT (for making me aware of it), Wikipedia (for the source of my links in this post), the estate of Willard Scott for the Joy of Living (which Wikipedia cites), some random blog author that had some clown trivia on it that happened to have been part of the training set for GPT?