4 ms·

Correct me if I'm wrong but at a very high level, the same way Google has scraped/crawled probably "trillions" of web pages across the Internet, hasn't ChatGPT

Correct me if I'm wrong but at a very high level, the same way Google has scraped/crawled probably "trillions" of web pages across the Internet, hasn't ChatGPT basically been "trained" on almost the exact same data? aka just a bunch of text from the Internet that they index/train a "data model" versus serving it "a la carte" from a search engine index that's been highly optimized to focus on relevancy?

marginalia_nu 4y ago
Yeah, I don't see how that affects anything though. They're fundamentally doing different things with the data.
- MuffinFlavored 4y ago
  I think you are right. Google is basically telling you "ok, we're just a middleman, here's the source, go leave our site and visit this 3rd party site, read it and try to form your own summary, but keep in mind the results might be inaccurate, but since you are leaving our page, we aren't responsible for delivering the content to you anymore" whereas ChatGPT is doing... "here's a (potentially inaccurate) summary I was able to build from one/potentially multiple sources without telling you what those sources are, but since I'm serving you the summary, I'm basically responsible for potentially serving you inaccurate information that you may draw conclusions from/act on"
  - anileated 4y ago
    It does not attribute no matter whether credit or caution is due. Imagine if Google did this with their index (trained a model and just stopped giving you links), web masters would be up in arms.
    - marginalia_nu 4y ago
      Most likely it can't, since as I understand it ChatGPT synthesizes a response from the entire training set, rather than simply summarizing any particular document.
      - anileated 4y ago
        I don’t think it’s unattainable, intuitively it feels like a couple of extra layers to reflect on where the information might have come from could do it (as well as obviously capturing attribution along with the data). The cynic in me says they would rather keep it ambiguous and not investigate the possibility though. Attribution means everyone can ask for their fair share of the profits made by running the systems trained on the data they supply. Shareholder value and all that.
        marginalia_nu 4y ago
        Given it's synthetic, there may not be a specific location the information came from. Ask yourself, who told you Russia has invaded Ukraine? Maybe someone did at first, but even then, the notion of a war in Ukraine has been reinforced by thousands of sources since then. It doesn't come from any one of them, but the common picture they all paint. What they'd end up having to do is the very human act of "googling" for sources that agree with what they already believe. Which is not the same.
        anileated 4y ago
        > Ask yourself, who told you Russia has invaded Ukraine? Rapidly changing events aren’t a great example. > Given it's synthetic, there may not be a specific location the information came from. For bits of sufficiently commonly known information, sure. But if GPT quotes an exact or close enough snippet to something somebody written, and/or part of the topic has been mostly covered in depth by one or two authors, they can be attributed. (Not very unlike a human mind would do it.) I can’t see how attribution is that red line beyond which lies something unattainable by LLM. > What they'd end up having to do is the very human act of "googling" for sources that agree with what they already believe. Which is not the same. Not sure I got this part in the context…

marginalia_nu 4y ago

Yeah, I don't see how that affects anything though. They're fundamentally doing different things with the data.

MuffinFlavored 4y ago
I think you are right. Google is basically telling you "ok, we're just a middleman, here's the source, go leave our site and visit this 3rd party site, read it and try to form your own summary, but keep in mind the results might be inaccurate, but since you are leaving our page, we aren't responsible for delivering the content to you anymore" whereas ChatGPT is doing... "here's a (potentially inaccurate) summary I was able to build from one/potentially multiple sources without telling you what those sources are, but since I'm serving you the summary, I'm basically responsible for potentially serving you inaccurate information that you may draw conclusions from/act on"
- anileated 4y ago
  It does not attribute no matter whether credit or caution is due. Imagine if Google did this with their index (trained a model and just stopped giving you links), web masters would be up in arms.
  - marginalia_nu 4y ago
    Most likely it can't, since as I understand it ChatGPT synthesizes a response from the entire training set, rather than simply summarizing any particular document.
    - anileated 4y ago
      I don’t think it’s unattainable, intuitively it feels like a couple of extra layers to reflect on where the information might have come from could do it (as well as obviously capturing attribution along with the data). The cynic in me says they would rather keep it ambiguous and not investigate the possibility though. Attribution means everyone can ask for their fair share of the profits made by running the systems trained on the data they supply. Shareholder value and all that.
      - marginalia_nu 4y ago
        Given it's synthetic, there may not be a specific location the information came from. Ask yourself, who told you Russia has invaded Ukraine? Maybe someone did at first, but even then, the notion of a war in Ukraine has been reinforced by thousands of sources since then. It doesn't come from any one of them, but the common picture they all paint. What they'd end up having to do is the very human act of "googling" for sources that agree with what they already believe. Which is not the same.
        anileated 4y ago
        > Ask yourself, who told you Russia has invaded Ukraine? Rapidly changing events aren’t a great example. > Given it's synthetic, there may not be a specific location the information came from. For bits of sufficiently commonly known information, sure. But if GPT quotes an exact or close enough snippet to something somebody written, and/or part of the topic has been mostly covered in depth by one or two authors, they can be attributed. (Not very unlike a human mind would do it.) I can’t see how attribution is that red line beyond which lies something unattainable by LLM. > What they'd end up having to do is the very human act of "googling" for sources that agree with what they already believe. Which is not the same. Not sure I got this part in the context…