5 ms·

It's almost as if we trained LLMs on text produced by people.

by lxe 2y ago

MrMcCall 2y ago
I love the posters that make fun of those corporate motivational posters. My favorite is: No one is as dumb as all of us. And they trained their PI* on that giant turd pile. * Pseudo Intelligence
- LoganDark 2y ago
  I don't count LLMs as intelligent. To a certain degree they can be a component of intelligence, but I don't count an LLM on its own.
  - tavavex 2y ago
    Artificial intelligence is a generic term for a very broad field that has existed for like 50-70 years, depending on who you ask. 'Intelligence' isn't praise or endorsement. I think it's a succinct word that does the job at explaining what the goal here is. All the "Artificial intelligence? Hah, more like Bad Unintelligence, am I right???" takes just sound so corny to me.
    - callc 2y ago
      > I think it's a succinct word that does the job at explaining what the goal here is. Sure. If the goal is intelligence then LLMs fail. LLMs do not currently have the same intelligence as humans. If a human being in front of me were to answer my question like an LLM does, I would think they are an overly confident parrot. Not saying LLMs are bad, they are an incredible tool. Just not intelligence. Words matter.
      - tavavex 2y ago
        Who said anything about matching human intelligence though? If that's the reference point, then nothing in the field of AI has ever had the 'right' to be called that. Computer vision, ML-based optimization of anything or ranking/recommendation systems are all considered to be within AI, despite none of them being remotely similar to 'human intelligence'. This is the main point of my post - I feel like people retroactively try to see AI as being some kind of an endorsement term, or having to do anything regarding humans - or that 'intelligence' is in itself an endorsement and something so extremely good that only humans can be bestowed with it. In reality, these comparisons only appeared after the boom of generative AI and would've been seen as ludicrous by any AI researchers prior to it.
    - LoganDark 2y ago
      I don't mean to sound corny. LLMs just don't really use or apply information in a way that I think should be considered intelligent. It just repeats its training data. I don't just repeat my training data (even if it was an influence on me)
      - antonvs 2y ago
        The idea that LLMs just repeat their training data is just wrong. It’s easy to test them and prove this is not the case. In some situations they may do that, typically when they don’t have much data on some topic. But in many other cases, it’s easy to verify that they are able to synthesize new output that is not simply a repetition of their training data. Software development is a great example, which also illustrates the ability of LLMs to reason (whether you want to call it e.g. “simulating reasoning” doesn’t matter - the results are what counts.) They can design new programs, write new code, debug code they’ve never seen before, and explain code they’ve never seen before. None of that would be possible if they were simply repeating their training data.
        LoganDark 2y ago
        If you've implemented a sampler before, the "repeating the training data" is technically the logits array that you do the sampling on. Good samplers and sometimes even the most basic samplers can produce acceptable output but in the end the output is still technically just a bunch of training data predictions averaged together... or something roughly like that. The fact that I don't consider them intelligent doesn't mean I don't find them useful, I just prefer to use them in a way that acknowledges their shortcomings.
        antonvs 2y ago
        First, to be clear, I'm not arguing that you should consider LLMs intelligent. I was responding more narrowly to the claim that an LLM "just repeats its training data." On a trivial level, it's obviously true that every token in an LLM's output must have existed in the training data. But that's as far as your observation goes. The point is that LLMs can produce novel sequences of tokens that exhibit the functional equivalent of "understanding" of the input and the expected output. Further, however they achieve it, functionally their output compares well to output that has been produced by a reasoning process. None of this would be expected if they simply repeated "a bunch of training data predictions averaged together... or something roughly like that." For example, if that were all that was happening, you couldn't reasonably expect them to respond to a prompt with decent, working new code that's fit for purpose. They would produce code that looks plausible, but that doesn't compile, or run, or do what was intended. One reason your model of the process fails to capture what's happening is because it's not taking into account the effects of latent space embeddings, and the resulting relationships between token representations. This is a major part of what enables an LLM to generalize and produce "correct" output, taking meaning into account, beyond simply repeating its training data. As for intelligence - again the question comes down to functional equivalence. If we use traditional ways of measuring intelligence, like IQ tests, then LLMs beat the average human. Of course, that's not as significant as might naively be imagined, but it hints at the problem: how do you define intelligence, and on what basis are you claiming an LLM doesn't have it? Ultimately, it's a question of definitions, and I suspect it'd actually be quite difficult to give a rigorous (non-handwavy) definition of intelligence that an LLM can't satisfy. This may partly be an indictment of our understanding of intelligence.
    - oneoverten 2y ago
      I think it's more that as the term is widely adopted via something like LLMs they convey different meaning to users of the tools branded by it. Since users and their perspective of "artificial intelligence" and its meaning have no relation to the original term from 50-70 years ago.
      - tavavex 2y ago
        Yes, I agree that the rise of gAI has had a major impact on this, but the fact that the meaning of AI in CS and AI in sci-fi are different was also very apparent before all this happened. Marketing and finance people have been trying to associate the hard-math, grounded-in-reality AI with "it's magical, it's smart like a human!" many times before, this time they just were very successful at it.
        oneoverten 2y ago
        Yes indeed, I think you are completely right on that
  - MrMcCall 2y ago
    Well said. If the database they're built with is well curated and the queries run against them make sense, then I imagine they could be a very, very good kind of local search engine. But, training one on twitter and reddit comments? Yikes!

It's almost as if we trained LLMs on text produced by people.

by lxe 2y ago

It's almost as if we trained LLMs on text produced by people.

I love the posters that make fun of those corporate motivational posters. My favorite is: No one is as dumb as all of us. And they trained their PI* on that giant turd pile. * Pseudo Intelligence

LoganDark 2y ago
I don't count LLMs as intelligent. To a certain degree they can be a component of intelligence, but I don't count an LLM on its own.
- tavavex 2y ago
  Artificial intelligence is a generic term for a very broad field that has existed for like 50-70 years, depending on who you ask. 'Intelligence' isn't praise or endorsement. I think it's a succinct word that does the job at explaining what the goal here is. All the "Artificial intelligence? Hah, more like Bad Unintelligence, am I right???" takes just sound so corny to me.
  - callc 2y ago
    > I think it's a succinct word that does the job at explaining what the goal here is. Sure. If the goal is intelligence then LLMs fail. LLMs do not currently have the same intelligence as humans. If a human being in front of me were to answer my question like an LLM does, I would think they are an overly confident parrot. Not saying LLMs are bad, they are an incredible tool. Just not intelligence. Words matter.
    - tavavex 2y ago
      Who said anything about matching human intelligence though? If that's the reference point, then nothing in the field of AI has ever had the 'right' to be called that. Computer vision, ML-based optimization of anything or ranking/recommendation systems are all considered to be within AI, despite none of them being remotely similar to 'human intelligence'. This is the main point of my post - I feel like people retroactively try to see AI as being some kind of an endorsement term, or having to do anything regarding humans - or that 'intelligence' is in itself an endorsement and something so extremely good that only humans can be bestowed with it. In reality, these comparisons only appeared after the boom of generative AI and would've been seen as ludicrous by any AI researchers prior to it.
  - LoganDark 2y ago
    I don't mean to sound corny. LLMs just don't really use or apply information in a way that I think should be considered intelligent. It just repeats its training data. I don't just repeat my training data (even if it was an influence on me)
    - antonvs 2y ago
      The idea that LLMs just repeat their training data is just wrong. It’s easy to test them and prove this is not the case. In some situations they may do that, typically when they don’t have much data on some topic. But in many other cases, it’s easy to verify that they are able to synthesize new output that is not simply a repetition of their training data. Software development is a great example, which also illustrates the ability of LLMs to reason (whether you want to call it e.g. “simulating reasoning” doesn’t matter - the results are what counts.) They can design new programs, write new code, debug code they’ve never seen before, and explain code they’ve never seen before. None of that would be possible if they were simply repeating their training data.
      - LoganDark 2y ago
        If you've implemented a sampler before, the "repeating the training data" is technically the logits array that you do the sampling on. Good samplers and sometimes even the most basic samplers can produce acceptable output but in the end the output is still technically just a bunch of training data predictions averaged together... or something roughly like that. The fact that I don't consider them intelligent doesn't mean I don't find them useful, I just prefer to use them in a way that acknowledges their shortcomings.
        antonvs 2y ago
        First, to be clear, I'm not arguing that you should consider LLMs intelligent. I was responding more narrowly to the claim that an LLM "just repeats its training data." On a trivial level, it's obviously true that every token in an LLM's output must have existed in the training data. But that's as far as your observation goes. The point is that LLMs can produce novel sequences of tokens that exhibit the functional equivalent of "understanding" of the input and the expected output. Further, however they achieve it, functionally their output compares well to output that has been produced by a reasoning process. None of this would be expected if they simply repeated "a bunch of training data predictions averaged together... or something roughly like that." For example, if that were all that was happening, you couldn't reasonably expect them to respond to a prompt with decent, working new code that's fit for purpose. They would produce code that looks plausible, but that doesn't compile, or run, or do what was intended. One reason your model of the process fails to capture what's happening is because it's not taking into account the effects of latent space embeddings, and the resulting relationships between token representations. This is a major part of what enables an LLM to generalize and produce "correct" output, taking meaning into account, beyond simply repeating its training data. As for intelligence - again the question comes down to functional equivalence. If we use traditional ways of measuring intelligence, like IQ tests, then LLMs beat the average human. Of course, that's not as significant as might naively be imagined, but it hints at the problem: how do you define intelligence, and on what basis are you claiming an LLM doesn't have it? Ultimately, it's a question of definitions, and I suspect it'd actually be quite difficult to give a rigorous (non-handwavy) definition of intelligence that an LLM can't satisfy. This may partly be an indictment of our understanding of intelligence.
  - oneoverten 2y ago
    I think it's more that as the term is widely adopted via something like LLMs they convey different meaning to users of the tools branded by it. Since users and their perspective of "artificial intelligence" and its meaning have no relation to the original term from 50-70 years ago.
    - tavavex 2y ago
      Yes, I agree that the rise of gAI has had a major impact on this, but the fact that the meaning of AI in CS and AI in sci-fi are different was also very apparent before all this happened. Marketing and finance people have been trying to associate the hard-math, grounded-in-reality AI with "it's magical, it's smart like a human!" many times before, this time they just were very successful at it.
      - oneoverten 2y ago
        Yes indeed, I think you are completely right on that
- MrMcCall 2y ago
  Well said. If the database they're built with is well curated and the queries run against them make sense, then I imagine they could be a very, very good kind of local search engine. But, training one on twitter and reddit comments? Yikes!