3 ms·

This is factually wrong, both in terms of quantity and quality. Current AI models are not "just sort of repeating and copying from memory". This is just an inc

This is factually wrong, both in terms of quantity and quality. Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform. AI skeptics often say things like this then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense.

pessimizer 4y ago
> Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform. You say this, but don't explain how. Because this is exactly what they are doing. > AI skeptics often say things like this I'm not really an AI skeptic. I think that we're very close to AI being indistinguishable from people. There are clearly problems that need to be solved, but I think the hardest problem was accepting the fact that humans are largely just copying and realizing that would be enough to get you 97% of the way there, especially if you gave a machine far more to copy than a human could consume. > then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense. Maybe I'm not expressing myself clearly, but it seems that you're just repeating my comment with a sneer. Agreeing angrily?
- mgraczyk 4y ago
  I'm disagreeing with the language you are using to characterize models. "copying from memory" implies that there is something being copied, and a memory that you are copying it from. I am pointing out that LLMs do not do this. It's not how they work. If you polled 1M random English speakers randomly and asked them whether or not a system that "just sort of repeating and copying from memory" could produce completely novel answers in response to completely novel questions, I suspect that the overwhelming majority would respond by saying no. Similarly if you asked 1000 people working on LLMs whether they work by "copying from memory", I suspect nearly all would say no. It would be accurate to say they are "generating text via a probabilistic model of language, which is encoded in the weights of a neural network", but there really is just no sense in which the models are "copying" anything. That being said, these models do "copy" some text in the sense that they can reconstruct some strings from their training input. For example every LLM I have played with can recite the first few paragraphs of A Tale of Two Cities verbatim. But that's a capability they have _in spite of_ their actual design, not because of it.
  - pessimizer 4y ago
    > I'm disagreeing with the language you are using to characterize models. "copying from memory" implies that there is something being copied, and a memory that you are copying it from. I am pointing out that LLMs do not do this. It's not how they work. Then we're arguing about the semantics of the word "copy." That is not an interesting argument when you know exactly what I mean and can express it clearly. edit: If it helps, either substitute your description in whenever I say 'pretty much copy' or change the word "copy" to whatever word you want to use. But even though I can't reproduce the opening paragraph to A Tale of Two Cities verbatim, I can certainly write something that is "copying" it without doing that, and anyone who was familiar with the book and read my paragraph would agree with me.
    - mgraczyk 4y ago
      It is semantics, but that was your whole point no? > That's because they're not modelling anything If we agree on "how LLMs work", then how can you claim that they aren't modeling anything? They are modeling language, and while it's unlikely current paradigms will be proving new mathematical truths, it's completely plausible to me that bigger models will be able to handle simple math word problems like those in the article, precisely because LLMs can model the "Alice", "Apple", and "Bob" entities.
      - pessimizer 4y ago
        I disagree that they are modeling language.* I think that not only bigger models but same-sized or much smaller models will be able to handle arbitrarily complicated word problems if they're eventually supplemented with some explicit model-building process. ----- * ...and that would be a completely semantic argument to have. I don't care whether it's called modeling, other than the fact that when I'm talking about modeling, I'm not talking about language probability, I'm talking about categories. But discussing what current AI is (a language model, copying?) is a waste of time, because I absolutely agree with your description of how it works, so we're talking about exactly the same thing.
  - deleted 4y ago
    [deleted]
sebastialonso 4y ago
can you actually share what "current AI models" are then? Not trying to be rude, but you just said "na ah" and then refused to argument any position.
- mgraczyk 4y ago
  Current LLMs are "modeling" something according to pretty much any sense of the word "model". In the technical, computational linguistics sense, LLMs are language models that give a conditional posterior distribution over sentences. Given some (constrained) context, the model tells you the posterior distribution over sentences in or around that context. In the nontechnical, layman sense of the word, they are a system that is used as an example of language. LLMs imitate language by generating new sentences. They are a "model" in the same way that an architectural model is a model, or in the same way that a statue is a model of a human. The other point I disagreed with is the characterization that LLMs "just sort of repeat and copy from memory". I went into more detail about that in other replies.
  - kcexn 4y ago
    A more layman way to describe it that avoids too much over simplification is that these learning models try to group things and apply probabilities to sequences of groupings. E.g.; A word is a grouping of letters, try to find the sequences of letters with the highest probabilities. A phrase is a grouping of words, with punctuation marks. Try to find the sequences of words with the highest probabilities. A sentence is a grouping of phrases. Try to find the highest probability sequences. A paragraph is a sequence of sentences. And so on and so on. Within very narrow domains (specific writing styles, say technical or legal writing), these models can be very accurate, since the sequencing of words into phrases, and phrases into sentences, and sentences into paragraphs etc., are very predictable. People call this kind of predictable sequencing a 'style', and it aids us in understanding text more quickly. More generally across all domains, it's much harder to accurately predict these sequences, because AI identify the 'style' of a text, purely from the text itself. No context surrounding the text is given to the AI, and so it guesses. For example: a political press release, will be written in one style of writing. And a company marketing press release will be written in a slightly different style of writing. As humans, we can easily distinguish between what is commercial marketing, and what is political, because we are given that information upfront. In latin (the choice language for some mathematicians and logicians for historical reasons), we have the information 'a priori'. A learning algorithm, isn't given that information up front, and must determine only from the text itself, whether it is more likely to be a marketing release selling some product, and therefore it should adopt a certain language style, or a political release selling an ideology and therefore should adopt a slightly different language style. When we don't know the right answer, and have no way to determine it, the solution that most computers are programmed to adopt is a minimax solution, i.e., minimise the maximum possible error. It does this by sort of mixing and matching both marketing and political styles. When a human reads it, sometimes it looks very strange and funny. Usually this is because it has some distinguishing feature, that we can immediately recognise as placing it as either a political or marketing document, i.e., a company name, a political party, a corporate or political letterhead, a famous person's name etc. The computer naturally doesn't know who Donald Trump is, since we haven't taught it who or what a Trump is, so it doesn't give it any precedence over any other word on the page. Actually, in the case of Donald Trump, I bet if you took the dates off of all of his tweets, even humans would have a hard time distinguishing if they were political or commercial in nature.
smeagull 4y ago
There is a reasoning loop that LLMs are clearly missing.

pessimizer 4y ago

> Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform. You say this, but don't explain how. Because this is exactly what they are doing. > AI skeptics often say things like this I'm not really an AI skeptic. I think that we're very close to AI being indistinguishable from people. There are clearly problems that need to be solved, but I think the hardest problem was accepting the fact that humans are largely just copying and realizing that would be enough to get you 97% of the way there, especially if you gave a machine far more to copy than a human could consume. > then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense. Maybe I'm not expressing myself clearly, but it seems that you're just repeating my comment with a sneer. Agreeing angrily?

mgraczyk 4y ago
I'm disagreeing with the language you are using to characterize models. "copying from memory" implies that there is something being copied, and a memory that you are copying it from. I am pointing out that LLMs do not do this. It's not how they work. If you polled 1M random English speakers randomly and asked them whether or not a system that "just sort of repeating and copying from memory" could produce completely novel answers in response to completely novel questions, I suspect that the overwhelming majority would respond by saying no. Similarly if you asked 1000 people working on LLMs whether they work by "copying from memory", I suspect nearly all would say no. It would be accurate to say they are "generating text via a probabilistic model of language, which is encoded in the weights of a neural network", but there really is just no sense in which the models are "copying" anything. That being said, these models do "copy" some text in the sense that they can reconstruct some strings from their training input. For example every LLM I have played with can recite the first few paragraphs of A Tale of Two Cities verbatim. But that's a capability they have _in spite of_ their actual design, not because of it.
- pessimizer 4y ago
  > I'm disagreeing with the language you are using to characterize models. "copying from memory" implies that there is something being copied, and a memory that you are copying it from. I am pointing out that LLMs do not do this. It's not how they work. Then we're arguing about the semantics of the word "copy." That is not an interesting argument when you know exactly what I mean and can express it clearly. edit: If it helps, either substitute your description in whenever I say 'pretty much copy' or change the word "copy" to whatever word you want to use. But even though I can't reproduce the opening paragraph to A Tale of Two Cities verbatim, I can certainly write something that is "copying" it without doing that, and anyone who was familiar with the book and read my paragraph would agree with me.
  - mgraczyk 4y ago
    It is semantics, but that was your whole point no? > That's because they're not modelling anything If we agree on "how LLMs work", then how can you claim that they aren't modeling anything? They are modeling language, and while it's unlikely current paradigms will be proving new mathematical truths, it's completely plausible to me that bigger models will be able to handle simple math word problems like those in the article, precisely because LLMs can model the "Alice", "Apple", and "Bob" entities.
    - pessimizer 4y ago
      I disagree that they are modeling language.* I think that not only bigger models but same-sized or much smaller models will be able to handle arbitrarily complicated word problems if they're eventually supplemented with some explicit model-building process. ----- * ...and that would be a completely semantic argument to have. I don't care whether it's called modeling, other than the fact that when I'm talking about modeling, I'm not talking about language probability, I'm talking about categories. But discussing what current AI is (a language model, copying?) is a waste of time, because I absolutely agree with your description of how it works, so we're talking about exactly the same thing.
- deleted 4y ago
  [deleted]

sebastialonso 4y ago

can you actually share what "current AI models" are then? Not trying to be rude, but you just said "na ah" and then refused to argument any position.

mgraczyk 4y ago
Current LLMs are "modeling" something according to pretty much any sense of the word "model". In the technical, computational linguistics sense, LLMs are language models that give a conditional posterior distribution over sentences. Given some (constrained) context, the model tells you the posterior distribution over sentences in or around that context. In the nontechnical, layman sense of the word, they are a system that is used as an example of language. LLMs imitate language by generating new sentences. They are a "model" in the same way that an architectural model is a model, or in the same way that a statue is a model of a human. The other point I disagreed with is the characterization that LLMs "just sort of repeat and copy from memory". I went into more detail about that in other replies.
- kcexn 4y ago
  A more layman way to describe it that avoids too much over simplification is that these learning models try to group things and apply probabilities to sequences of groupings. E.g.; A word is a grouping of letters, try to find the sequences of letters with the highest probabilities. A phrase is a grouping of words, with punctuation marks. Try to find the sequences of words with the highest probabilities. A sentence is a grouping of phrases. Try to find the highest probability sequences. A paragraph is a sequence of sentences. And so on and so on. Within very narrow domains (specific writing styles, say technical or legal writing), these models can be very accurate, since the sequencing of words into phrases, and phrases into sentences, and sentences into paragraphs etc., are very predictable. People call this kind of predictable sequencing a 'style', and it aids us in understanding text more quickly. More generally across all domains, it's much harder to accurately predict these sequences, because AI identify the 'style' of a text, purely from the text itself. No context surrounding the text is given to the AI, and so it guesses. For example: a political press release, will be written in one style of writing. And a company marketing press release will be written in a slightly different style of writing. As humans, we can easily distinguish between what is commercial marketing, and what is political, because we are given that information upfront. In latin (the choice language for some mathematicians and logicians for historical reasons), we have the information 'a priori'. A learning algorithm, isn't given that information up front, and must determine only from the text itself, whether it is more likely to be a marketing release selling some product, and therefore it should adopt a certain language style, or a political release selling an ideology and therefore should adopt a slightly different language style. When we don't know the right answer, and have no way to determine it, the solution that most computers are programmed to adopt is a minimax solution, i.e., minimise the maximum possible error. It does this by sort of mixing and matching both marketing and political styles. When a human reads it, sometimes it looks very strange and funny. Usually this is because it has some distinguishing feature, that we can immediately recognise as placing it as either a political or marketing document, i.e., a company name, a political party, a corporate or political letterhead, a famous person's name etc. The computer naturally doesn't know who Donald Trump is, since we haven't taught it who or what a Trump is, so it doesn't give it any precedence over any other word on the page. Actually, in the case of Donald Trump, I bet if you took the dates off of all of his tweets, even humans would have a hard time distinguishing if they were political or commercial in nature.

smeagull 4y ago

There is a reasoning loop that LLMs are clearly missing.