6 ms·

LLMs are excellent at text transformation. It's their core strength and I don't see it being used enough.

PaulHoule 2y ago
Particularly those that are basically linear, that don’t involve major changes in the order of things or a deep consideration of relationships between things. They can’t sort a list but they can translate languages, for instance, given that a list sorted almost right is wrong but that we will often settle for an almost right translation.
msp26 2y ago
Isn't this already their main use case for business? We use them primarily for extracting structured data from other forms.
xanderlewis 2y ago
It’s not only their core strength — it’s what transformers were designed to do and, arguably, it’s all they can do. Any other supposed ability to reason or even retain knowledge (rather than simply regurgitate text without ‘understanding’ its intended meaning) is just a side effect of this superhuman ability.
- stavros 2y ago
  I see your point, but I think there's more to it. It's kind of like saying "all humans can do is perceive and produce sound, any other ability is just a side-effect". We might be focusing too much on their mechanism for "perception" and overlooking other capabilities they've developed.
  - mjburgess 2y ago
    Sure, but that claim wouldn't be true for humans, right? So it's a nonsequiteur. The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using inherited systems of meaning acquired as very young children in adaption within their environments, etc. In the case of transformers all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. If you project both of these into "problems an office worker has"-space, then they can appear simimlar -- but this projection is an incredibly dumb one, and offered as a sales pitch by charlatans looking to pretend that a system which can generate office emails can communicate.
    - diego_sandoval 2y ago
      > all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. I don't think that's all they can do. I think they know more than what is explicitly stated in their training sets. They can generalize knowledge and generalize relationships between the concepts that are in the training sets. They're currently mediocre at it, but the results we observe from SOTA generative models are not explainable without accepting that they can create an internal model of the world that's more than just a decompression algorithm. I'm going to step away from LLMs for a moment, but: How are video generator models capable of creating videos with accurate shadows and lighting that is consistent in the entire frame and consistent between frames? You can't do that simply by taking a weighted average of the sections of videos you've seen in your training set. You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own". I think the same principle applies to LLMs.
      - carlmr 2y ago
        >You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own". Compression is understanding. If you have a model which explains shadows you can compress your video data much better. Since you "understand" how shadows work.
    - fenomas 2y ago
      > all they can do is, in fact, sample from a compression of historical texts To me, results like the Othello paper make any sort of "stochastic parrot" thinking completely untenable. https://thegradient.pub/othello/ https://thegradient.pub/othello/
      - samatman 2y ago
        This result is an argument for the conclusion you are reading it as arguing against.
  - xanderlewis 2y ago
    Well, yes — absolutely. You could say something similar about any system with complex emergent behaviour. 'All computers can do are NAND operations and any other ability is just a side effect', or something. However, I do think that in this case it's meaningful. The claim isn't that LLMs are genuinely exhibiting reasoning ability — I think it's quite clear to anyone who probes them for long enough that they're not. I was fooled initially too, but you soon come to realise it's a clever trick (albeit not one contrived by any of the human designers themselves). The claim is usually some pseudo-philosophical claim that the very definition of reasoning is simply 'outputting (at least some of the time) correct sentences' and so there's no more to be said. But this is just silly. It's quite obvious that being able to manipulate language and effectively have access to a vast (fuzzily encoded) database of knowledge will mean you can output true and pertinent statements a lot of the time. But this doesn't require reasoning at all. Note that I'm not claiming that LLMs exhibit reasoning and other abilities 'as a side effect' of language manipulation ability — I'm claiming there's no reason to believe they have these abilities at all based on the available evidence. Humans are just very easily convinced by beings that seem to speak our language and are overly inclined to attribute all sorts of desires, internal thought processes and whatever else for which there are no evidence.
    - famouswaffles 2y ago
      >I think it's quite clear to anyone who probes them for long enough that they're not. I disagree and so do a lot of people who've used them for a long while. This is just an assertion that you wish to be true rather than something that actually is. What happens is that for some bizarre reason, for machines, lots of humans have a standard of reasoning that only exists in fiction. Devise any reasoning test you like that would cleanly separate humans from LLMs. I'll wait. > The claim is usually some pseudo-philosophical claim that the very definition of reasoning is simply 'outputting (at least some of the time) correct sentences' and so there's no more to be said. There is nothing philosophical or pseudo-philosophical about saying reasoning is determined by output. If anything, the opposite is what's philosophical nonsense. The idea that there exists some "real" reasoning that humans perform and "fake" reasoning that LLMs perform and yet somehow no testable way to distinguish this is purely the realm of fiction and philosophy. If you're claiming a distinction that doesn't actually distinguish, you're just making stuff up. LLMs clearly reason. They do things, novel things that no sane mind would see a human do and call anything else. They do things that are impossible to describe as anything else unless you subscribe to what i like to call statistical magic - https://news.ycombinator.com/item?id=41141118 https://news.ycombinator.com/item?id=41141118 And all things considered, LLMs are pretty horrible memorizers. Getting one to regurgitate Training data is actually really hard. There's no database of knowledge. It clearly does not work that way.
- baq 2y ago
  > it’s all they can do this overlooks how they do it. we don't really know. it might be logical reasoning, it might be a very efficient content addressable human-knowledge-in-a-blob-of-numbers lookup table... it doesn't matter if they work, which they do, sometimes scarily well. dismissing their abilities because they 'don't reason' is missing the forest for the trees in that they'd be capable of reasoning if they were able to run sat solvers on their output mid generation.
  - SiempreViernes 2y ago
    Dismissing claims that LLMs "reason" because these machines perform no actions similar to reasoning seems pretty motivated. And I don't think "blindly take input from a reasoning capable system" counts as reasoning.
    - hobs 2y ago
      Does it? I think Blindsight (the book) had a good commentary on reason being a thing we think is a conscious process but doesn't have to be. I think most people talking past each other are really discussing whether the GPT is conscious, has a mental model of self, that kind of thing, as long as your definition of reasoning doesn't include consciousness it clearly does it (though not well.)
    - foobarbecue 2y ago
      "pretty motivated"? Did you mean biased?
      - ChadNauseam 2y ago
        I assume they meant motivated as shorthand for "motivated reasoning" which implies a bias that's motivating them to reason a certain way
        foobarbecue 2y ago
        Oh, interesting -- that's a new one for me.
- sitkack 2y ago
  Hinton claims they do reason. I am going to go with Hinton on this.
  - xanderlewis 2y ago
    Hinton's opinions on LLMs are frankly bonkers. Just because you're famous — and intelligent and successful — doesn't mean you can't be completely wrong. Also: what's his rationale? It's no use simply claiming something without evidence. And as far as I (and seemingly most others) can see, there's no such evidence other than that they can sometimes output sentences that happen to be true. But so can Wikipedia — does that mean Wikipedia is reasoning? Also, any form of reasoning in the usual sense of the word would surely require the ability to allocate arbitrary amounts of computation (i.e. thought) to each question. LLMs don't do this — they don't sit and ponder; each token takes exactly the same amount of computation to produce. Once they hit an 'end of text' token, they're done. Even empirically speaking, LLMs' ability to reason can be seen to be nonexistent. Just try asking basic mathematics questions. As soon as you ask anything for which the answer isn't available — practically verbatim — on the web already, it produces intelligent-sounding gibberish. This whole idea that 'LLMs must be able to reason because in order to learn to fake reasoning you must learn to actually reason' is like some kind of inverted no true Scotsman fallacy.
    - sitkack 2y ago
      Does slime mold reason? Yes, Hinton can be wrong, is wrong on many things like his misunderstanding on Chomsky and language. But I also think he has spent thousands of hours testing these systems scientifically. Your last sentence puts a lot of words in peoples mouths. But to continue down that line, fake reasoning and actual reasoning sounds like the Chinese Room. Is that the argument you are making? We don't understand our own mental processes well enough, so I try to not anthropomorphize reasoning and cognition.
      - xanderlewis 2y ago
        > Your last sentence puts a lot of words in peoples mouths. Well, it’s the most common sentiment I see on both here and (before I gave up) the AI-centred parts of reddit. It’s not quite the Chinese Room, since LLMs can’t even simulate reasoning very well. So there’s no need to debate the distinction between ‘fake reasoning and actual reasoning’ — there may or may not be a difference, but it’s not the point I’m making. As for Hinton: I’m sure he has. But inventors are often not experts on their own creations/discoveries, and are probably just as prone to FUD and panic in the face of surprising developments as the rest of us. No one predicted that autoregressive transformers would get us this far, least of all the experts whose decades of work lead us to this point.
worldsayshi 2y ago
One potential benefit should be that with the right tooling around it it should be able to translate your code base to a different language and/or framework more or less at the push of a button. So if a team is wondering if it would be worth it to switch a big chunk of the code base from python to elixir they don't have to wonder anymore. I tried translating a python script to javascript the other day and it was flawless. I would expect it to scale with a bit of hand-railing.
- adamdiy 2y ago
  see projects like https://github.com/joshpxyne/gpt-migrate https://github.com/joshpxyne/gpt-migrate think there's also a YC company recently focusing on the nasty, big migrations with LLM help
  - worldsayshi 2y ago
    It seems that this kind of application can really change how the tech industry can evolve down the line. Maybe we will more quickly converge on tech stacks if everyone can test new one's out "within a week".
- scarface_74 2y ago
  ChatGPT is trained well enough on all things AWS that it can do a decent job translating Python based SDK code to Node and other languages, translate between CloudFormation/Terraform/CDK (in various languages). It does a well at writing simple to medium complexity automation scripts around AWS. If it gets something wrong, I tell it to “verify your answer using the documentation available on the web”
  - kamaal 2y ago
    >>ChatGPT is trained well enough on all things AWS It was scary to me how to chatting with GPT or Claude would give me information which was a lot more clear than what I could deduce after hours of reading AWS documentation. Perhaps, the true successor to Google search has arrived. One big drawback of Google was asking questions that can't be converted to a full long conversation. To that end. LLM chat is the ultimate socratic learning method tool till date.
greenthrow 2y ago
The problem is the use case is where you don't care about the risk of hallucinations or you can validate the output without already having the data in a useful format. Plus you need to lack the knowledge/skill to do it more quickly using awk/python/perl/whatever.
- worldsayshi 2y ago
  That's why having good test suites and tools are more important than ever.
- j_maffe 2y ago
  I think text transformation is a sufficiently predictable task that one could make a transformer that completely avoids hallucinations. Most LLMs have high temperatures which introduces randomness and therefore hallucinations into the result.
deleted 2y ago
[deleted]

Particularly those that are basically linear, that don’t involve major changes in the order of things or a deep consideration of relationships between things. They can’t sort a list but they can translate languages, for instance, given that a list sorted almost right is wrong but that we will often settle for an almost right translation.

msp26 2y ago

Isn't this already their main use case for business? We use them primarily for extracting structured data from other forms.

xanderlewis 2y ago

It’s not only their core strength — it’s what transformers were designed to do and, arguably, it’s all they can do. Any other supposed ability to reason or even retain knowledge (rather than simply regurgitate text without ‘understanding’ its intended meaning) is just a side effect of this superhuman ability.

stavros 2y ago
I see your point, but I think there's more to it. It's kind of like saying "all humans can do is perceive and produce sound, any other ability is just a side-effect". We might be focusing too much on their mechanism for "perception" and overlooking other capabilities they've developed.
- mjburgess 2y ago
  Sure, but that claim wouldn't be true for humans, right? So it's a nonsequiteur. The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using inherited systems of meaning acquired as very young children in adaption within their environments, etc. In the case of transformers all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. If you project both of these into "problems an office worker has"-space, then they can appear simimlar -- but this projection is an incredibly dumb one, and offered as a sales pitch by charlatans looking to pretend that a system which can generate office emails can communicate.
  - diego_sandoval 2y ago
    > all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric. I don't think that's all they can do. I think they know more than what is explicitly stated in their training sets. They can generalize knowledge and generalize relationships between the concepts that are in the training sets. They're currently mediocre at it, but the results we observe from SOTA generative models are not explainable without accepting that they can create an internal model of the world that's more than just a decompression algorithm. I'm going to step away from LLMs for a moment, but: How are video generator models capable of creating videos with accurate shadows and lighting that is consistent in the entire frame and consistent between frames? You can't do that simply by taking a weighted average of the sections of videos you've seen in your training set. You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own". I think the same principle applies to LLMs.
    - carlmr 2y ago
      >You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own". Compression is understanding. If you have a model which explains shadows you can compress your video data much better. Since you "understand" how shadows work.
  - fenomas 2y ago
    > all they can do is, in fact, sample from a compression of historical texts To me, results like the Othello paper make any sort of "stochastic parrot" thinking completely untenable. https://thegradient.pub/othello/ https://thegradient.pub/othello/
    - samatman 2y ago
      This result is an argument for the conclusion you are reading it as arguing against.
- xanderlewis 2y ago
  Well, yes — absolutely. You could say something similar about any system with complex emergent behaviour. 'All computers can do are NAND operations and any other ability is just a side effect', or something. However, I do think that in this case it's meaningful. The claim isn't that LLMs are genuinely exhibiting reasoning ability — I think it's quite clear to anyone who probes them for long enough that they're not. I was fooled initially too, but you soon come to realise it's a clever trick (albeit not one contrived by any of the human designers themselves). The claim is usually some pseudo-philosophical claim that the very definition of reasoning is simply 'outputting (at least some of the time) correct sentences' and so there's no more to be said. But this is just silly. It's quite obvious that being able to manipulate language and effectively have access to a vast (fuzzily encoded) database of knowledge will mean you can output true and pertinent statements a lot of the time. But this doesn't require reasoning at all. Note that I'm not claiming that LLMs exhibit reasoning and other abilities 'as a side effect' of language manipulation ability — I'm claiming there's no reason to believe they have these abilities at all based on the available evidence. Humans are just very easily convinced by beings that seem to speak our language and are overly inclined to attribute all sorts of desires, internal thought processes and whatever else for which there are no evidence.
  - famouswaffles 2y ago
    >I think it's quite clear to anyone who probes them for long enough that they're not. I disagree and so do a lot of people who've used them for a long while. This is just an assertion that you wish to be true rather than something that actually is. What happens is that for some bizarre reason, for machines, lots of humans have a standard of reasoning that only exists in fiction. Devise any reasoning test you like that would cleanly separate humans from LLMs. I'll wait. > The claim is usually some pseudo-philosophical claim that the very definition of reasoning is simply 'outputting (at least some of the time) correct sentences' and so there's no more to be said. There is nothing philosophical or pseudo-philosophical about saying reasoning is determined by output. If anything, the opposite is what's philosophical nonsense. The idea that there exists some "real" reasoning that humans perform and "fake" reasoning that LLMs perform and yet somehow no testable way to distinguish this is purely the realm of fiction and philosophy. If you're claiming a distinction that doesn't actually distinguish, you're just making stuff up. LLMs clearly reason. They do things, novel things that no sane mind would see a human do and call anything else. They do things that are impossible to describe as anything else unless you subscribe to what i like to call statistical magic - https://news.ycombinator.com/item?id=41141118 https://news.ycombinator.com/item?id=41141118 And all things considered, LLMs are pretty horrible memorizers. Getting one to regurgitate Training data is actually really hard. There's no database of knowledge. It clearly does not work that way.
baq 2y ago
> it’s all they can do this overlooks how they do it. we don't really know. it might be logical reasoning, it might be a very efficient content addressable human-knowledge-in-a-blob-of-numbers lookup table... it doesn't matter if they work, which they do, sometimes scarily well. dismissing their abilities because they 'don't reason' is missing the forest for the trees in that they'd be capable of reasoning if they were able to run sat solvers on their output mid generation.
- SiempreViernes 2y ago
  Dismissing claims that LLMs "reason" because these machines perform no actions similar to reasoning seems pretty motivated. And I don't think "blindly take input from a reasoning capable system" counts as reasoning.
  - hobs 2y ago
    Does it? I think Blindsight (the book) had a good commentary on reason being a thing we think is a conscious process but doesn't have to be. I think most people talking past each other are really discussing whether the GPT is conscious, has a mental model of self, that kind of thing, as long as your definition of reasoning doesn't include consciousness it clearly does it (though not well.)
  - foobarbecue 2y ago
    "pretty motivated"? Did you mean biased?
    - ChadNauseam 2y ago
      I assume they meant motivated as shorthand for "motivated reasoning" which implies a bias that's motivating them to reason a certain way
      - foobarbecue 2y ago
        Oh, interesting -- that's a new one for me.
sitkack 2y ago
Hinton claims they do reason. I am going to go with Hinton on this.
- xanderlewis 2y ago
  Hinton's opinions on LLMs are frankly bonkers. Just because you're famous — and intelligent and successful — doesn't mean you can't be completely wrong. Also: what's his rationale? It's no use simply claiming something without evidence. And as far as I (and seemingly most others) can see, there's no such evidence other than that they can sometimes output sentences that happen to be true. But so can Wikipedia — does that mean Wikipedia is reasoning? Also, any form of reasoning in the usual sense of the word would surely require the ability to allocate arbitrary amounts of computation (i.e. thought) to each question. LLMs don't do this — they don't sit and ponder; each token takes exactly the same amount of computation to produce. Once they hit an 'end of text' token, they're done. Even empirically speaking, LLMs' ability to reason can be seen to be nonexistent. Just try asking basic mathematics questions. As soon as you ask anything for which the answer isn't available — practically verbatim — on the web already, it produces intelligent-sounding gibberish. This whole idea that 'LLMs must be able to reason because in order to learn to fake reasoning you must learn to actually reason' is like some kind of inverted no true Scotsman fallacy.
  - sitkack 2y ago
    Does slime mold reason? Yes, Hinton can be wrong, is wrong on many things like his misunderstanding on Chomsky and language. But I also think he has spent thousands of hours testing these systems scientifically. Your last sentence puts a lot of words in peoples mouths. But to continue down that line, fake reasoning and actual reasoning sounds like the Chinese Room. Is that the argument you are making? We don't understand our own mental processes well enough, so I try to not anthropomorphize reasoning and cognition.
    - xanderlewis 2y ago
      > Your last sentence puts a lot of words in peoples mouths. Well, it’s the most common sentiment I see on both here and (before I gave up) the AI-centred parts of reddit. It’s not quite the Chinese Room, since LLMs can’t even simulate reasoning very well. So there’s no need to debate the distinction between ‘fake reasoning and actual reasoning’ — there may or may not be a difference, but it’s not the point I’m making. As for Hinton: I’m sure he has. But inventors are often not experts on their own creations/discoveries, and are probably just as prone to FUD and panic in the face of surprising developments as the rest of us. No one predicted that autoregressive transformers would get us this far, least of all the experts whose decades of work lead us to this point.

worldsayshi 2y ago

One potential benefit should be that with the right tooling around it it should be able to translate your code base to a different language and/or framework more or less at the push of a button. So if a team is wondering if it would be worth it to switch a big chunk of the code base from python to elixir they don't have to wonder anymore. I tried translating a python script to javascript the other day and it was flawless. I would expect it to scale with a bit of hand-railing.

adamdiy 2y ago
see projects like https://github.com/joshpxyne/gpt-migrate https://github.com/joshpxyne/gpt-migrate think there's also a YC company recently focusing on the nasty, big migrations with LLM help
- worldsayshi 2y ago
  It seems that this kind of application can really change how the tech industry can evolve down the line. Maybe we will more quickly converge on tech stacks if everyone can test new one's out "within a week".
scarface_74 2y ago
ChatGPT is trained well enough on all things AWS that it can do a decent job translating Python based SDK code to Node and other languages, translate between CloudFormation/Terraform/CDK (in various languages). It does a well at writing simple to medium complexity automation scripts around AWS. If it gets something wrong, I tell it to “verify your answer using the documentation available on the web”
- kamaal 2y ago
  >>ChatGPT is trained well enough on all things AWS It was scary to me how to chatting with GPT or Claude would give me information which was a lot more clear than what I could deduce after hours of reading AWS documentation. Perhaps, the true successor to Google search has arrived. One big drawback of Google was asking questions that can't be converted to a full long conversation. To that end. LLM chat is the ultimate socratic learning method tool till date.

greenthrow 2y ago

The problem is the use case is where you don't care about the risk of hallucinations or you can validate the output without already having the data in a useful format. Plus you need to lack the knowledge/skill to do it more quickly using awk/python/perl/whatever.

worldsayshi 2y ago
That's why having good test suites and tools are more important than ever.
j_maffe 2y ago
I think text transformation is a sufficiently predictable task that one could make a transformer that completely avoids hallucinations. Most LLMs have high temperatures which introduces randomness and therefore hallucinations into the result.

deleted 2y ago

[deleted]