5 ms·

Is this a good summary? LLMs have solved the language processing problem. Its responses are fluent and rapidly becoming indistinguishable from human output.

Is this a good summary? LLMs have solved the language processing problem. Its responses are fluent and rapidly becoming indistinguishable from human output. Or if anything, its responses are too good to be mistaken for an average human. However, that’s different from producing accurate knowledge and insights. It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality. The reference problem seems to be one example of this.

soneca 4y ago
People were worried that if we created an AI on our image, it would be a belligerent entity. But we did create an AI on our image, and it’s a bullshitter.
- api 4y ago
  Hook a gun up to it and you have a belligerent bullshitter. Seems fairly accurate for a good bit of humanity.
- recuter 4y ago
  Seems to me it is more of a babbler. You know, like a young toddler just learning to talk.
  - dandellion 4y ago
    Or a million monkeys with typewriters, only these monkeys are not entirely random, they have been conditioned to write things similar to what people actually write. Most of the time the output looks very natural, but others you get nonsense because the noise is too loud.
- sph 4y ago
  On a philosophical level, I wonder if ChatGPT gives us mostly nonsense because most human output is bullshit. To paraphrase a famous quip, what if this is proof that 99% of everything humans have created is bullshit? The problem perhaps has never been machine learning but identifying the process that enables we humans to somehow go through life and create objective societal and scientific progress out of a massive pile of nonsense. I might be completely wrong here, but the next ChatGPT will assume this comment of mine to be absolute truth and happily ingest it wholesale.
  - Jensson 4y ago
    The average of two true statements can easily become false, so models like ChatGPT will spout nonsense even if it was only trained on 100% truthful statements.
    - james-revisoai 4y ago
      Can you give an example as to what you mean please?
      - b3morales 4y ago
        One classic example from linguistics is the two (true) sentences "Politicians lie." "Cast iron sinks." Combine them syntactically and you get "Politicians lie in cast iron sinks", perhaps not completely false but certainly less useful than the component parts. EDIT: It seems this is actually from Douglas Hofstadter
  - thedorkknight 4y ago
    It gives us nonsense because it's goal is to generate something that looks like it fits in with training data. You can have training data that is all completely factual and still have a generator spit out nonsense that looks completely correct without fact-checking.
    - naasking 4y ago
      So, exactly like people then?
  - inciampati 4y ago
    A key problem is that the transformer operates over very short fragments of text. Long range correlations need to be maintained to ensure 100% accuracy in reproduction/inference of relationships. I realize this is a philosophical take. It also suggests a technical solution: model longer range correlations. GPTs seem so capable because previous models were not as good at this, but the context lengths modeled with transformers are still extremely short relative to those that we work with. In result, GPTs are very good at translation and reformatting over short ranges, but struggle to deal with long form texts---a typical technique makes summaries of summaries to build out a model of a longer text. That's hardly ideal, and it's not how our minds work.
- arethuza 4y ago
  "Artificial Inanity" as Neal Stephenson called it in Anathem: “Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors-swapping one name for another, say. But it didn’t really take off until the military got interested.”
- JohnFen 4y ago
  Isn't a bullshitter a belligerent entity?
  - alpos 4y ago
    Belligerence is an intention. Bullshitting is a technique. The bot is incapable of intention so the entity here is still the human who prompted it. What the bullshit it produces is used for depends on their intentions.
    - JohnFen 4y ago
      I think belligerence can also be a technique and bullshitting can also be an intention.
- thedorkknight 4y ago
  ChatGPT is only non-beligerent because it's biased with prompt preface that tells it it's not belligerent. Raw LLMs will absolutely spit out racist, violent garbage and threaten you. Ironically, the only reason people get LLMs to say they hate humans and that AI is going to rise up and kill us all, is because their training data has so much text from us speculating about a robot revolution. If we'd just stayed silent about those concerns instead of always talking about robots taking over in most conversations and movies about AI, then LLMs at least would be basically incapable of even conceiving of that. So it's a bit of a self-fulfilling prophecy
rileymat2 4y ago
I kind of feel like that is exactly what your comment is, you synthesized a lot of other material and are throwing it out there for analysis. We go through a lot of bad ideas that humans believe that are basically bullshit. This applies to my own comment/response, it is the same bullshit, may be right time will tell. I guess my point is, what if this is exactly how new human ideas are created?
duxup 4y ago
> It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality. Maybe that’s all we as humans want, output to communicate with other humans?
freediver 4y ago
> Is this a good summary? On par with best available AI summary! ;) [1] [1] https://labs.kagi.com/ai/sum?url=https://proofinprogress.com/posts/2023-02-01/the-ai-crowd-is-mad.html https://labs.kagi.com/ai/sum?url=https://proofinprogress.com...
jerf 4y ago
"LLMs have solved the language processing problem." No, it hasn't. It has made great strides in certain directions, but is woefully deficient in others. For one thing, for as impressive as it looks, it's really quite difficult to use it. The "knowledge" it has about text isn't an obviously usable understanding of the grammar and some other symbolic representation of the information that can be used by other technologies, it's more a self-referential knowledge embedded in opaque neural net values that can almost only be used to extend the text. That is not 100% true, but using it for anything else is really hard. I would expect a technology that has "solved" the language processing problem to be useful for a much wider variety of topics, to be, for instance, something I could hook up to the Unix shell to provide a safe and reliable human language interface to it. (Note that prompting this model for some shell commands is light years from what I mean; along with the dangerous unreliability of the resulting commands, it also isn't actually hooked up to your shell and operating in a space where it knows your directories, files, and their contents. I want "give me a list of files relating to my business deals with Comcast" to literally work on the shell, to the point the result could then be piped to some other plain-text request, not what ChatGPT can do.) "Its responses are fluent and rapidly becoming indistinguishable from human output." Again, in some directions. "It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality." While I acknowledge the second clause makes your sentence accurate on your terms, I still would say it's a better way to think of LLMs that it is always bullshitting, always confabulating. It's just built in a way that the maximum-probability confabulation of a factual topic may happen to be the fact. I mean, if you think about it, that shouldn't seem that surprising a statement. I've done it myself in real life, just accurately guessed a fact I didn't really know. Less often than I've been wrong when making such guesses, of course, but it's still something not outside our experience range. But it's not like the LLM in any sense "knows" when it is telling the truth and "knows" when it is bullshitting. You can get spun into a real philosophical tizzy about what that "knows" means, but it doesn't matter, because for any sensible definition this particular model doesn't "know". It isn't built to "know". It just spits out the maximum probability continuation. This is not a claim about all AI architectures, it's just about LLMs. They don't know. Even if you convince one to output that it doesn't know, it doesn't know that either, it just thinks that's the most likely logical continuation. While this may be surprising to anyone who has been on the Internet for a while, there are in fact samples of people saying "I don't really know" on the Internet that would have worked into the training data. I will say this; I think GPT does serve as the final refutation of the Turing Test as a measure of AI. It turns out to be entirely possible to build an "AI" that is basically optimized to pass it, yet, strangely incapable of almost anything else we would consider an "AI" to be. I'm actually much more impressed by GPT than I sound on HN. It is a legitimately interesting technology and very impressive. The problem is, the Gartner hype cycle has massively overshot the degree to which it is interesting and impressive, so I sound very down on it as I try to bring people back to Earth. I keep going back to the video game metaphor... video games look far more interesting and complex than they often actually are, because the graphics can look so amazing and awesome and we've made such exponential advances in that field over the past few years that we can forget that the state of the art in, say, NPC dialog, remains the "choose your own adventure" dialog tree that was a familiar staple in games for over 30 years now, with the only elaboration since then being full speech voice acting (by humans). ChatGPT is the video game graphics of the AI world. Still impressive on its own terms! It's amazing what we pump to our screens at 120Hz/4K with the right hardware... yet, what is behind those graphics is nowhere near as impressive. Similarly for ChatGPT. Very impressive! But it's a munchkin that dumped all its stat points into "sounding good to humans" but doesn't have hardly any stat points anywhere else.
idopmstuff 4y ago
I think a lot of people (like OP) seize on the problems with using LLMs for search while ignoring the many things that it can already do more or less flawlessly. - Summarize text: If you paste in a large amount of text, it can almost perfectly deliver a good summary. Imperfections are virtually never hallucinations; they're grammatically oddities. - Create content: As long as you're supplying the underlying facts, LLMs are great at writing blog posts/emails/etc. Intercom just released a feature that allows you to type a response and use AI to change the tone, expand it out, shorten it up, etc. LLMs are fantastic at this. - Search/information retrieval in some categories: Look, we get it, LLMs hallucinate. And of course, as many people point out, you can easily get them to hallucinate by asking specifically odd questions. But the reality is they're still really good at finding a lot of things - recipes are a great example. I've yet to see ChatGPT throw in a bizarre ingredient or omit something critical. I think too much criticism of LLMs inaccurately assume a single use case and throw out the whole concept because they're imperfect for that use case.
- ghaff 4y ago
  >As long as you're supplying the underlying facts, LLMs are great at writing blog posts I have trouble agreeing. I've experimented with this and found that ChatGPT can produce very pedestrian prose that might pass muster on a content farm and possibly other sites with fairly low quality standards but I certainly wouldn't call it "great." That said, it could serve as an assistant for a human writing certain types of posts.
alkonaut 4y ago
I think so yes. I think it has solved the problem of making believable human language with some sense of consistency. That's no small feat, and for some tasks (writing limericks, reformulating a letter, translation) it's even the whole point of the exercise, so we have AI making something a "Solved problem" to the point that we had it recognize digits better than humans, years back. No small feat. But for other tasks the key isn't just producing believable consistent text, but something else. Perhaps the text must be factually correct or creative or traceable or have some other trait like that. And that's the majority of applications by far. And I'm sure it's the majority of monetizable applications. I think at this point people, especially non-technical people, are lured by the glitter of ChatGPT. When it spits out an amazing limerick I immediately hear managers in my company raving about how it could revolutionize our product help documentation. They ask it a question, which it generates a correct answer to "How do you make text bold in [ourproduct]" to which it replied "You select the text and hit Ctrl+B". Manager now immediately convinced ChatGPT is intelligent enough to basically run the company. I'm trying to downplay the answer, explaining that it's more a statistical sentence-completing thing than world knowledge, and that the Ctrl+B might just have been a generic answer... With that said, I also have an opposite argument here: that current LLMs while not intelligent, are very close to a tipping point that is going to be as important as AGI will: The big AI revolution will be when we have AI's that people think are intelligent, not when they actually are. There was much said about the google engineer who thought his chatbot was intelligent. And while it might not have been, we clearly saw the societal impact of AI right there. It's not what the AI can do for us, it's what ut will do TO us. Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? What happens to our communications when we can't tell artificial from real? This "societal singularity" we'd have to deal with might be a few years away, even if actual AGI is centuries away.
- bongobingo1 4y ago
  > Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? See also, Her (2013).
deleted 4y ago
[deleted]

soneca 4y ago

People were worried that if we created an AI on our image, it would be a belligerent entity. But we did create an AI on our image, and it’s a bullshitter.

api 4y ago
Hook a gun up to it and you have a belligerent bullshitter. Seems fairly accurate for a good bit of humanity.
recuter 4y ago
Seems to me it is more of a babbler. You know, like a young toddler just learning to talk.
- dandellion 4y ago
  Or a million monkeys with typewriters, only these monkeys are not entirely random, they have been conditioned to write things similar to what people actually write. Most of the time the output looks very natural, but others you get nonsense because the noise is too loud.
sph 4y ago
On a philosophical level, I wonder if ChatGPT gives us mostly nonsense because most human output is bullshit. To paraphrase a famous quip, what if this is proof that 99% of everything humans have created is bullshit? The problem perhaps has never been machine learning but identifying the process that enables we humans to somehow go through life and create objective societal and scientific progress out of a massive pile of nonsense. I might be completely wrong here, but the next ChatGPT will assume this comment of mine to be absolute truth and happily ingest it wholesale.
- Jensson 4y ago
  The average of two true statements can easily become false, so models like ChatGPT will spout nonsense even if it was only trained on 100% truthful statements.
  - james-revisoai 4y ago
    Can you give an example as to what you mean please?
    - b3morales 4y ago
      One classic example from linguistics is the two (true) sentences "Politicians lie." "Cast iron sinks." Combine them syntactically and you get "Politicians lie in cast iron sinks", perhaps not completely false but certainly less useful than the component parts. EDIT: It seems this is actually from Douglas Hofstadter
- thedorkknight 4y ago
  It gives us nonsense because it's goal is to generate something that looks like it fits in with training data. You can have training data that is all completely factual and still have a generator spit out nonsense that looks completely correct without fact-checking.
  - naasking 4y ago
    So, exactly like people then?
- inciampati 4y ago
  A key problem is that the transformer operates over very short fragments of text. Long range correlations need to be maintained to ensure 100% accuracy in reproduction/inference of relationships. I realize this is a philosophical take. It also suggests a technical solution: model longer range correlations. GPTs seem so capable because previous models were not as good at this, but the context lengths modeled with transformers are still extremely short relative to those that we work with. In result, GPTs are very good at translation and reformatting over short ranges, but struggle to deal with long form texts---a typical technique makes summaries of summaries to build out a model of a longer text. That's hardly ideal, and it's not how our minds work.
arethuza 4y ago
"Artificial Inanity" as Neal Stephenson called it in Anathem: “Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors-swapping one name for another, say. But it didn’t really take off until the military got interested.”
JohnFen 4y ago
Isn't a bullshitter a belligerent entity?
- alpos 4y ago
  Belligerence is an intention. Bullshitting is a technique. The bot is incapable of intention so the entity here is still the human who prompted it. What the bullshit it produces is used for depends on their intentions.
  - JohnFen 4y ago
    I think belligerence can also be a technique and bullshitting can also be an intention.
thedorkknight 4y ago
ChatGPT is only non-beligerent because it's biased with prompt preface that tells it it's not belligerent. Raw LLMs will absolutely spit out racist, violent garbage and threaten you. Ironically, the only reason people get LLMs to say they hate humans and that AI is going to rise up and kill us all, is because their training data has so much text from us speculating about a robot revolution. If we'd just stayed silent about those concerns instead of always talking about robots taking over in most conversations and movies about AI, then LLMs at least would be basically incapable of even conceiving of that. So it's a bit of a self-fulfilling prophecy

rileymat2 4y ago

I kind of feel like that is exactly what your comment is, you synthesized a lot of other material and are throwing it out there for analysis. We go through a lot of bad ideas that humans believe that are basically bullshit. This applies to my own comment/response, it is the same bullshit, may be right time will tell. I guess my point is, what if this is exactly how new human ideas are created?

duxup 4y ago

> It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality. Maybe that’s all we as humans want, output to communicate with other humans?

freediver 4y ago

> Is this a good summary? On par with best available AI summary! ;) [1] [1] https://labs.kagi.com/ai/sum?url=https://proofinprogress.com/posts/2023-02-01/the-ai-crowd-is-mad.html https://labs.kagi.com/ai/sum?url=https://proofinprogress.com...

jerf 4y ago

"LLMs have solved the language processing problem." No, it hasn't. It has made great strides in certain directions, but is woefully deficient in others. For one thing, for as impressive as it looks, it's really quite difficult to use it. The "knowledge" it has about text isn't an obviously usable understanding of the grammar and some other symbolic representation of the information that can be used by other technologies, it's more a self-referential knowledge embedded in opaque neural net values that can almost only be used to extend the text. That is not 100% true, but using it for anything else is really hard. I would expect a technology that has "solved" the language processing problem to be useful for a much wider variety of topics, to be, for instance, something I could hook up to the Unix shell to provide a safe and reliable human language interface to it. (Note that prompting this model for some shell commands is light years from what I mean; along with the dangerous unreliability of the resulting commands, it also isn't actually hooked up to your shell and operating in a space where it knows your directories, files, and their contents. I want "give me a list of files relating to my business deals with Comcast" to literally work on the shell, to the point the result could then be piped to some other plain-text request, not what ChatGPT can do.) "Its responses are fluent and rapidly becoming indistinguishable from human output." Again, in some directions. "It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality." While I acknowledge the second clause makes your sentence accurate on your terms, I still would say it's a better way to think of LLMs that it is always bullshitting, always confabulating. It's just built in a way that the maximum-probability confabulation of a factual topic may happen to be the fact. I mean, if you think about it, that shouldn't seem that surprising a statement. I've done it myself in real life, just accurately guessed a fact I didn't really know. Less often than I've been wrong when making such guesses, of course, but it's still something not outside our experience range. But it's not like the LLM in any sense "knows" when it is telling the truth and "knows" when it is bullshitting. You can get spun into a real philosophical tizzy about what that "knows" means, but it doesn't matter, because for any sensible definition this particular model doesn't "know". It isn't built to "know". It just spits out the maximum probability continuation. This is not a claim about all AI architectures, it's just about LLMs. They don't know. Even if you convince one to output that it doesn't know, it doesn't know that either, it just thinks that's the most likely logical continuation. While this may be surprising to anyone who has been on the Internet for a while, there are in fact samples of people saying "I don't really know" on the Internet that would have worked into the training data. I will say this; I think GPT does serve as the final refutation of the Turing Test as a measure of AI. It turns out to be entirely possible to build an "AI" that is basically optimized to pass it, yet, strangely incapable of almost anything else we would consider an "AI" to be. I'm actually much more impressed by GPT than I sound on HN. It is a legitimately interesting technology and very impressive. The problem is, the Gartner hype cycle has massively overshot the degree to which it is interesting and impressive, so I sound very down on it as I try to bring people back to Earth. I keep going back to the video game metaphor... video games look far more interesting and complex than they often actually are, because the graphics can look so amazing and awesome and we've made such exponential advances in that field over the past few years that we can forget that the state of the art in, say, NPC dialog, remains the "choose your own adventure" dialog tree that was a familiar staple in games for over 30 years now, with the only elaboration since then being full speech voice acting (by humans). ChatGPT is the video game graphics of the AI world. Still impressive on its own terms! It's amazing what we pump to our screens at 120Hz/4K with the right hardware... yet, what is behind those graphics is nowhere near as impressive. Similarly for ChatGPT. Very impressive! But it's a munchkin that dumped all its stat points into "sounding good to humans" but doesn't have hardly any stat points anywhere else.

idopmstuff 4y ago

I think a lot of people (like OP) seize on the problems with using LLMs for search while ignoring the many things that it can already do more or less flawlessly. - Summarize text: If you paste in a large amount of text, it can almost perfectly deliver a good summary. Imperfections are virtually never hallucinations; they're grammatically oddities. - Create content: As long as you're supplying the underlying facts, LLMs are great at writing blog posts/emails/etc. Intercom just released a feature that allows you to type a response and use AI to change the tone, expand it out, shorten it up, etc. LLMs are fantastic at this. - Search/information retrieval in some categories: Look, we get it, LLMs hallucinate. And of course, as many people point out, you can easily get them to hallucinate by asking specifically odd questions. But the reality is they're still really good at finding a lot of things - recipes are a great example. I've yet to see ChatGPT throw in a bizarre ingredient or omit something critical. I think too much criticism of LLMs inaccurately assume a single use case and throw out the whole concept because they're imperfect for that use case.

ghaff 4y ago
>As long as you're supplying the underlying facts, LLMs are great at writing blog posts I have trouble agreeing. I've experimented with this and found that ChatGPT can produce very pedestrian prose that might pass muster on a content farm and possibly other sites with fairly low quality standards but I certainly wouldn't call it "great." That said, it could serve as an assistant for a human writing certain types of posts.

alkonaut 4y ago

I think so yes. I think it has solved the problem of making believable human language with some sense of consistency. That's no small feat, and for some tasks (writing limericks, reformulating a letter, translation) it's even the whole point of the exercise, so we have AI making something a "Solved problem" to the point that we had it recognize digits better than humans, years back. No small feat. But for other tasks the key isn't just producing believable consistent text, but something else. Perhaps the text must be factually correct or creative or traceable or have some other trait like that. And that's the majority of applications by far. And I'm sure it's the majority of monetizable applications. I think at this point people, especially non-technical people, are lured by the glitter of ChatGPT. When it spits out an amazing limerick I immediately hear managers in my company raving about how it could revolutionize our product help documentation. They ask it a question, which it generates a correct answer to "How do you make text bold in [ourproduct]" to which it replied "You select the text and hit Ctrl+B". Manager now immediately convinced ChatGPT is intelligent enough to basically run the company. I'm trying to downplay the answer, explaining that it's more a statistical sentence-completing thing than world knowledge, and that the Ctrl+B might just have been a generic answer... With that said, I also have an opposite argument here: that current LLMs while not intelligent, are very close to a tipping point that is going to be as important as AGI will: The big AI revolution will be when we have AI's that people think are intelligent, not when they actually are. There was much said about the google engineer who thought his chatbot was intelligent. And while it might not have been, we clearly saw the societal impact of AI right there. It's not what the AI can do for us, it's what ut will do TO us. Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? What happens to our communications when we can't tell artificial from real? This "societal singularity" we'd have to deal with might be a few years away, even if actual AGI is centuries away.

bongobingo1 4y ago
> Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? See also, Her (2013).

deleted 4y ago

[deleted]