4 ms·

Oh yes. The old "it's AI" when it's convenient and "it's just a tool" when it's convenient. Which one is it? I this it's just an autocomplete on steroids, a po

by x86x87 4y ago

Oh yes. The old "it's AI" when it's convenient and "it's just a tool" when it's convenient. Which one is it? I this it's just an autocomplete on steroids, a powerful bullshitting machine. The fear I have is that this bullshit is going to pollute the digital world pretty much the same way SEO made it impossible to get good results when using a search engine.

deleted 4y ago
[deleted]
BoorishBears 4y ago
I never get this "it's just an autocomplete on steroids" take. When you form a sentence do you catalog every single word in the english language then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow? - AI is a tool. And it's as much a bullshit generator as a human telling a story they definitely completely remember very vividly... https://www.newyorker.com/science/maria-konnikova/idea-happened-memory-recollection https://www.newyorker.com/science/maria-konnikova/idea-happe... If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.
- nextaccountic 4y ago
  > When you form a sentence do you catalog every single word in the english language then pick one before saying a word? That's how markov chain chat bots (a very old technology) works: https://stackoverflow.com/questions/5306729/how-do-markov-chain-chatbots-work https://stackoverflow.com/questions/5306729/how-do-markov-ch... https://www.baeldung.com/cs/markov-chain-chatbots https://www.baeldung.com/cs/markov-chain-chatbots That's not how ChatGPT works though, because of the attention mechanism
  - jhbadger 4y ago
    Exactly. The way that LLMs have been "debunked" in simplified writings on the Web and in media is to suggest that they are just Markov chains like the famous "Mark V. Shaney"[1] bot from the 1980s, and LLMs are far more powerful than that. Yes, we need to debunk claims that LLMs have achieved sentience and such nonsense, but let's not ignore just how amazing they are. [1] https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney
  - BoorishBears 4y ago
    I'm plenty familiar with flash attention, but you didn't understand what you quoted. GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator. - The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence] So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent. If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete. - Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..." The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of. But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely. The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up. And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider. - It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please. We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations. What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
- jzb 4y ago
  ChatGPT doesn’t just misidentify the guy - it entirely fabricates a crime and attaches it to a suspect. I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”) I expected it might give a bland summary or something, but that’s not what happened. It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song. That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own. Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
  - asdff 4y ago
    Every time I see people on HN say how they love to use chatgpt to generate ideas I think of this. It seems a lot more work coming up with prompts then having to vet them to see if the output is even sensible or not than it does to come up with some sensible search terms to query real data that actually contains what you are looking for.
    - BoorishBears 4y ago
      Using LLMs for factual concepts is by far the most boring application of them. Often times people use LLMs for generating ideas that don't have a factual basis. ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built. If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
  - BoorishBears 4y ago
    You're starting off with a distinction without difference. You could throw darts at a spinning wheel with real names and imagined crimes. The point is that it doesn't matter what the seed for the false statement is, it's the act of spreading it that's problematic. You're also muddling a point that I can agree with: Treating ChatGPT as an infallible expert is wrong. But that applies to so many other things. Even expert witnesses are not infallible. So I disagree with characterizing hallucinations as the problem, it's the application that's problematic. Blindly and pasting factual content from ChatGPT is a bad idea, just like blindly taking a single source of information as gospel is a bad idea. Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.
Dalewyn 4y ago
>Oh yes. The old "it's AI" when it's convenient and "it's just a tool" when it's convenient. Which one is it? An "AI" is a tool.

by x86x87 4y ago

deleted 4y ago

[deleted]

BoorishBears 4y ago

I never get this "it's just an autocomplete on steroids" take. When you form a sentence do you catalog every single word in the english language then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow? - AI is a tool. And it's as much a bullshit generator as a human telling a story they definitely completely remember very vividly... https://www.newyorker.com/science/maria-konnikova/idea-happened-memory-recollection https://www.newyorker.com/science/maria-konnikova/idea-happe... If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.

nextaccountic 4y ago
> When you form a sentence do you catalog every single word in the english language then pick one before saying a word? That's how markov chain chat bots (a very old technology) works: https://stackoverflow.com/questions/5306729/how-do-markov-chain-chatbots-work https://stackoverflow.com/questions/5306729/how-do-markov-ch... https://www.baeldung.com/cs/markov-chain-chatbots https://www.baeldung.com/cs/markov-chain-chatbots That's not how ChatGPT works though, because of the attention mechanism
- jhbadger 4y ago
  Exactly. The way that LLMs have been "debunked" in simplified writings on the Web and in media is to suggest that they are just Markov chains like the famous "Mark V. Shaney"[1] bot from the 1980s, and LLMs are far more powerful than that. Yes, we need to debunk claims that LLMs have achieved sentience and such nonsense, but let's not ignore just how amazing they are. [1] https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney
- BoorishBears 4y ago
  I'm plenty familiar with flash attention, but you didn't understand what you quoted. GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator. - The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence] So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent. If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete. - Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..." The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of. But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely. The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up. And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider. - It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please. We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations. What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
jzb 4y ago
ChatGPT doesn’t just misidentify the guy - it entirely fabricates a crime and attaches it to a suspect. I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”) I expected it might give a bland summary or something, but that’s not what happened. It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song. That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own. Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
- asdff 4y ago
  Every time I see people on HN say how they love to use chatgpt to generate ideas I think of this. It seems a lot more work coming up with prompts then having to vet them to see if the output is even sensible or not than it does to come up with some sensible search terms to query real data that actually contains what you are looking for.
  - BoorishBears 4y ago
    Using LLMs for factual concepts is by far the most boring application of them. Often times people use LLMs for generating ideas that don't have a factual basis. ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built. If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
- BoorishBears 4y ago
  You're starting off with a distinction without difference. You could throw darts at a spinning wheel with real names and imagined crimes. The point is that it doesn't matter what the seed for the false statement is, it's the act of spreading it that's problematic. You're also muddling a point that I can agree with: Treating ChatGPT as an infallible expert is wrong. But that applies to so many other things. Even expert witnesses are not infallible. So I disagree with characterizing hallucinations as the problem, it's the application that's problematic. Blindly and pasting factual content from ChatGPT is a bad idea, just like blindly taking a single source of information as gospel is a bad idea. Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.

Dalewyn 4y ago

>Oh yes. The old "it's AI" when it's convenient and "it's just a tool" when it's convenient. Which one is it? An "AI" is a tool.