12 ms·

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart

• topologie 4y ago
  While reading your comment I realized that being "confidently wrong" is actually really human, so... yay? I mean, that's one step closer to machines thinking like humans, right? :)
• Eisenstein 4y ago
  I think it is better to assume that normal people can be deceived by confident language than to assume this is a problem with 'dumb people'.
  • kerkeslager 4y ago
    I'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people". If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.
    • Eisenstein 4y ago
      Very good point. Everyone should do this.
    • prox 4y ago
      Yeah this is a common sales tactic, and talented/ charismatic people can certainly employ confidence for an uptick in sales. The way to combat this is the same for all hoaxes/disinfo and so on, epistemology. The attitude of the Socratic method. As I write this I wonder if ChatGPT can use the Socratic method on itself.
      • Eisenstein 4y ago
        The Socratic method requires self-awareness, otherwise the questions fall flat. Unfortunately I think that there is a greater chance that an LLM will become capable of this before a majority of humans will.
      • kerkeslager 4y ago
        Part of what's extremely frustrating about talking to people about AI is that people say things which seem like they understand what's going on in one breath, and then in the next breath say something that makes absolutely no sense if they understood what they just said. This post is a great example of that. Okay, so you apply epistemology to ChatGPT: when you ask ChatGPT a question, how does it know the answer? The answer is: it doesn't know the answer. All it knows is how people string words together: it doesn't have any understanding of what the words mean. So no, it can't use the Socratic method on itself or anyone. It can't ask questions to stimulate critical thinking, because it's incapable of critical thinking. It can't draw out ideas and underlying presuppositions, because it doesn't have ideas or underlying presuppositions (or suppositions). It's not even capable of asking questions: it's just stringing together text that matches the pattern of what a question is, without even a the understanding that the text is a question, or that the following text in the pattern is an answer. "Question" and "answer" are not concepts that ChatGPT understands because ChatGPT doesn't understand concepts.
• vernon99 4y ago
  > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence Even if ChatGPT itself is, systems built on top are definitely not, this is just getting starting.
  • jonny_eh 4y ago
    GPT4 is already blowing ChatGPT out of the water in what it can do.
• candiddevmike 4y ago
  This (among other things) is why OpenAI releasing it to the general public without considering the effects was irresponsible, IMO.
  • zamnos 4y ago
    There's an alternate reality where OpenAI was, instead, EvenMoreClosedAI, and the productivity multiplier effect was held close to their chest, and only elites had access to it. I'm not sure that reality is better.
    • porkbeer 4y ago
      What makes you think thats not the case now? Do you think they would tell you?
      • zamnos 4y ago
        It is known that some got access to GPT-4 before the rest of the world did. That OpenAI eventually released it to the world is what counts though. Hopefully GPT-5 will also be released too the world but we shall have to see.
  • alfor 4y ago
    Google and others had strong AI, but they are keep internal to maximize profits and spy on people. OpenAI put it out there so we can see it, interact and have the conversation. They have way more inside, ChatGPT is there to test it out in the real world progressively and so we get use to artificial superintelligence.
• flanked-evergl 4y ago
  > The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact. When you interpret it for what it is, it is not confidently wrong, it just generated what it thinks is most likely based on the input tokens. Sometimes, if the input tokens contain some counter-argument the model will generate text that would usually occur if an claim was refuted, but again, this is not based on reason, or fact, or logic. ChatGPT is not lying to people, it can't lie, at least not in the sense of "to make an untrue statement with intent to deceive". ChatGPT has no intent. It can generate text that is not in accordance with fact and is not derivable by reason from its training data, but why would you expect that from it? > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. I agree here, I think you can only get so far with a language model, maybe if we get a couple orders of mangitude more parameters it magically becomes AGI, but I somehow don't quite feel it, I think there is more to human intelligence than a LLM, way more. Of course, that is coming, but that would not be this paradigm, which is basically trying to overextend LLM. LLMs are great, they are useful, but if you want a model that reasons, you will likely have to train it for that, or possibly more likely, combine ML with something symbolic reasoning.
  • danpat 4y ago
    > but why would you expect that from it? If you understand what it is doing, then you don't. But the layman will just see a computer that talks in language they understand, and will infer intent and sentience are behind that, because that's the only analog they have for a thing that can talk back to them with words that appear to make sense at the complexity level that ChatGPT is achieving. Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.
    • 4y ago
• shagymoe 4y ago
  >indistinguishable from what dumb people think smart people are like. Since about 2016, we have overwhelming evidence that even "smart people" are "fooled" by "confidently wrong".
  • xyzelement 4y ago
    Especially true if one has a definitive opinion on which ones are the fooled ones :)
    • shagymoe 4y ago
      Yes and even more true when "confidently wrong" statements are provably false.
  • 8note 4y ago
    Why 2016?
    • drakmo 4y ago
      Please show me the rock you lived under. I want to move in!
• Herval_freire 4y ago
  [dead]
• whimsicalism 4y ago
  > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. I don't think you have a shred of evidence to back up this assertion.
  • skaushik92 4y ago
    … with confident language no less xD
    • rakah 4y ago
      Butlerian Jihad.
      • 8note 4y ago
        That doesn't help the root problem though, confidently wrong people are just as convincing as confidently wrong robots
      • asynchronous 4y ago
        “It is illegal to make a machine in the likeness of a human mind.”
        • int_19h 4y ago
          And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.
          • VectorLock 4y ago
            Better than getting extincted, I suppose.
            • at_a_remove 4y ago
• sebzim4500 4y ago
  >Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm I hope you appreciate the irony of making this confident statement without evidence in a thread complaining about hallucinations.
  • nfw2 4y ago
    It was not a confident statement, at least not the way ChatGPT is confident. There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold. If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncertain statements is fine as long as it is clear the uncertainty is acknowledged. ChatGPT has no concept of uncertainty. It casually constructs false statements to same way it constructs real knowledge backed by evidence. That's the problem.
    • deleted 4y ago
      [deleted]
    • motoxpro 4y ago
      Based on my understanding, it is the year 2035. I don't think that statement is any better. The only reason you would follow up is because you have prior knowledge that it is wrong. Based on my understanding of multiplication, 324113 * 23244 = 138492923 Is the above better?
      • shjake 4y ago
        > Is the above better? No. It is tangential though.
• m463 4y ago
  "Often wrong, never in doubt." An old saying, but frequently applies to the difficult people in your life. Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes you wonder)
  • 8note 4y ago
    Re wikipedia, I think it's most interesting how you choose which pages are trustworthy and which ones are not, without doing a detailed analysis on all the sources listed. What are the working heuristics?
  • cultureswitch 4y ago
    What's terrible about some Wikipedia pages is when you personally don't know they are controversial. It should be vastly more prominent when there are tensions on an article and even admins shouldn't have the power to hide this.
    • howenterprisey 4y ago
      Admins do not, in fact, have the power to hide tensions on an article, as even the act of deletion produces public logs. Certainly if you don't know to look for them, you won't see them, though.
      • ThunderSizzle 4y ago
        The main page should become a disjointed piece of both sides writing areas of the article. Right now, admins pick the side and moderate the other away into the history.
    • fl7305 4y ago
      Good point. Would it be that hard to write a Chrome plugin that adds a "Tensionmeter" to each Wikipedia page? Based on the edit and discussion history, the Tensionmeter could be green, yellow, or red. The different parts of the article text could be colorcoded according to similar metrics.
• theknocker 4y ago
  [dead]
• cultureswitch 4y ago
  GPT-2, 3 and 4 keep on showing that increasing the size of the model keeps on making the results better without slowing down. This is remarkable, because usually in practical machine learning applications there is a quickly reached plateau of effectiveness beyond which a bigger model doesn't yield better results. With these ridiculously huge LLMs, we're not even close yet. And this was exciting news in papers from years ago talking about the upcoming GPT3 btw.
  • worrycue 4y ago
    Yet they still can’t get it to shut the hell up if it doesn’t know and to not make shit up to pad its answers.
    • bsder 4y ago
      "I don't know" isn't in the training set, after all. Welcome to Eternal September on the Internet. The first "AI" that actually says "I don't know" in response to a question will get my attention.
• yosito 4y ago
  > stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference Until the demagogues train the AI.
• htag 4y ago
  > what dumb people think smart people are like What do dumb people think smart people are like? Is this a trope, or common idiom? I've never heard this before.
  • deleted 4y ago
    [deleted]
  • Barrin92 4y ago
    the more common idiom is "XYZ is a stupid person's idea of what a smart person looks like" and it's usually applied to slimy hucksters or influencers. I think the archetypical example is the "bookshelves in my garage" guy who went viral years ago. (https://youtu.be/Cv1RJTHf5fk https://youtu.be/Cv1RJTHf5fk)
• pdonis 4y ago
  > individuals whose intelligence is roughly equivalent to ChatGPT's There aren't any such individuals. Even the least intelligent human is much, much more intelligent than ChatGPT, because even the least intelligent human has some semantic connection between their mental processes and the real world. ChatGPT has none. It is not intelligent at all.
• cyanydeez 4y ago
  Have you not seen current politics. What people believe is largely based on motivated reasoning rather than anything else. ChatGPT is basically a free propaganda machine, much easier that 4chan
• jalapenos 4y ago
  This is no worse than humans. We can't protect people from being misled by other humans with big mouths. That's their responsibility. Likewise, it's their responsibility not to treat text they read from the internet, coming from an AI or otherwise, as perfect truth. There's always a certain undercurrent of narcissism that flows beneath paternalism. Basically, "they couldn't possibly be as smart as me, so I have to protect them for their own good".
• lamontcg 4y ago
  > The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.
  • oneshtein 4y ago
    [flagged]
    • f6v 4y ago
      I thought ChatGPT posts were the most annoying thing on the internet right now. Then there this. Does your answer have anything to do with LLMs or AI? All I see is family anecdotes.
      • DennisAleynikov 4y ago
        It's an example of the dangers gpt poses today. Not sure where you got the impression these are family anecdotes
        • f6v 4y ago
          I don’t think the post above makes a good point because there’s no links to evidence. They talk about “declassified archives”, “US spy plane”. Like, what is that? Saying “Russian lies” 10 times in every paragraph doesn’t turn a rant into compelling argument. I mean, I get it, the poster above is Ukrainian and wants to turn every discussion in a certain direction. It’s just there’re many places on the internet they can vent.
          • oneshtein 4y ago
            [flagged]
      • 4y ago
• pinkcan 4y ago
  ChatGPT scales better than "demagogues", so it does have more than a "little practical difference".
• oezi 4y ago
  > The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" I think we are just seeing Dunning-Krüger in the machine: It isn't smart enough to know it doesn't know. It likely isn't very far though.
• Joeri 4y ago
  Your characterization of “dumb people” as somehow being more prone to misinformation is inaccurate and disrespectful. Highly intelligent people are as prone to irrational thinking, and some research suggests even more prone. Go look at some of the most awful personalities on TV or in history, often they are quite intelligent. If you want to school yourself on just how dumb smart people are I suggest going through the back catalog of the “you are not so smart” podcast. Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. ChatGPT is extremely poorly understood. People see it as a text completion engine but with the size of the model and the depth it has it is more accurate in my understanding to see it as a pattern combination and completion engine. The fascinating part is that the human brain is exclusively about patterns, combining and completing them, and those patterns are transferred between generations through language (sight or hearing not required). GPT acquires its patterns in a similar way. A GPT approach may therefore in theory be able to capture all the patterns a human mind can. And maybe not, but I get the impression nobody knows. Yet plenty of smart people have no problem making confident statements either way, which ties back to the beginning of this comment and ironically is exactly what GPT is accused of. Is GPT4 at its ceiling of capability, or is it a path to AGI? I don’t know, and I believe nobody can know. After all, nobody truly understands how these models do what they do, not really. The precautionary principle therefore should apply and we should be wary of training these models further.
  • 23547234573457 4y ago
    [flagged]
• gersh 4y ago
  It looks like it'll be competition for phony experts and politicians. I think it is easier to improve AI algorithms than deal human with human liars.
• furyofantares 4y ago
  > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence Is this performance art? I mean it could end up right but I think you basically just made it up and then stated it confidently.
  • shjake 4y ago
    They didn’t state it particularly confidently though
    • furyofantares 4y ago
      That's true, or at least not in the way ChatGPT does. "Based on my understanding" does convey some lack of confidence in a sense. But it's also implying they might have more understanding than other posters here, when the content of their post indicates they have less.
• nathias 4y ago
  > unless I don't think there is a possible unless, they are told repeatedly not to trust politicians, yet here we are ...
• whiddershins 4y ago
  The real problem is that smart people are even easier to program than stupid people and they do even more damage once they are programmed.
• deleted 4y ago
  [deleted]
• ThunderSizzle 4y ago
  Many of the smarter people are still wrong on what happened on many topics in 2020. They were fooled by various arguments that flew in the face of reality and logic because fear and authority was used instead. The people that avoid this programming isn't based on smart or stupid. It's based on how disagreeable and conscientious you are. A more agreeable and conscientious person can be swayed more easily by confidence and emotional appeals.