5 ms·

It certainly sounds like it's doing that. Though if you're an expert in a subject you ask it about, you can see when it transitions from "knowledge of the subj

by Syonyk 3y ago

It certainly sounds like it's doing that. Though if you're an expert in a subject you ask it about, you can see when it transitions from "knowledge of the subject" to "Your uncle who made a living hustling pool and can tell a story made of nothing but whoppers of lies, sounding like the most honest person you've ever met." It's reasonable to assume it does the same thing over subjects you know less about.

darkerside 3y ago
Which also happens with tons of user generated content on the Internet, case in point, this thread
- arp242 3y ago
  But with user generated content like this thread other people can criticize each other and discuss things. If someone is spectacularly wrong then chances are that will be pointed out. ChatGPT is essentially an echo chamber between you and ChatGPT.
  - darkerside 3y ago
    That's a personal problem, not a technology problem. If you let yourself be brainwashed by your weird uncle, alt news, or ChatGPT, that's gotta be on you.
    - arp242 3y ago
      "Brainwashed" is a hugely extremist scenario. The point is that if I ask about topics I don't know about, then by definition I can't judge what is or isn't true. That is also the case on HN where I'm reading a discussion about a topic I don't know about, but that on HN there are some signals such as votes and replies to comments, whereas on ChatGPT there are no external signals. Is this perfect? Of course not. Does it usually work kind of okay? I'd say it does. For books or sites or most things these signals exist too; I can look up what others have said about it. Even your uncle at a birthday party has some controls because other family members can call him out on hos bullshit, or they can later tell "psst, what Jack told you is a load of rubbish", or whatever. There is no "generic review" of ChatGPT because it tells every single person someone else. There is no shared content we're both looking at. ChatGPT very much works in a substantially different way than most other public sources of information.
      - darkerside 3y ago
        > For books or sites or most things these signals exist too; I can look up what others have said about it. That's no different from ChatGPT. The control on books at least is a publisher, but that is not really the case for a website. That's why fake news is a real issue. And your uncle may talk to you in private. Or maybe it's your parents around the family dinner table. There are many sources of dangerous misinformation, not just ChatGPT. Education and critical thinking is the real solution.
        arp242 3y ago
        Let's not lose track of what this discussion is about: it's not about whether your uncle might be feeding you bullshit, it's about whether ChatGPT is a good source to learn about new topics you're not knowledgable on. Your crackpot uncle is not the baseline here – a well-reviewed book (or documentary series, or podcast, or whatever) is. That there are crackpot books out there is besides the point; I can trivially find out which the good books and crackpot books are. With ChatGPT you need to verify every single reply it gives you. And there is nuance here. Take Horrible Histories. I like Horrible Histories – it's fun, and generally the history is accurate. But it's not perfect and has mistakes, and there's a ton of lists enumerating these mistakes, so you can safely watch the show and then read these lists to ensure you're not believing a mistake. There is no "historical inaccuracies in ChatGPT" list. It would be impossible to construct such a list. You need to verify every single reply it gives you, and every detail of that not just the general gist. General critical thinking helps you surprisingly little because many of the signals of reliability that you usually rely on are lacking. Thinking you can trust ChatGPT on topics you're not already fairly knowledgable on and can't be easily empirically tested (e.g. math, programming to some degree) is a good way to get fooled.
        darkerside 3y ago
        I think you overestimate the reliability of the random crap you hear from other human beings on a daily basis
        3y ago
kromem 3y ago
It's really bizarre we ended up with the largest LLM deployment being around something it really doesn't do that well at (knowledge retrieval). Hopefully soon we'll see it reading the story itself, stripping out irrelevant ads/SEO, and summarizing it. I actually have a dream of one day going back to a flip phone where I'll get an AI call summarizing emails or messages that are important, as well as any relevant news stories where I can ask for more detail if needed. The Internet is amazing, but honestly I don't know that the degree to which I personally engage with it is the ideal cost/benefit as opposed to using an intermediary.
- Syonyk 3y ago
  > I actually have a dream of one day going back to a flip phone... Do it. Sonim makes some absolute brick chonks of phones these days. > ...where I'll get an AI call summarizing emails or messages that are important, as well as any relevant news stories where I can ask for more detail if needed. I just try to not have unimportant emails come in, though eBay can't be convinced to turn off their damned shipping notifications (or I've not found how to do it). The 1st of the year is a very good time to start wrangling your inbox - every email that comes in is either of value, or you figure out how to eliminate it. Unsubscribe from everything you don't really care about. And news summary emails are a thing. I route them and various Substack newsletters to my Kobo via Pocket. > The Internet is amazing, but honestly I don't know that the degree to which I personally engage with it is the ideal cost/benefit as opposed to using an intermediary. You get a lot of the benefits with far less use than most people assume sane. I check email via mail clients, when my desktop computers are on. It doesn't come to my phone. I'm off instant messengers most evenings. And I turn my cell phone off at night (which also leads to, on the Sonim XP3+, about two weeks of useful battery life). Just start.

by Syonyk 3y ago

darkerside 3y ago

Which also happens with tons of user generated content on the Internet, case in point, this thread

arp242 3y ago
But with user generated content like this thread other people can criticize each other and discuss things. If someone is spectacularly wrong then chances are that will be pointed out. ChatGPT is essentially an echo chamber between you and ChatGPT.
- darkerside 3y ago
  That's a personal problem, not a technology problem. If you let yourself be brainwashed by your weird uncle, alt news, or ChatGPT, that's gotta be on you.
  - arp242 3y ago
    "Brainwashed" is a hugely extremist scenario. The point is that if I ask about topics I don't know about, then by definition I can't judge what is or isn't true. That is also the case on HN where I'm reading a discussion about a topic I don't know about, but that on HN there are some signals such as votes and replies to comments, whereas on ChatGPT there are no external signals. Is this perfect? Of course not. Does it usually work kind of okay? I'd say it does. For books or sites or most things these signals exist too; I can look up what others have said about it. Even your uncle at a birthday party has some controls because other family members can call him out on hos bullshit, or they can later tell "psst, what Jack told you is a load of rubbish", or whatever. There is no "generic review" of ChatGPT because it tells every single person someone else. There is no shared content we're both looking at. ChatGPT very much works in a substantially different way than most other public sources of information.
    - darkerside 3y ago
      > For books or sites or most things these signals exist too; I can look up what others have said about it. That's no different from ChatGPT. The control on books at least is a publisher, but that is not really the case for a website. That's why fake news is a real issue. And your uncle may talk to you in private. Or maybe it's your parents around the family dinner table. There are many sources of dangerous misinformation, not just ChatGPT. Education and critical thinking is the real solution.
      - arp242 3y ago
        Let's not lose track of what this discussion is about: it's not about whether your uncle might be feeding you bullshit, it's about whether ChatGPT is a good source to learn about new topics you're not knowledgable on. Your crackpot uncle is not the baseline here – a well-reviewed book (or documentary series, or podcast, or whatever) is. That there are crackpot books out there is besides the point; I can trivially find out which the good books and crackpot books are. With ChatGPT you need to verify every single reply it gives you. And there is nuance here. Take Horrible Histories. I like Horrible Histories – it's fun, and generally the history is accurate. But it's not perfect and has mistakes, and there's a ton of lists enumerating these mistakes, so you can safely watch the show and then read these lists to ensure you're not believing a mistake. There is no "historical inaccuracies in ChatGPT" list. It would be impossible to construct such a list. You need to verify every single reply it gives you, and every detail of that not just the general gist. General critical thinking helps you surprisingly little because many of the signals of reliability that you usually rely on are lacking. Thinking you can trust ChatGPT on topics you're not already fairly knowledgable on and can't be easily empirically tested (e.g. math, programming to some degree) is a good way to get fooled.
        darkerside 3y ago
        I think you overestimate the reliability of the random crap you hear from other human beings on a daily basis
        3y ago

kromem 3y ago

It's really bizarre we ended up with the largest LLM deployment being around something it really doesn't do that well at (knowledge retrieval). Hopefully soon we'll see it reading the story itself, stripping out irrelevant ads/SEO, and summarizing it. I actually have a dream of one day going back to a flip phone where I'll get an AI call summarizing emails or messages that are important, as well as any relevant news stories where I can ask for more detail if needed. The Internet is amazing, but honestly I don't know that the degree to which I personally engage with it is the ideal cost/benefit as opposed to using an intermediary.

Syonyk 3y ago
> I actually have a dream of one day going back to a flip phone... Do it. Sonim makes some absolute brick chonks of phones these days. > ...where I'll get an AI call summarizing emails or messages that are important, as well as any relevant news stories where I can ask for more detail if needed. I just try to not have unimportant emails come in, though eBay can't be convinced to turn off their damned shipping notifications (or I've not found how to do it). The 1st of the year is a very good time to start wrangling your inbox - every email that comes in is either of value, or you figure out how to eliminate it. Unsubscribe from everything you don't really care about. And news summary emails are a thing. I route them and various Substack newsletters to my Kobo via Pocket. > The Internet is amazing, but honestly I don't know that the degree to which I personally engage with it is the ideal cost/benefit as opposed to using an intermediary. You get a lot of the benefits with far less use than most people assume sane. I check email via mail clients, when my desktop computers are on. It doesn't come to my phone. I'm off instant messengers most evenings. And I turn my cell phone off at night (which also leads to, on the Sonim XP3+, about two weeks of useful battery life). Just start.