5 ms·

I’ll just pick a topic and then ask for a summary and then ask clarifying questions. Probably don't want to start quoting hallucinated facts!

• WendyTheWillow 3y ago
  Good thing podcasts never have false/misleading information on them...
  • vore 3y ago
    This is an absurd argument that you are just making for the sake of argument and you know it. There's a difference between someone getting things wrong and a LLM shamelessly making something up out of the whole cloth.
    • WendyTheWillow 3y ago
      No; I'm much more confident in ChatGPT's accuracy than I am in a random podcast's accuracy. Podcasts tend to be relatively terrible, accuracy wise. If the alternative is learning from ChatGPT, your odds of getting correct information is substantially higher. Podcasts are entertaining! But not where I would go to learn anything.
      • apwell23 3y ago
        why are you forced to listen to "random" podcasts vs podcasts by people you trust? This argument is so dishonest its infuriating.
        • WendyTheWillow 3y ago
          Trust is not a rational behavior, nor is it indicative of reliability of fact.
          • apwell23 3y ago
            Sure but that doesn't make it "random" . I don't get why top comment was listening to podcasts knowing full well that you will be given possibly inaccurate facts. I have hard time accepting that podcasts are audible wikipedia, such a strange take is top voted comment.
            • WendyTheWillow 3y ago
              I listen to podcasts; they’re fun! But I am comfortable operating with incomplete information, and thus know how to treat a low quality source. I also chat with LLMs to better understand topics, and am better off than the folks here who can’t do that as a result. The main thing I’m getting from this discussion is that a lot more very smart people seem to have deluded themselves into thinking knowledge is objective or “locked in” than I had initially realized. The desire for certainty is an extremely human thing, but it’s a dead end, intellectually.
              • apwell23 3y ago
                yea exactly "chatgpt replaced my podcasts" is kind of silly. People don't listen to joe rogan experience to learn facts about how particular jiu jitsu choke works. I don't know what kind of podcasts op replaced with chatgpt but i call BS .
      • Martinussen 3y ago
        Listen to better podcasts. :P
        • WendyTheWillow 3y ago
          It's a mistake to listen to a podcast to learn important information.
          • 4ggr0 3y ago
            That's just like, your opinion, man. There are podcasts for almost every topic where experts are present. Journalists, Scientists, Activists, Researchers etc. can be heard in podcasts, I don't really see why it's generally a mistake to listen to a podcast to learn important information. During the pandemic my partner was attending university from home and listening to their professors via MS Teams, these classes were also recorded so that they could listen to them at a later point. In some ways that's just a professional podcast.
            • WendyTheWillow 3y ago
              I think you know that isn’t a podcast in the sense that we’re discussing. And depending on the class and the institution, I may still trust ChatGPT more than what gets taught.
              • 4ggr0 3y ago
                Of course the part about university classes is different, but you seem to ignore everything else I've said. There are tons of podcasts involving experts talking about their field of expertise, how can it be a mistake to listen to such podcasts to gather information?
                • WendyTheWillow 3y ago
                  It’s a mistake because what drives podcasts are their popularity, not their accuracy. There is no “sort by accuracy” button in any podcasting app, nor are they peer reviewed. Furthermore, podcasts are not a review of the body of knowledge on a subject; they’re often a complete layperson interviewing a single member of a given field, at best. Almost never do the views of any individual actually represent any field as a whole. So once we’ve thrown out the concept of accuracy and completeness, ChatGPT fares exceedingly well in comparison. You’d do much worse than ChatGPT for idle conversation level accuracy.
      • Taurenking 3y ago
        [dead]
      • numpad0 3y ago
        Problem with ChatGPT isn't that they get facts wrong, but that they're what the categorical name suggests, large language models. At one point I came across this series of "are CJK languages related" questions in Quora with cached ChatGPT responses[1], all grammatically correct and very natural, largely turboencabulators, sometimes contradicting even within a single response. Podcasters? They're not _this_ inconsistent. 1: https://imgur.com/a/7tlNDll https://imgur.com/a/7tlNDll
        • Last5Digits 3y ago
          Be aware that you're talking about Quora's implementation of ChatGPT here. As far as I know, the cached answers were generated with an incredibly outdated version, which is definitely not indicative of its current quality. Even worse, I think they actually prime it with answers already posted on the thread, or even just related threads. For example, one of the answers to the first question mentions the same Altaic root as ChatGPTs answer, and I've found multiple people that are seeing their own rephrased answers in the response. If you preprompt ChatGPT with questionable data, then the answer quality will be massively degraded. I've noticed many times now that Bing will rephrase incorrect information or construct a very shallow summary out of unrelated articles when internet searches are allowed, but is able to generate a cohesive and detailed summary when they're disabled. Throwing random answers - some contradicting each other and some talking about subtly different aspects of the topic - into a session without further guidance just isn't a great idea.
        • kaszanka 3y ago
          What's the issue with the answer for "Are Korean and Chinese mutually intelligible"? The highlighted part is definitely true, at least.
          • numpad0 3y ago
            That part is correct, but not consistent with others. My problem is not that GPTs are too often wrong, it's that they always prioritize syntax over facts since they are _language_ models. The sentence "Colorless green ideas" always make more sense to LLM than "Water is wet", simply because the latter is syntactically invalid, and that would be problematic for many use cases including Podcast replacement. Sometimes us humans want AI to say "water is definitively wet", and that has been attempted by forcing LLM to accept that factoids are more syntactically correct, but that isn't a solution and it's still an architectural problem for these pseudo-AGI apps.
      • codethief 3y ago
        Slightly off-topic/meta but this debate reminds me of one people had 20 years ago: Should you trust stuff you read on Wikipedia or not? In the beginning people were skeptical but over time, as Wikipedia matured, the answer has become (I think): Don't blindly trust what you read on Wikipedia but in most cases it's sufficiently accurate as a starting point for further investigation. In fact, I would argue people do trust Wikipedia to a rather high degree these days, sometimes without questioning. Or at least I know I do, whether I want to or not, because I'm so used to Wikipedia being correct. I'm wondering what this means for the future of LLMs: Will we also start trusting them more and more?
    • akho 3y ago
      Is there a similar difference between an LLM getting things wrong and someone shamelessly making something up out of the whole cloth? That's closer to what actually happens.
      • JW_00000 3y ago
        I'm pretty sure that in the podcasts I listen to, no one is shamelessly making something up completely.
    • colechristensen 3y ago
      The kinds of mistakes an LLM makes aren’t nearly as bad as the kind of nonsense you can find humans saying, particularly in podcasts.
    • Fatnino 3y ago
      Podcasts make stuff up from thin air all the time. Sometimes even maliciously.
  • denlekke 3y ago
    they definitely do but i think in podcasts i generally have a better ability to evaluate how much trust i should place in what i'm hearing. i know if i'm listening to a professional chef podcast, i can probably trust they will generally be right talking about how they like to bake a turkey. if i'm listening to the hot wings guy interview zac efron i know to be less blindly trusting of their info on astrophysics. with chatgpt i don't know it's "experience" or "education" on a topic and it has no social accountability motivating it to make sure it gets things right so i can't estimate how much i should trust it in the same way.
    • WendyTheWillow 3y ago
      I guarantee ChatGPT won't lie to you about how to bake a turkey. The paranoia around hallucination is wildly overblown, especially given the low stakes of this context.
      • denlekke 3y ago
        i appreciate your confidence and would love to know how far you would go with the guarantee ! it makes me realize that there is at least one avenue for some level of trust about gpt accuracy and that's my general awareness of how much written content on the topic it probably had access to during training. i think maybe your earlier comment was about the average trustworthiness of all podcasts vs the same for all gpt responses. i would probably side with gpt4 in that context. however, there are plenty of situations where the comparison is between a podcast from the best human in the world at something and gpt which might have less training data or maybe the risks for the topic aren't eating an uncooked turkey but learning cpr wrong or having an airbag not deploy
        • WendyTheWillow 3y ago
          There are zero podcasts from “the best person in the world”, the very concept is absurd. No one person is particularly worth listening to individually, and as a podcast??? Good lord no. LLMs beat podcasts when it comes to, “random exploration of an unfamiliar topic”, every single time. The real issue here is that you trust podcasts so completely, by the way, not that ChatGPT is some oracle of knowledge. More generally, a skill all people need to develop is the ability to explore an idea without accepting whatever you first find. If you’re spending an afternoon talking with ChatGPT about a topic, you should be able to A) use your existing knowledge to give a rough first-pass validation of the information you’re getting, which will catch most hallucinations out of the gate, as they’re rarely subtle, and B) take what you learn with a hefty grain of salt, as if you’re hearing it from a stranger in a bar. This is an important skill, and absolutely applies to both podcasts and LLMs. Honestly why such profound deference to podcasts in particular?
  • superultra 3y ago
    One thing that could happen, sooner rather than later, is that agents are deployed that have access to a facial database of information relevant to the topic. So, there could be a GPT of that day’s NYT Daily with a high ranked database of info on, say, Israel and Hamas, and the text synthesis generates an overview but the user can “ask” for clarifying details to drill down, and responses.
  • deleted 3y ago
    [deleted]
• superultra 3y ago
  I consider the information I’m receiving at a “water cooler” level of factual accuracy. That said there are a few topics I’ve asked where I know enough to know (I had a grad degree in US history) and I’ve found it’s about on par with what you’d dig up through Google. I’ve also found recently that it’ll state when it can’t equivocally say something. Or, in some cases, when I have misunderstood something and repeat it back incorrectly to try and clarify, it’ll correct me. 6 months ago, over text interface, it never would have corrected me: it almost always assumed the user was right and went along with it. But again - it’s not like I’m fact checking Frank at the water cooler (or, often anyway). It’s shooting the shit and it’s great for that.
• mensetmanusman 3y ago
  It can't be worse than TED talks, about half of which suffer from the reproducibility crisis (and therefore likely false).