4 ms·

You're getting some pushback about the analogy to divination, but I think most people here are reasonably technically literate and they assume that everyone els

You're getting some pushback about the analogy to divination, but I think most people here are reasonably technically literate and they assume that everyone else in society has the same understanding of how LLMs work that they do. When I chat about LLM usage with non-technical friends and family it does indeed seem as though they're using these AI chatbots as oracles. When I suggest that they should be wary because these LLMs tend to hallucinate they're generally taken aback - they had no idea that what the chatbot was telling them might not be factually correct. I hope this revelation changes their relationship with LLM chatbots - I think we the technorati need to be educating non-technical users of these things as much as possible in order to demystify them so that people don't treat them like oracles.

kelseyfrog 1y ago
Thank you. I really appreciated your comment. > I think we the technorati need to be educating non-technical users of these things as much as possible in order to demystify them so that people don't treat them like oracles. Exactly. That phrase "meeting people where they're at" comes to mind. Less as a slogan and more as an pedagogical principle. It's not enough to deliver information, it's important to consider how people make sense of the world in the first place. Like you pointed out, the analogy to divination isn't meant to mystify the tech. It's meant to describe how, to many people, this interface feels. And when people interact with a system in a way that feels like consulting an oracle, we can't dismiss that as ignorance. We have to understand it as a real feature of how people relate to symbolic systems. That includes search engines, card catalogs, and yes, LLMs. This is one of the densest concentrations of AI-literate minds on the internet. That's exactly why I think it's worth introducing frames from outside the dominant paradigm: anthropology, semiotics, sociology. It's not to be sill or weird, but to illuminate things engineers might otherwise take for granted. It's easy to forget how much unspoken cultural infrastructure supports what we call "information retrieval." If a few comments dismiss that perspective as silly or unscientific, I don't take it personally. If anything, it reassures me I'm tapping into something unfamiliar but worth sharing and worth having deep discussion on. Thanks again for engaging in good faith. That's the kind of exchange that makes this place valuable.
Kim_Bruning 1y ago
Oh, that's a very important point. Yeah, we definitely want to educate the people around us that these tools/agents are very new technology and far from perfect (and definitely not anything like traditional computation)
QuantumGood 1y ago
I only recommend Perplexity to non-technical users looking for a news or general information interpreter. Others can search the web, but seem not do use search as their primary source.
dotancohen 1y ago
I often phrase it something along these lines: "They are designed to return grammatically valid sentences, not factually correct sentences. If they return something that looks like a fact, either that was in their training data or they made it up to return a grammatically valid sentence. Either way, double check." Of course, nobody listens anyway.
- UncleOxidant 1y ago
  I know someone who is the accountant for a smallish company. He mentioned to me that he was using chatGPT like a spreadsheet. I was like, no you definitely don't want to do that.
flowerbard 1y ago
Ultimately this is about mastery of a tool. The problem is that you can’t teach mastery. I can’t tell someone how to drive in ice in a way where they can really understand it. I can’t explain how certain specific news sources are biased and how to critically think. I can’t explain how to cut wood on a table saw so it’s perfectly straight. The only way to learn is through repeated usage and practice. You can tell users that a LLM can make mistakes — and many tools do — but what does making mistakes really mean? Will it give it a recipe for a cake when I ask for a cupcake? Does it give 14 if I ask to add 3 and 4? Will it agree with me even when I suggest something totally wrong? What does hallucinate mean? That means it will give me a fantasy story if I ask how to change my oil filter?

kelseyfrog 1y ago

Thank you. I really appreciated your comment. > I think we the technorati need to be educating non-technical users of these things as much as possible in order to demystify them so that people don't treat them like oracles. Exactly. That phrase "meeting people where they're at" comes to mind. Less as a slogan and more as an pedagogical principle. It's not enough to deliver information, it's important to consider how people make sense of the world in the first place. Like you pointed out, the analogy to divination isn't meant to mystify the tech. It's meant to describe how, to many people, this interface feels. And when people interact with a system in a way that feels like consulting an oracle, we can't dismiss that as ignorance. We have to understand it as a real feature of how people relate to symbolic systems. That includes search engines, card catalogs, and yes, LLMs. This is one of the densest concentrations of AI-literate minds on the internet. That's exactly why I think it's worth introducing frames from outside the dominant paradigm: anthropology, semiotics, sociology. It's not to be sill or weird, but to illuminate things engineers might otherwise take for granted. It's easy to forget how much unspoken cultural infrastructure supports what we call "information retrieval." If a few comments dismiss that perspective as silly or unscientific, I don't take it personally. If anything, it reassures me I'm tapping into something unfamiliar but worth sharing and worth having deep discussion on. Thanks again for engaging in good faith. That's the kind of exchange that makes this place valuable.

Kim_Bruning 1y ago

Oh, that's a very important point. Yeah, we definitely want to educate the people around us that these tools/agents are very new technology and far from perfect (and definitely not anything like traditional computation)

QuantumGood 1y ago

I only recommend Perplexity to non-technical users looking for a news or general information interpreter. Others can search the web, but seem not do use search as their primary source.

dotancohen 1y ago

I often phrase it something along these lines: "They are designed to return grammatically valid sentences, not factually correct sentences. If they return something that looks like a fact, either that was in their training data or they made it up to return a grammatically valid sentence. Either way, double check." Of course, nobody listens anyway.

UncleOxidant 1y ago
I know someone who is the accountant for a smallish company. He mentioned to me that he was using chatGPT like a spreadsheet. I was like, no you definitely don't want to do that.

flowerbard 1y ago

Ultimately this is about mastery of a tool. The problem is that you can’t teach mastery. I can’t tell someone how to drive in ice in a way where they can really understand it. I can’t explain how certain specific news sources are biased and how to critically think. I can’t explain how to cut wood on a table saw so it’s perfectly straight. The only way to learn is through repeated usage and practice. You can tell users that a LLM can make mistakes — and many tools do — but what does making mistakes really mean? Will it give it a recipe for a cake when I ask for a cupcake? Does it give 14 if I ask to add 3 and 4? Will it agree with me even when I suggest something totally wrong? What does hallucinate mean? That means it will give me a fantasy story if I ask how to change my oil filter?