6 ms·

This is just one of many misconceptions about LLMs. If you are at all involved in trying to build/deploy a product based around LLMs, your head will explode as

This is just one of many misconceptions about LLMs. If you are at all involved in trying to build/deploy a product based around LLMs, your head will explode as you talk to users and realize the vast chasm between how folks who understand them think about and use them, and folks who don't understand them. It's frustrating on both sides. It's quite unclear if we'll ever hit the point where (like, a car for example) someone who has no idea how it works gets as much out of it as someone who does.

SunlitCat 2y ago
And yet, you need a drivers license to operate a car. Computers, AI and other technology everyone is allowed to use, even without basic understanding of how stuff works.
- danielmarkbruce 2y ago
  Generally, any situation that is as dangerous for other people as driving a car requires a license or is illegal. It has nothing to do with understanding the underlying technology. About the last thing we need is a law requiring a license to use ChatGPT.
  - SunlitCat 2y ago
    Operating technology without understanding what you're doing, can be dangerous for other people as well and shouldn't be underestimated. Of course, the license requirement wasn't meant literally, but at least some basic understanding that not downloading everything from everywhere is a good thing to do, or that clicking on that link and entering your details is okay just because the mail said "WE ARE YOUR BANK AND YOU NEED TO GIVE ALL DETAILS!!!" and so on. If you point that out, you are often meet with "Ah. Technology is so difficult to understand. They should make it easier."
- siva7 2y ago
  I really think that an AI license would be the best solution, at least for school kids but ideally for everyone in a professional context. Operating an AI is very different than operating a computer as the former can cause real damage to the tech-"illiterate".
  - EnigmaFlare 2y ago
    Talking to AI can't cause real damage any more than talking to a person can. And talking to a person can do worse. So you'll want a license for that, which is awful.
    - SunlitCat 2y ago
      Of course a license isn't the solution to the main problem itself. The problem is, some people are under the assumption that AI is technology and technology can't ever be wrong and if it's wrong it's the fault of someone else.
      - EnigmaFlare 2y ago
        Are those significant numbers of real people or do you just imagine they exist to make you feel superior? From what I've seen, non-technical people generally see AI as usually wrong. They laugh at it not knowing how many fingers to draw on a person, for instance. Just today, I heard DJs on a local radio station using ChatGPT to write recipes. They had a laugh at the silly things it came up with and concluded that we don't need to worry about AI taking our jobs yet. I think people have a greater weakness for trusting other humans. The world is full of people who do destructive things to themselves and others because they trusted confident liers or other believers who had themselves been fooled. Even on a small group level, people tend to favor ideas from those with higher social status rather than evaluating the ideas themselves. AI not being a respected human wouldn't have that social power.
  - goatlover 2y ago
    Anymore than the wider internet or social media can?
    - SunlitCat 2y ago
      And yet, different governments are regulating social media (and to some extend the wider internet). Not that this is necessarily a good thing, but we have seen in the last few years what can happen if people believe everything someone says on some social media channels.
- margalabargala 2y ago
  I would bet that most people with a driver's license do not understand how cars work.
  - SunlitCat 2y ago
    Funnily I'm doing my drivers license over here currently (yeah, I finally decided that after all those years, I could need one) and there are quite a few technical car related questions in the theoretical exam.
- nomel 2y ago
  > even without basic understanding of how stuff works. That's fantastic! That's the ideal scenario, for all technology: a useful tool. There's a good chance you don't understand how your computer works either. The quantum physics that makes your SD card work, the startup sequence for the PCIE devices in your computer, the wrap around gate's used in your CPU, how the doping materials are mined, how the combine is constructed that was used to make the bread on your sandwich, etc. That's great, and such a relief. You don't have to waste your time on carrying around all that knowledge that took mankind billions of man hours to accumulate. You can just get personally interesting shit done, and use your sandwich at lunchtime without a license.
  - SunlitCat 2y ago
    I totally agree, technology should be a useful tool. But! As more sophisticated a tool becomes, the more care one has to use it. I don't need to understand the inner technical details of my computer, that's true. But I need to understand that connecting to a world wide network can open up a can of worms not only harming me but others along the way as well.
    - nomel 2y ago
      Sure, but at some point, the emergent behavior, like having to worry about your computer being infected, has nothing to do with the actual technology under it, and more about fundamentals of complex systems, and game theory.
hamasho 2y ago
I agree. I think many people confuse LLM abilities because LLMs are so similar to humans that users expect them to do everything humans can do. Remembering what was said is a basic human ability, so it might surprise some users that LLMs can’t do that (although ChatGPT has recently implemented a memory feature).
- Intralexical 2y ago
  I think OpenAI has built a brand on deliberately encouraging the conflation.
alt227 2y ago
> someone who has no idea how it works gets as much out of it as someone who does I seriously doubt that the average driver can get the same out of their car as a trained race driver can.
- ang_cire 2y ago
  There are a lot of things that you just straight can't get from an LLM if you don't understand what a given model is best at. Some models will spit out decently-good python code, but fail to consistently make a sonnet in proper iambic-pentameter, while another will do so brilliantly. Most people don't find themselves using a pickup truck when they're trying to race motorcycles, or accidentally get into a Miata when they want a minivan, because even laypeople know what those broad archetypes (coupe/van, truck/motorcycle) are, and signify. With LLMs, that's not the case at all.
  - TeMPOraL 2y ago
    At laypeople level, or even at the level of "anyone who isn't making an in-house tool/business out of LLMs", the only "broad archetypes" that matter is "GPT-4" and "not GPT-4". You don't need to break down the latter, unless you absolutely can't use the former.
roywiggins 2y ago
It doesn't help that LLMs themselves don't know what they're capable of, and will say stuff like "I'll remember this for next time!" when they absolutely won't. Of course people are confused!
- nomel 2y ago
  ChatGPT's new memory makes this much less confusing, since there's an indicator for when a persistent memory is actually stored. And, it does sometimes store it when claimed.
smusamashah 2y ago
For common non-tech folk, this is how computers always worked. You tell these machines to do things for you, and they do. Why are you gasping at the image/text/audio it just made.
- Suppafly 2y ago
  >For common non-tech folk, this is how computers always worked. That's one thing that a lot of us tech seem to forget, the way we interact with technology is completely different from how most of the population does. It's not just a matter of a few degrees of difference, we operate in entirely different manners.
  - danielmarkbruce 2y ago
    There is a further step with LLMs that seems to blow up peoples mental models. The combo of probabilistic outputs + randomish decoding, 'knowledge' being in parameters + context, generation one token at a time... it's just so freaking weird.
    - Suppafly 2y ago
      Plus it doesn't help that the only way regular end users are able to interact with them is through a chat interface that abstracts away the information being used to prompt a response.
idontpost 2y ago
[dead]

SunlitCat 2y ago

And yet, you need a drivers license to operate a car. Computers, AI and other technology everyone is allowed to use, even without basic understanding of how stuff works.

danielmarkbruce 2y ago
Generally, any situation that is as dangerous for other people as driving a car requires a license or is illegal. It has nothing to do with understanding the underlying technology. About the last thing we need is a law requiring a license to use ChatGPT.
- SunlitCat 2y ago
  Operating technology without understanding what you're doing, can be dangerous for other people as well and shouldn't be underestimated. Of course, the license requirement wasn't meant literally, but at least some basic understanding that not downloading everything from everywhere is a good thing to do, or that clicking on that link and entering your details is okay just because the mail said "WE ARE YOUR BANK AND YOU NEED TO GIVE ALL DETAILS!!!" and so on. If you point that out, you are often meet with "Ah. Technology is so difficult to understand. They should make it easier."
siva7 2y ago
I really think that an AI license would be the best solution, at least for school kids but ideally for everyone in a professional context. Operating an AI is very different than operating a computer as the former can cause real damage to the tech-"illiterate".
- EnigmaFlare 2y ago
  Talking to AI can't cause real damage any more than talking to a person can. And talking to a person can do worse. So you'll want a license for that, which is awful.
  - SunlitCat 2y ago
    Of course a license isn't the solution to the main problem itself. The problem is, some people are under the assumption that AI is technology and technology can't ever be wrong and if it's wrong it's the fault of someone else.
    - EnigmaFlare 2y ago
      Are those significant numbers of real people or do you just imagine they exist to make you feel superior? From what I've seen, non-technical people generally see AI as usually wrong. They laugh at it not knowing how many fingers to draw on a person, for instance. Just today, I heard DJs on a local radio station using ChatGPT to write recipes. They had a laugh at the silly things it came up with and concluded that we don't need to worry about AI taking our jobs yet. I think people have a greater weakness for trusting other humans. The world is full of people who do destructive things to themselves and others because they trusted confident liers or other believers who had themselves been fooled. Even on a small group level, people tend to favor ideas from those with higher social status rather than evaluating the ideas themselves. AI not being a respected human wouldn't have that social power.
- goatlover 2y ago
  Anymore than the wider internet or social media can?
  - SunlitCat 2y ago
    And yet, different governments are regulating social media (and to some extend the wider internet). Not that this is necessarily a good thing, but we have seen in the last few years what can happen if people believe everything someone says on some social media channels.
margalabargala 2y ago
I would bet that most people with a driver's license do not understand how cars work.
- SunlitCat 2y ago
  Funnily I'm doing my drivers license over here currently (yeah, I finally decided that after all those years, I could need one) and there are quite a few technical car related questions in the theoretical exam.
nomel 2y ago
> even without basic understanding of how stuff works. That's fantastic! That's the ideal scenario, for all technology: a useful tool. There's a good chance you don't understand how your computer works either. The quantum physics that makes your SD card work, the startup sequence for the PCIE devices in your computer, the wrap around gate's used in your CPU, how the doping materials are mined, how the combine is constructed that was used to make the bread on your sandwich, etc. That's great, and such a relief. You don't have to waste your time on carrying around all that knowledge that took mankind billions of man hours to accumulate. You can just get personally interesting shit done, and use your sandwich at lunchtime without a license.
- SunlitCat 2y ago
  I totally agree, technology should be a useful tool. But! As more sophisticated a tool becomes, the more care one has to use it. I don't need to understand the inner technical details of my computer, that's true. But I need to understand that connecting to a world wide network can open up a can of worms not only harming me but others along the way as well.
  - nomel 2y ago
    Sure, but at some point, the emergent behavior, like having to worry about your computer being infected, has nothing to do with the actual technology under it, and more about fundamentals of complex systems, and game theory.

hamasho 2y ago

I agree. I think many people confuse LLM abilities because LLMs are so similar to humans that users expect them to do everything humans can do. Remembering what was said is a basic human ability, so it might surprise some users that LLMs can’t do that (although ChatGPT has recently implemented a memory feature).

Intralexical 2y ago
I think OpenAI has built a brand on deliberately encouraging the conflation.

alt227 2y ago

> someone who has no idea how it works gets as much out of it as someone who does I seriously doubt that the average driver can get the same out of their car as a trained race driver can.

ang_cire 2y ago
There are a lot of things that you just straight can't get from an LLM if you don't understand what a given model is best at. Some models will spit out decently-good python code, but fail to consistently make a sonnet in proper iambic-pentameter, while another will do so brilliantly. Most people don't find themselves using a pickup truck when they're trying to race motorcycles, or accidentally get into a Miata when they want a minivan, because even laypeople know what those broad archetypes (coupe/van, truck/motorcycle) are, and signify. With LLMs, that's not the case at all.
- TeMPOraL 2y ago
  At laypeople level, or even at the level of "anyone who isn't making an in-house tool/business out of LLMs", the only "broad archetypes" that matter is "GPT-4" and "not GPT-4". You don't need to break down the latter, unless you absolutely can't use the former.

roywiggins 2y ago

It doesn't help that LLMs themselves don't know what they're capable of, and will say stuff like "I'll remember this for next time!" when they absolutely won't. Of course people are confused!

nomel 2y ago
ChatGPT's new memory makes this much less confusing, since there's an indicator for when a persistent memory is actually stored. And, it does sometimes store it when claimed.

smusamashah 2y ago

For common non-tech folk, this is how computers always worked. You tell these machines to do things for you, and they do. Why are you gasping at the image/text/audio it just made.

Suppafly 2y ago
>For common non-tech folk, this is how computers always worked. That's one thing that a lot of us tech seem to forget, the way we interact with technology is completely different from how most of the population does. It's not just a matter of a few degrees of difference, we operate in entirely different manners.
- danielmarkbruce 2y ago
  There is a further step with LLMs that seems to blow up peoples mental models. The combo of probabilistic outputs + randomish decoding, 'knowledge' being in parameters + context, generation one token at a time... it's just so freaking weird.
  - Suppafly 2y ago
    Plus it doesn't help that the only way regular end users are able to interact with them is through a chat interface that abstracts away the information being used to prompt a response.

idontpost 2y ago

[dead]