4 ms·

People make these mistakes too. Several times in my high school shop class kids shorted out 9V batteries trying to build circuits because they didn't understand

People make these mistakes too. Several times in my high school shop class kids shorted out 9V batteries trying to build circuits because they didn't understand how electronics work. At no point did our teacher stop them from doing so - on at least one occasion I unplugged one from a breadboard before it got too toasty to handle (and I was/am an electronics nublet). Similarly there was also a lot of hand-wringing about the Gemini pizza glue in a world where people do wacky stuff like cook fish in a dishwasher or defrost chicken overnight on the counter or put cooked steak on the same plate it was on when raw just a few minutes prior. LLMs are just surfacing the fact that assessing and managing risk is an acquired, difficult-to-learn skill. Most people don't know what they don't know and fail to think about what might happen if they do something (correctly or otherwise) before they do it, let alone what they'd do if it goes wrong.

nozzlegear 8mo ago
Well said, but I'd add that LLMs are also surfacing the fact that there's a swathe of people out there who will treat the machines as more trustworthy than humans by default, and don't believe they need to do any assessment or risk management in the first place.
- hnlmorg 8mo ago
  People are just lazy. It’s got nothing to do with LLMs having more trust because they’re a machine because most people would happily trust their friend over an expert. They’d trust the first blog post they find online over an expert. Most people are just too lazy and not skilled enough to perform independent review. And to be fair to those people, coming to topics with a research mindset is genuinely hard and time consuming. So I can’t actually blame people for being lazy. All LLMs do is provide an even easier way to “research”. But it’s not like people were disbelieving random Facebook posts, online scams, and word-of-mouth before LLMs.
  - noduerme 8mo ago
    As right as this may be, it elides the crucial difference between asking LLMs and all the other methods of asking questions you enumerated. The difference is not between the quality of information you might get from a friend or a blog versus an LLM. The difference is the centralization and feeding of the same poor quality information to massive numbers of people at scale. At least whatever bonkers theory someone "researches" on their own is going to be a heterodox set of ideas, with a limited blast radius. Even a major search engine up-ranking a site devoted to, like, how horse dewormers can cure covid, doesn't present it as if that link is the answer to how to cure covid, right? LLMs have a pernicious combination of sounding authoritative while speaking gibberish. Their real skill is not in surfacing the truth from a mass of data, it's in presenting a set of assertions as truth in a way that might satisfy the maximum number of people with limited curiosity, and in establishing an artificial sense of trust. That's why LLMs are likely the most demonic thing ever made by man. They are machines built to lie, tell half-truths, obfuscate and flatter at the same time. Doesn't that sound enough like every religion's warning about the devil?
    - hnlmorg 8mo ago
      But nothing has changed there. People have been posting intelligent-sounding gibberish on social media and blogs for years before LLMs. The problem with centralisation isn’t that it gobbles up data. It’s that it allows those weights to be dictated by a small few who might choose to skew the model more favourably to the messaging they’ve want to promote. And this is a genuine concern. But it’s also not a new problem either. We already have that problem with new broadcasters, newspaper publications, social media ethics teams, and so on and so forth. The new problem LLMs bring to human interaction isn’t any of the issues described above. It’s with LLMs replacing human contact in situations where you need something with a conscience to step in. For example, conversations leading to AI promoting negative thoughts from people with mental health problems because the chat history starts to overwhelm the context window, resulting in the system prompt doing a poorer job of weighting the conversation away from dangerous topics like suicide. This isn’t to say that the points which you’ve addressed aren’t real problems that exist. They definitely do exist. But they’ve also always existed, even before GPT was invented. We’ve just never properly addressed those problems because: either there’s no incentive to. If you are powerful enough to control the narrative then why would you use that power to turn the narrative against you? …or there simply isn’t a good way of solving that problem. eg I might hate stupid conspiracy theories, but censoring research is a much worse alternative. So we just have to allow nutters to share their dumb ideas in the hope that enough legitimate research is published, and enough people are sensible enough to read it, that the nutters don’t have any meaningful impact on society.
- deleted 8mo ago
  [deleted]
azan_ 8mo ago
What’s wrong with dishwasher salmon?
- astura 8mo ago
  It doesn't get hot enough to be a safe cooking method https://youtu.be/dSwzau2_KF8?t=1108 https://youtu.be/dSwzau2_KF8?t=1108
  - wafflemaker 8mo ago
    In Norway we eat plenty of salmon which is quite raw or even raw (in sushi). It has to be frozen and thawed first, to kill parasites. A friend that studied fish production did recommend not eating salmon though and eating trout instead (ørret in Norwegian). Based on scientific evidence difference is pretty small (15% fish not surviving for salmon vs 12% for trout). But rainbow trout does have more DHA per kg.
godelski 8mo ago
What's your point? The AI is being sold as an expert, not a student. These are categorically different things. The mistake in the post is one that can be avoided by taking a single class at a community college. No PhD required, not even a B.S., not even an electricians certificate. So I don't get your point. You're comparing a person in a learning environment to the equivalent of a person claiming to have a PhD in electrical engineering. A student letting the magic smoke escape from a basic circuit is a learnable experience (a memorable one that has high impact), especially when done in a learning environment where an expert can ensure more dangerous mistakes are less likely or non existent. But the same action from a PhD educated engineer would make you reasonably question their qualifications. Yes, humans make mistakes but if you follow the AI's instructions and light things on fire you get sued. If you follow the engineer's instructions and set things on fire then that engineer gets fired likely loses their license. So what is your point?
- bethekidyouwant 8mo ago
  No one thinks their breadboard wont catch on fire because an AI agent told them it wouldn’t. Its never been easier to learn because of these agents.
  - godelski 8mo ago
    That's a very strong claim. I don't think people expect their circuits to ignite, LLM instruction or not. But I'd expect learning from a book or dedicated website would be less likely for that to occur. (Even accounting for bad manufacturing) You're biased because you're not considering that by definition the student is inexperienced. Unknown unknowns. Tons of people don't know very basic things (why would they?) like circuits with capacitors bring dangerous when the power is off. Why are you defending there LLM? Would you be as nice to a person? I'd expect not because these threads tend to point out a person's idiocy. I'm not sure why we give greater leeway to the machine. I'm not sure why we forgive them as if they are a student learning but someone posting similar instructions on a blog gets (rightfully) thrashed. That blog writer is almost never claiming PhD expertise I agree that LLMs can greatly aid in learning. But I also think they can greatly hinder learning. I'm not sure why anyone thinks it's any different than when people got access to the internet. We gave people access to all the information in the world and people "do their own research" and end up making egregious errors because they don't know how to research (naively think it's "searching for information"), what questions to ask, or how to interrogate data (and much more). Instead we've ended up with lots of conspiratorial thinking. Now a sycophantic search engine is going to fix that? I'm unconvinced. Mostly because we can observe the result.
kortilla 8mo ago
The difference is that LLMs pretend to be experts on all things. The high school shop kids aren’t under the impression they can build a smart toaster or whatever.
Majromax 8mo ago
> Several times in my high school shop class kids shorted out 9V batteries trying to build circuits because they didn't understand how electronics work. At no point did our teacher stop them from doing so Yes, and that's okay because the classroom is a learning environment. However, LLMs don't learn; a model that releases the magic smoke in this session will be happy to release it all over again next time. > LLMs are just surfacing the fact that assessing and managing risk is an acquired, difficult-to-learn skill. Which makes the problem worse, not better. If risk management is a difficult skill, then that means we can't extrapolate from 'easy' demonstrations of said skill to argue that an LLM is generally safe for more sensitive tasks. Overall, it seems like LLMs have a long tail of failures. Even while their mean or median performance is good, they seem exponentially more likely than a similarly-competent human to advise something like `rm -rf /`. This is a deeply unintuitive behaviour, precisely because our 'human-like' intuition is engaged with resepct to the average/median skill.

nozzlegear 8mo ago

Well said, but I'd add that LLMs are also surfacing the fact that there's a swathe of people out there who will treat the machines as more trustworthy than humans by default, and don't believe they need to do any assessment or risk management in the first place.

hnlmorg 8mo ago
People are just lazy. It’s got nothing to do with LLMs having more trust because they’re a machine because most people would happily trust their friend over an expert. They’d trust the first blog post they find online over an expert. Most people are just too lazy and not skilled enough to perform independent review. And to be fair to those people, coming to topics with a research mindset is genuinely hard and time consuming. So I can’t actually blame people for being lazy. All LLMs do is provide an even easier way to “research”. But it’s not like people were disbelieving random Facebook posts, online scams, and word-of-mouth before LLMs.
- noduerme 8mo ago
  As right as this may be, it elides the crucial difference between asking LLMs and all the other methods of asking questions you enumerated. The difference is not between the quality of information you might get from a friend or a blog versus an LLM. The difference is the centralization and feeding of the same poor quality information to massive numbers of people at scale. At least whatever bonkers theory someone "researches" on their own is going to be a heterodox set of ideas, with a limited blast radius. Even a major search engine up-ranking a site devoted to, like, how horse dewormers can cure covid, doesn't present it as if that link is the answer to how to cure covid, right? LLMs have a pernicious combination of sounding authoritative while speaking gibberish. Their real skill is not in surfacing the truth from a mass of data, it's in presenting a set of assertions as truth in a way that might satisfy the maximum number of people with limited curiosity, and in establishing an artificial sense of trust. That's why LLMs are likely the most demonic thing ever made by man. They are machines built to lie, tell half-truths, obfuscate and flatter at the same time. Doesn't that sound enough like every religion's warning about the devil?
  - hnlmorg 8mo ago
    But nothing has changed there. People have been posting intelligent-sounding gibberish on social media and blogs for years before LLMs. The problem with centralisation isn’t that it gobbles up data. It’s that it allows those weights to be dictated by a small few who might choose to skew the model more favourably to the messaging they’ve want to promote. And this is a genuine concern. But it’s also not a new problem either. We already have that problem with new broadcasters, newspaper publications, social media ethics teams, and so on and so forth. The new problem LLMs bring to human interaction isn’t any of the issues described above. It’s with LLMs replacing human contact in situations where you need something with a conscience to step in. For example, conversations leading to AI promoting negative thoughts from people with mental health problems because the chat history starts to overwhelm the context window, resulting in the system prompt doing a poorer job of weighting the conversation away from dangerous topics like suicide. This isn’t to say that the points which you’ve addressed aren’t real problems that exist. They definitely do exist. But they’ve also always existed, even before GPT was invented. We’ve just never properly addressed those problems because: either there’s no incentive to. If you are powerful enough to control the narrative then why would you use that power to turn the narrative against you? …or there simply isn’t a good way of solving that problem. eg I might hate stupid conspiracy theories, but censoring research is a much worse alternative. So we just have to allow nutters to share their dumb ideas in the hope that enough legitimate research is published, and enough people are sensible enough to read it, that the nutters don’t have any meaningful impact on society.
deleted 8mo ago
[deleted]

azan_ 8mo ago

What’s wrong with dishwasher salmon?

astura 8mo ago
It doesn't get hot enough to be a safe cooking method https://youtu.be/dSwzau2_KF8?t=1108 https://youtu.be/dSwzau2_KF8?t=1108
- wafflemaker 8mo ago
  In Norway we eat plenty of salmon which is quite raw or even raw (in sushi). It has to be frozen and thawed first, to kill parasites. A friend that studied fish production did recommend not eating salmon though and eating trout instead (ørret in Norwegian). Based on scientific evidence difference is pretty small (15% fish not surviving for salmon vs 12% for trout). But rainbow trout does have more DHA per kg.

godelski 8mo ago

What's your point? The AI is being sold as an expert, not a student. These are categorically different things. The mistake in the post is one that can be avoided by taking a single class at a community college. No PhD required, not even a B.S., not even an electricians certificate. So I don't get your point. You're comparing a person in a learning environment to the equivalent of a person claiming to have a PhD in electrical engineering. A student letting the magic smoke escape from a basic circuit is a learnable experience (a memorable one that has high impact), especially when done in a learning environment where an expert can ensure more dangerous mistakes are less likely or non existent. But the same action from a PhD educated engineer would make you reasonably question their qualifications. Yes, humans make mistakes but if you follow the AI's instructions and light things on fire you get sued. If you follow the engineer's instructions and set things on fire then that engineer gets fired likely loses their license. So what is your point?

bethekidyouwant 8mo ago
No one thinks their breadboard wont catch on fire because an AI agent told them it wouldn’t. Its never been easier to learn because of these agents.
- godelski 8mo ago
  That's a very strong claim. I don't think people expect their circuits to ignite, LLM instruction or not. But I'd expect learning from a book or dedicated website would be less likely for that to occur. (Even accounting for bad manufacturing) You're biased because you're not considering that by definition the student is inexperienced. Unknown unknowns. Tons of people don't know very basic things (why would they?) like circuits with capacitors bring dangerous when the power is off. Why are you defending there LLM? Would you be as nice to a person? I'd expect not because these threads tend to point out a person's idiocy. I'm not sure why we give greater leeway to the machine. I'm not sure why we forgive them as if they are a student learning but someone posting similar instructions on a blog gets (rightfully) thrashed. That blog writer is almost never claiming PhD expertise I agree that LLMs can greatly aid in learning. But I also think they can greatly hinder learning. I'm not sure why anyone thinks it's any different than when people got access to the internet. We gave people access to all the information in the world and people "do their own research" and end up making egregious errors because they don't know how to research (naively think it's "searching for information"), what questions to ask, or how to interrogate data (and much more). Instead we've ended up with lots of conspiratorial thinking. Now a sycophantic search engine is going to fix that? I'm unconvinced. Mostly because we can observe the result.

kortilla 8mo ago

The difference is that LLMs pretend to be experts on all things. The high school shop kids aren’t under the impression they can build a smart toaster or whatever.

Majromax 8mo ago

> Several times in my high school shop class kids shorted out 9V batteries trying to build circuits because they didn't understand how electronics work. At no point did our teacher stop them from doing so Yes, and that's okay because the classroom is a learning environment. However, LLMs don't learn; a model that releases the magic smoke in this session will be happy to release it all over again next time. > LLMs are just surfacing the fact that assessing and managing risk is an acquired, difficult-to-learn skill. Which makes the problem worse, not better. If risk management is a difficult skill, then that means we can't extrapolate from 'easy' demonstrations of said skill to argue that an LLM is generally safe for more sensitive tasks. Overall, it seems like LLMs have a long tail of failures. Even while their mean or median performance is good, they seem exponentially more likely than a similarly-competent human to advise something like `rm -rf /`. This is a deeply unintuitive behaviour, precisely because our 'human-like' intuition is engaged with resepct to the average/median skill.