5 ms·

Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of c

Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient. The flip side was, calligraphy was sufficient evidence for both his education to whoever hired him, and for a recipient of a document, of its official nature. Calligraphy itself or course didn't make him efficient or smart or fair. That's long gone of course, but we had similar heuristics. I am reminded of the Reddit story about an AI-generated mushroom atlas that had factual errors and lead to someone getting poisoned. We can no longer assume that a book is legit simply because it looks legit. The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true. LLMs are fantastic at breaking our heuristics as to what is and isn't legit, but not as good at being right.

matwood 2y ago
> We can no longer assume that a book is legit simply because it looks legit. The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet". I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. Maybe LLM education will shift people back toward skepticism - unlikely, but I can hope.
- mrweasel 2y ago
  One of the issues today is the volume of content produced, and that journalism and professional writing is dying. LLMs produce large amounts of "good enough" quality to make a profit. In the 90s we could reasonably trust that that the major news sites and corporate websites was true, while random forums required a bit more critical reading. Today even formerly trusted sites may be using LLMs to generate content along with automatic translations. I wouldn't necessarily put the blame on LLMs, this just make it easier. The trolls and spammers was always there, now they just have a more powerful tool. The commercial sites now have a tool they don't understand, which they apply liberally, because it reduces cost, or their staff use it, to get out of work, keep up with deadlines or to cover up incompetence. So, not the fault of the LLMs, but their use is worsening existing trends.
  - duskwuff 2y ago
    > Today even formerly trusted sites may be using LLMs to generate content along with automatic translations. Yep - or they're commingling promotional content with their journalism, a la Forbes / CNN / CNET / About.com / etc. There's still quality content online but it's getting harder to find under the tidal wave of garbage.
- honzabe 2y ago
  > I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. I think that previous generations were not any different. For most people, trusting is the default mode and you need to learn to distrust a source. I know many people who still have not learned that about the internet in general. These are often older people. They believe insane things just because there exists a nicely looking website claiming that thing.
  - DowagerDave 2y ago
    Not sure of the context here is for "previous generation" but I've been around since early in the transition from university/military network to public network, and the reality was the internet just wasn't that big, and it was primarily made up of people who looked, acted and valued the same things. Now it's not even the website of undetermined providence that is believed; positions are established based on just the headline, shared 2nd or 3rd hand!
- SllX 2y ago
  > The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet". I received the same warnings, actually it was more like “don’t trust everything you read on the internet”, but it quickly became apparent that the last three words were redundant, and could have been rephrased more accurately as “don’t trust everything you read and hear and see”. Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. If you can’t change what someone sees, then you can probably change how they see it.
  - DowagerDave 2y ago
    I feel like there was also a brief window where "many amateur eyes in public" trumped "private experts"; wikipedia, open source software, etc. This doesn't seem the case in a hyper-partisan and bifurcated society where there is little trust.
  - throwaway98797 2y ago
    Believe half of what you see and none of what you hear
    - SllX 2y ago
      True!
  - ben_w 2y ago
    > Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. Indeed. When I was 14-18 in the UK, the opinion pieces in the news were derogatorily describing "media studies" as "Mickey Mouse studies". In retrospect such courses were teaching critical analysis of media sources, in much the same way that my history GCSE went into the content of historical media and explored how both primary and secondary sources each had the potential to be either accurate or biased. Even now, even here, I see people treat the media itself as pure and only the people being reported upon as capable of wrongdoing — e.g. insisting that climate scientists in the 70s generally expected an imminent ice age, because that's what the newspapers were saying.
sevensor 2y ago
> Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient. Similar story! Family lore has it that he was from a farming family of modest means, but he was hired to write insurance policies because of his beautiful handwriting, and this was a big step up in the world.
newswasboring 2y ago
> The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true. What?! Someone just made up something and then got mad at it. This is specially weird when you even acknowledge its a made up story. If we start evaluating new things like this nothing will ever progress.
llm_trw 2y ago
>That's long gone of course, but we had similar heuristics. To quote someone about this: >>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life. A book looking legit, a paper being peer reviewed, an expert saying something, none of those things were _ever_ good heuristics. It's just that it was the done thing. Now we have to face the fact that our heuristics are obviously broken and we have to start thinking about every topic. To quote someone else about this: >>Most people would rather die than think. Which explains neatly the politics of the last 10 years.
- hprotagonist 2y ago
  > To quote someone about this: >>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life. So, same as it ever was? Smoke, nothing but smoke. [That’s what the Quester says.] There’s nothing to anything—it’s all smoke. What’s there to show for a lifetime of work, a lifetime of working your fingers to the bone? One generation goes its way, the next one arrives, but nothing changes—it’s business as usual for old planet earth. The sun comes up and the sun goes down, then does it again, and again—the same old round. The wind blows south, the wind blows north. Around and around and around it blows, blowing this way, then that—the whirling, erratic wind. All the rivers flow into the sea, but the sea never fills up. The rivers keep flowing to the same old place, and then start all over and do it again. Everything’s boring, utterly boring— no one can find any meaning in it. Boring to the eye, boring to the ear. What was will be again, what happened will happen again. There’s nothing new on this earth. Year after year it’s the same old thing. Does someone call out, “Hey, this is new”? Don’t get excited—it’s the same old story. Nobody remembers what happened yesterday. And the things that will happen tomorrow? Nobody’ll remember them either. Don’t count on being remembered. c. 450BC
  - llm_trw 2y ago
    One is a complaint that everything is constantly changing, the other that nothing ever changes. I don't think you could misunderstand what either is trying to say harder if you tried.
ziml77 2y ago
The mushroom thing is almost certainly true. There's tons of trash AI generated foraging books being published to Amazon. Atomic Shrimp has a video on it.

matwood 2y ago

> We can no longer assume that a book is legit simply because it looks legit. The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet". I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. Maybe LLM education will shift people back toward skepticism - unlikely, but I can hope.

mrweasel 2y ago
One of the issues today is the volume of content produced, and that journalism and professional writing is dying. LLMs produce large amounts of "good enough" quality to make a profit. In the 90s we could reasonably trust that that the major news sites and corporate websites was true, while random forums required a bit more critical reading. Today even formerly trusted sites may be using LLMs to generate content along with automatic translations. I wouldn't necessarily put the blame on LLMs, this just make it easier. The trolls and spammers was always there, now they just have a more powerful tool. The commercial sites now have a tool they don't understand, which they apply liberally, because it reduces cost, or their staff use it, to get out of work, keep up with deadlines or to cover up incompetence. So, not the fault of the LLMs, but their use is worsening existing trends.
- duskwuff 2y ago
  > Today even formerly trusted sites may be using LLMs to generate content along with automatic translations. Yep - or they're commingling promotional content with their journalism, a la Forbes / CNN / CNET / About.com / etc. There's still quality content online but it's getting harder to find under the tidal wave of garbage.
honzabe 2y ago
> I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. I think that previous generations were not any different. For most people, trusting is the default mode and you need to learn to distrust a source. I know many people who still have not learned that about the internet in general. These are often older people. They believe insane things just because there exists a nicely looking website claiming that thing.
- DowagerDave 2y ago
  Not sure of the context here is for "previous generation" but I've been around since early in the transition from university/military network to public network, and the reality was the internet just wasn't that big, and it was primarily made up of people who looked, acted and valued the same things. Now it's not even the website of undetermined providence that is believed; positions are established based on just the headline, shared 2nd or 3rd hand!
SllX 2y ago
> The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet". I received the same warnings, actually it was more like “don’t trust everything you read on the internet”, but it quickly became apparent that the last three words were redundant, and could have been rephrased more accurately as “don’t trust everything you read and hear and see”. Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. If you can’t change what someone sees, then you can probably change how they see it.
- DowagerDave 2y ago
  I feel like there was also a brief window where "many amateur eyes in public" trumped "private experts"; wikipedia, open source software, etc. This doesn't seem the case in a hyper-partisan and bifurcated society where there is little trust.
- throwaway98797 2y ago
  Believe half of what you see and none of what you hear
  - SllX 2y ago
    True!
- ben_w 2y ago
  > Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. Indeed. When I was 14-18 in the UK, the opinion pieces in the news were derogatorily describing "media studies" as "Mickey Mouse studies". In retrospect such courses were teaching critical analysis of media sources, in much the same way that my history GCSE went into the content of historical media and explored how both primary and secondary sources each had the potential to be either accurate or biased. Even now, even here, I see people treat the media itself as pure and only the people being reported upon as capable of wrongdoing — e.g. insisting that climate scientists in the 70s generally expected an imminent ice age, because that's what the newspapers were saying.

sevensor 2y ago

> Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient. Similar story! Family lore has it that he was from a farming family of modest means, but he was hired to write insurance policies because of his beautiful handwriting, and this was a big step up in the world.

newswasboring 2y ago

> The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true. What?! Someone just made up something and then got mad at it. This is specially weird when you even acknowledge its a made up story. If we start evaluating new things like this nothing will ever progress.

llm_trw 2y ago

>That's long gone of course, but we had similar heuristics. To quote someone about this: >>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life. A book looking legit, a paper being peer reviewed, an expert saying something, none of those things were _ever_ good heuristics. It's just that it was the done thing. Now we have to face the fact that our heuristics are obviously broken and we have to start thinking about every topic. To quote someone else about this: >>Most people would rather die than think. Which explains neatly the politics of the last 10 years.

hprotagonist 2y ago
> To quote someone about this: >>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life. So, same as it ever was? Smoke, nothing but smoke. [That’s what the Quester says.] There’s nothing to anything—it’s all smoke. What’s there to show for a lifetime of work, a lifetime of working your fingers to the bone? One generation goes its way, the next one arrives, but nothing changes—it’s business as usual for old planet earth. The sun comes up and the sun goes down, then does it again, and again—the same old round. The wind blows south, the wind blows north. Around and around and around it blows, blowing this way, then that—the whirling, erratic wind. All the rivers flow into the sea, but the sea never fills up. The rivers keep flowing to the same old place, and then start all over and do it again. Everything’s boring, utterly boring— no one can find any meaning in it. Boring to the eye, boring to the ear. What was will be again, what happened will happen again. There’s nothing new on this earth. Year after year it’s the same old thing. Does someone call out, “Hey, this is new”? Don’t get excited—it’s the same old story. Nobody remembers what happened yesterday. And the things that will happen tomorrow? Nobody’ll remember them either. Don’t count on being remembered. c. 450BC
- llm_trw 2y ago
  One is a complaint that everything is constantly changing, the other that nothing ever changes. I don't think you could misunderstand what either is trying to say harder if you tried.

ziml77 2y ago

The mushroom thing is almost certainly true. There's tons of trash AI generated foraging books being published to Amazon. Atomic Shrimp has a video on it.