4 ms·

From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and pe

by poszlem 6mo ago

From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis." "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune

TeMPOraL 6mo ago
> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune I always preferred this take: “Civilization advances by extending the number of important operations which we can perform without thinking of them.” ― Alfred North Whitehead It's both opposite and complementary to your Frank Herbert quote.
- notpachet 6mo ago
  I guess it hinges on your definition of "civilization".
  - hackable_sand 6mo ago
    Or "advance"
- delecti 6mo ago
  I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there. > “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
- ori_b 6mo ago
  It's very clear to me that many people have achieved peak civilization -- no evidence of thought remains.
gdulli 6mo ago
Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
- chungusamongus 6mo ago
  I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao
  - yubblegum 6mo ago
    I agree that many people miss the subtle irony of Frank Herbert's books. He seems to be debating himself to a certain extent in that series. That said, there is no obvious reason to posit that the intergalactic feudal system, CHOAM, or the empire, came to be because of the butlerian jihad. The concrete side effects of the jihad were in fact hyper specialization of cognitive faculties in humans: mentats, guild navigators, and soldiers all possess super human specialized abilities.
    - chungusamongus 6mo ago
      Yeah. I'm not saying the jihad was the cause. But elimating AI didn't prevent the feudal system. It didn't really help, in other words. Honestly, I kinda think Herbert just didn't want to have AI or sophisticated robots in his narrative, so he contrived an elaborate reason why that tech doesn't exist.
      - yubblegum 6mo ago
        But it did "help". Mentats had supercomputer computation capabilities; navigators folded space and charted non-collision paths; warriors had robocop like abilities. These were developed precisely because the use of thinking machines was forbidden. I don't think feudalism is going away one way or another. It persists [in various forms] because of certain biological realities, ranging from genetics to loyalty engendered by familial relations. [This is merely an observation.] In sum, the argument against current AI trends isn't that once addressed we will wake up in utopia. No. The point is that these natural tendencies of humans are hugely amplified and set in generational stone once the elite have control over thinking machines and lord it over a population that has experienced generationally diminished independent cognitive abilities. p.s. All this somehow reminded me of 'Spock's Brain' episode of Startrek /g Note: the elite there were overcome because Kirk and his landing team were cognitive high performers ..
wmeredith 6mo ago
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing. Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"
- miltonlost 6mo ago
  And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).
  - mwigdahl 6mo ago
    The real question is whether we're worse off or better off overall than we were in Socrates' Athens.
    - randallsquared 6mo ago
      There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.
      - mwigdahl 6mo ago
        Agreed. And I think he was wrong. Literacy allows individual humans to be exposed to and understand far more of the world's culture and knowledge than the conveyance of knowledge through recitation of epic poems would ever have allowed. Hell, I would never have had the pleasure of arguing with you without it! :)

by poszlem 6mo ago

TeMPOraL 6mo ago

> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune I always preferred this take: “Civilization advances by extending the number of important operations which we can perform without thinking of them.” ― Alfred North Whitehead It's both opposite and complementary to your Frank Herbert quote.

notpachet 6mo ago
I guess it hinges on your definition of "civilization".
- hackable_sand 6mo ago
  Or "advance"
delecti 6mo ago
I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there. > “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
ori_b 6mo ago
It's very clear to me that many people have achieved peak civilization -- no evidence of thought remains.

gdulli 6mo ago

Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

chungusamongus 6mo ago
I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao
- yubblegum 6mo ago
  I agree that many people miss the subtle irony of Frank Herbert's books. He seems to be debating himself to a certain extent in that series. That said, there is no obvious reason to posit that the intergalactic feudal system, CHOAM, or the empire, came to be because of the butlerian jihad. The concrete side effects of the jihad were in fact hyper specialization of cognitive faculties in humans: mentats, guild navigators, and soldiers all possess super human specialized abilities.
  - chungusamongus 6mo ago
    Yeah. I'm not saying the jihad was the cause. But elimating AI didn't prevent the feudal system. It didn't really help, in other words. Honestly, I kinda think Herbert just didn't want to have AI or sophisticated robots in his narrative, so he contrived an elaborate reason why that tech doesn't exist.
    - yubblegum 6mo ago
      But it did "help". Mentats had supercomputer computation capabilities; navigators folded space and charted non-collision paths; warriors had robocop like abilities. These were developed precisely because the use of thinking machines was forbidden. I don't think feudalism is going away one way or another. It persists [in various forms] because of certain biological realities, ranging from genetics to loyalty engendered by familial relations. [This is merely an observation.] In sum, the argument against current AI trends isn't that once addressed we will wake up in utopia. No. The point is that these natural tendencies of humans are hugely amplified and set in generational stone once the elite have control over thinking machines and lord it over a population that has experienced generationally diminished independent cognitive abilities. p.s. All this somehow reminded me of 'Spock's Brain' episode of Startrek /g Note: the elite there were overcome because Kirk and his landing team were cognitive high performers ..

wmeredith 6mo ago

> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing. Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"

miltonlost 6mo ago
And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).
- mwigdahl 6mo ago
  The real question is whether we're worse off or better off overall than we were in Socrates' Athens.
  - randallsquared 6mo ago
    There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.
    - mwigdahl 6mo ago
      Agreed. And I think he was wrong. Literacy allows individual humans to be exposed to and understand far more of the world's culture and knowledge than the conveyance of knowledge through recitation of epic poems would ever have allowed. Hell, I would never have had the pleasure of arguing with you without it! :)