6 ms·

I actually wrote up quite a few thoughts related to this a few days ago but my take is far more pessimistic: https://www.neilwithdata.com/outsourced-thinking ht

I actually wrote up quite a few thoughts related to this a few days ago but my take is far more pessimistic: https://www.neilwithdata.com/outsourced-thinking https://www.neilwithdata.com/outsourced-thinking My fundamental argument: The way the average person is using AI today is as "Thinking as a Service" and this is going to have absolutely devastating long term consequences, training an entire generation not to think for themselves.

jakubtomanik 8mo ago
I believe that collectively we passed that point long before the onset of LLMs. I have a feeling that throughout the human history vast amounts of people ware happy to outsource their thinking and even pay to do so. We just used to call those arrangements religions.
- deleted 8mo ago
  [deleted]
- junofan 8mo ago
  That’s a bit cynical. Religion is more like a technology. It was continuously invented to solve problems and increase capacity. Newer religions superseded older and survived based on productive and coercive supremacy.
  - noduerme 8mo ago
    If religion is a technology, it's inarguably one that prevented the development of a lot of other technologies for long periods of time. Whether that was a good thing is open to interpretation.
    - kjkjadksj 8mo ago
      On the other hand it produced a lot of related technology. Calendars, mathematics, writing, agricultural practices, government and economic systems. Most of this stuff emerged as an effort to document and proliferate spiritual ideas.
      - noduerme 8mo ago
        I see your point, but I'd say religion's main technological purpose is as a storage system for the encoding of other technologies (and social patterns) into rituals, the reasons for which don't need to be understood; to the point that it actively discourages examination of their reasons, as what we could call an error-checking protocol. So a religion tends to freeze those technologies in the time at the point of inception, and to treat any reexamining of them as heresy. Calendars are useful for iron age farming, but you can't get past a certain point as a civilization if you're unwilling to reconsider your position that the sun and stars revolve around the earth, for example.
        throw4847285 8mo ago
        This is ahistorical, whiggish nonsense. The actual world is not a game of Civilization II.
        noduerme 8mo ago
        Eh? I was talking about Galileo's trial for heresy.
        throw4847285 8mo ago
        Then you also understand nothing about Galileo.
        kjkjadksj 8mo ago
        I think it is hard to fully remove religious practice from species. I think it exist along a spectrum and that there are base ritualistic behaviors most animals engage with (e.g. a pets ritual around eating or play), organized social order sort of rituals (e.g. birds expecting a particular mating dance performed well and this sensibility shared among the local group of birds), and finally what we observe in our own development as a species, higher religion, but that is merely iteratively developed from layering these simple behaviors onto simple behaviors until the whole is quite elaborate in fact. In that sense I think getting caught up in “religion bad for tech” zeitgeist misses the point of what religion actually is. Collectively shared ritual. Belief in God, and specific shades of that, is just the step of the dance the bird does in this case. Taking a step back, plenty of atheists engage in collectively shared ritual too. Belief in the 9-5, the bludgeon that is the four years to specialize vs lifelong apprentanceships towards true mastery, economics constraining choice rather than pure skill. Do these rituals not also hold our species and technological development back? If we talk about religion, it is worth also considering the mountain of other blockers towards progress we have built for ourselves in this collectively agreed upon daily society ritual we all partake upon.
- latexr 8mo ago
  Religions may outsource opinions on morality, but no one went to their spiritual leader to ask about the Pythagorean theorem or the population of Zimbabwe.
  - cwnyth 8mo ago
    Well, now, that's not actually true: [1] https://plato.stanford.edu/entries/pythagoreanism/ https://plato.stanford.edu/entries/pythagoreanism/ [2] https://en.wikipedia.org/wiki/Pythia https://en.wikipedia.org/wiki/Pythia
    - latexr 8mo ago
      Obviously I was using the Pythagorean theorem as a random not literal example. But I’m also curious about what you mean. Mind linking to the specific relevant parts? Linking to humongous articles doesn’t help much.
      - cwnyth 8mo ago
        I was linking it partially tongue in cheek, but oracles and the auspices in antiquity were specifically not about morality. They were about predicting the future. If you wanted to know if you should invade Carthage on a certain day, you'd check the chickens. Literally. And plenty of medical practices were steeped in religious fare, too. If you go back further, a lot of shamanistic practices divine the facts about the present reality. In the words of Terrence McKenna, "[Shamans] cure disease (and another way of putting that is: they have a remarkable facility for choosing patients who will recover), they predict weather (very important), they tell where game has gone, the movement of game, and they seem to have a paranormal ability to look into questions, as I mentioned, who’s sleeping with who, who stole the chicken, who—you know, social transgressions are an open book to them." All very much dealing with facts, not morality. With regards to Pythagoreanism, Pythagoras himself thought of mathematics in religious ways. From the entry on Pythagoras (https://plato.stanford.edu/entries/pythagoras/ https://plato.stanford.edu/entries/pythagoras/) in the SEP: > The cosmos of the acusmata, however, clearly shows a belief in a world structured according to mathematics, and some of the evidence for this belief may have been drawn from genuine mathematical truths such as those embodied in the “Pythagorean” theorem and the relation of whole number ratios to musical concords. There are numerous sections throughout both of these entries that discuss Pythagoras, mathematics, and religion. Plato too is another fruitful avenue, if you wanted to explore that further.
deleted 8mo ago
[deleted]
roenxi 8mo ago
That would have devastating consequences in the pre-LLM era, yes. What is less obvious is whether it'll be an advantage or disadvantage going forward. It is like observing that cars will make people fat and lazy and have devastating consequences on health outcomes - that is exactly what happened but the net impact was still positive because cars boost wealth, lifestyles and access to healthcare so much that the net impact is probably positive even if people get less exercise. It is unclear that a human thinking about things is going to be an advantage in 10, 20 years. Might be, might not be. In 50 years people will probably be outraged if a human makes an important decision without deferring to an LLM's opinion. I'm quite excited that we seem to be building scaleable superintelligences that can patiently and empathetically explain why people are making stupid political choices and what policy prescriptions would actually get a good outcome based on reading all the available statistical and theoretical literature. Screw people primarily thinking for themselves on that topic, the public has no idea.
- gdulli 8mo ago
  If you told me this was a verbatim cautionary sci-fi short story from 1953 I'd believe it.
  - junofan 8mo ago
    Eh 1953 was more about what’s going to happen to the people left behind, e.g. Childhood’s End. The vast majority of people will be better off having the market-winning AI tell them what to do.
    - beedeebeedee 8mo ago
      Or how about that vast majority gets a decent education and higher standard of living so they can spend time learning and thinking on their own? You and a lot of folks seem to take for granted our unjust economy and its consequences, when we could easily change it.
      - roenxi 8mo ago
noduerme 8mo ago
I think you hit the nail on the head. Without years of learning by doing, experience in the saddle as you put it, who would be equipped to judge or edit the output of AI? And as knowledge workers with hands-on experience age out of the workforce, who will replace us? The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. We don't usually need to worry that a calculator might be giving us the wrong result, or an inferior result. It simply gives us an objective fact. Whereas the output of LLMs can be subjectively considered good or bad - even when it is accurate. So imagine teaching an architecture student to draw plans for a house, with a calculator that spit out incorrect values 20% of the time, or silently developed an opinion about the height of countertops. You'd not just have a structurally unsound plan, you'd also have a student who'd failed to learn anything useful.
- MrDarcy 8mo ago
  On the other hand the incorrect values may drive architects to think more critically about what their tools are producing.
  - noduerme 8mo ago
    On the whole, not trusting one's own tools is a regression, not an advancement. The cognitive load it imposes on even the most capable and careful person can lead to all sorts of downstream effects.
- hamasho 8mo ago
  > The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. This really resonates with me. If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. We are using AI for a lot of small tasks inside big systems, or even for designing the entire architecture, and we still need to validate the answers by ourselves, at least for the foreseeable future. But outsourcing thinking reduces a lot of brain powers to do that, because it often requires understanding problems' detailed structure and internal thinking path. In current situation, by vibing and YOLOing most problems, we are losing the very ability we still need and can't replace with AI or other tools.
rco8786 8mo ago
I'll say that I'm still kinda on the fence here, but I will point out that your argument is exactly the same as the argument against calculators back in the 70s/80s, computers and the internet in the 90s, etc.
- zephen 8mo ago
  To some extent, the argument against calculators is perfectly valid. The cash register says you owe $16.23, you give the cashier $21.28, and all hell breaks loose.
  - Joker_vD 8mo ago
    Um, no? The cashier punches your $21.28 into the register, and it tells her that she needs to give you $5.05 in change.
    - zephen 8mo ago
      At some places this is true. At other places, they take the $20, look confused, and fart around with the change for awhile.
  - 1718627440 8mo ago
    My experience is more that you give €20.28 and the cashier asks you whether you have €1.
    - zephen 8mo ago
      I think Europe is a few years behind the US in many respects, including the dumbing down of the population.
      - 1718627440 8mo ago
        I wish we wouldn't be just behind, but would choose another path, but the fear is that we are really just behind.
benSaiyen 8mo ago
Too late. Outsourcing has already accomplished this. No one is making cool shit for themselves. Everyone is held hostage ensuring Wall Street growth. The "cross our fingers and hope for the best" position we find ourselves in politically is entirely due to labor capture. The US benefited from a social network topology of small businesses. No single business being a lynch pin that would implode everything. Now the economy is a handful of too big to fails eroding links between human nodes by capturing our agency. I argued as hard as I could against shipping electronics manufacturing overseas so the next generation would learn real engineering skills. But 20 something me had no idea how far up the political tree the decision was made back then. I helped train a bunch of people's replacements before the telecom focused network hardware manufacturer I worked for then shut down. American tech workers are now primarily cloud configurators and that's being automated away. This is a decades long play on the part of aging leadership to ensure Americans feel their only choice is capitulate. What are we going to do, start our own manufacturing business? Muricans are fish in a barrel. And some pretty well connected people are hinting at similar sense of what's wrong: https://www.barchart.com/story/news/36862423/weve-done-our-country-a-great-disservice-by-offshoring-nvidias-jensen-huang-says-we-have-to-create-prosperity-for-all-not-just-phds https://www.barchart.com/story/news/36862423/weve-done-our-c...
jordanb 8mo ago
There's an Isaac Asimov story where people are "educated" by programming knowledge into their brains, Matrix style. A certain group of people have something wrong with their brain where they can't be "educated" and are forced to learn by studying and such. The protagonist of the story is one of these people and feels ashamed at his disability and how everyone around him effortlessly knows things he has to struggle to learn. He finds out (SPOILER) that he was actually selected for a "priesthood" of creative/problem solvers, because the education process gives knowledge without the ability to apply it creatively. It allows people to rapidly and easily be trained on some process but not the ability to reason it out.
- kej 8mo ago
  Do you remember the title of that story, by chance?
  - nemobius 8mo ago
    Profession (1957) https://en.wikipedia.org/wiki/Profession_(novella) https://en.wikipedia.org/wiki/Profession_(novella)
  - yaantc 8mo ago
    Profession as sibling said, available here: https://www.inf.ufpr.br/renato/profession.html https://www.inf.ufpr.br/renato/profession.html The wikipedia entry also has link to the text but the above is nicer IMHO, just the raw text. From a previous HN discussion some weeks ago!
godelski 8mo ago
I think the comparison to giving change is a good one, especially given how frequently the LLM hype crowd uses the fictitious "calculator in your pocket" story. I've been in the exact situation you've described, long before LLMs came out and cashiers have had calculators in front of them for longer than we've had smartphones. I'll add another analogy. I tell people when I tip I "round off to the nearest dollar, move the decimal place (10%), and multiply by 2" (generating a tip that will be in the ballpark of 18%), and am always told "that's too complicated". It's a 3 step process where the hardest thing is multiplying a number by 2 (and usually a 2 digit number...). It's always struck me as odd that the response is that this is too complicated rather than a nice tip (pun intended) for figuring out how much to tip quickly and with essentially zero thinking. If any of those three steps appear difficult to you then your math skills are below that of elementary school. I also see a problem with how we look at math and coding. I hear so often "abstraction is bad" yet, that is all coding (and math) is. It is fundamentally abstraction. The ability to abstract is what makes humans human. All creatures abstract, it is a necessary component of intelligence, but humans certainly have a unique capacity for it. Abstraction is no doubt hard, but when in life was anything worth doing easy? I think we unfortunately are willing to put significantly more effort into justifying our laziness than we will to be not lazy. My fear is that we will abdicate doing worthwhile things because they are hard. It's a thing people do every day. So many people love to outsource their thinking. Be it to a calculator, Google, "the algorithm", their favorite political pundit, religion, or anything else. Anything to abdicate responsibility. Anything to abdicate effort. So I think AI is going to be no different from calculators, as you suggest. They can be great tools to help people do so much. But it will be far more commonly used to outsource thinking, even by many people considered intelligent. Skills atrophy. It's as simple as that.
- userbinator 8mo ago
  I briefly taught a beginner CS course over a decade ago, and at the time it was already surprising and disappointing how many of my students would reach for a calculator to do single-digit arithmetic; something that was a requirement to be committed to memory when I was still in school. Not surprisingly, teaching them binary and hex was extremely frustrating. I tell people when I tip I "round off to the nearest dollar, move the decimal place (10%), and multiply by 2" (generating a tip that will be in the ballpark of 18%), and am always told "that's too complicated". I would tell others to "shift right once, then divide by 2 and add" for 15%, and get the same response. However, I'm not so sure what you mean by a problem with thinking that abstraction is bad. Yes, abstraction is bad --- because it is a way to hide and obscure the actual details, and one could argue that such dependence on opaque things, just like a calculator or AI, is the actual problem.
polyrenn 8mo ago
> Can you audit/review/identify issues in a codebase if you've never written code? Actual knowledge about systems work much better more often than not, LLMs are not sentient and still need to be driven to get decent results.

jakubtomanik 8mo ago

I believe that collectively we passed that point long before the onset of LLMs. I have a feeling that throughout the human history vast amounts of people ware happy to outsource their thinking and even pay to do so. We just used to call those arrangements religions.

deleted 8mo ago
[deleted]
junofan 8mo ago
That’s a bit cynical. Religion is more like a technology. It was continuously invented to solve problems and increase capacity. Newer religions superseded older and survived based on productive and coercive supremacy.
- noduerme 8mo ago
  If religion is a technology, it's inarguably one that prevented the development of a lot of other technologies for long periods of time. Whether that was a good thing is open to interpretation.
  - kjkjadksj 8mo ago
    On the other hand it produced a lot of related technology. Calendars, mathematics, writing, agricultural practices, government and economic systems. Most of this stuff emerged as an effort to document and proliferate spiritual ideas.
    - noduerme 8mo ago
      I see your point, but I'd say religion's main technological purpose is as a storage system for the encoding of other technologies (and social patterns) into rituals, the reasons for which don't need to be understood; to the point that it actively discourages examination of their reasons, as what we could call an error-checking protocol. So a religion tends to freeze those technologies in the time at the point of inception, and to treat any reexamining of them as heresy. Calendars are useful for iron age farming, but you can't get past a certain point as a civilization if you're unwilling to reconsider your position that the sun and stars revolve around the earth, for example.
      - throw4847285 8mo ago
        This is ahistorical, whiggish nonsense. The actual world is not a game of Civilization II.
        noduerme 8mo ago
        Eh? I was talking about Galileo's trial for heresy.
        throw4847285 8mo ago
        Then you also understand nothing about Galileo.
      - kjkjadksj 8mo ago
        I think it is hard to fully remove religious practice from species. I think it exist along a spectrum and that there are base ritualistic behaviors most animals engage with (e.g. a pets ritual around eating or play), organized social order sort of rituals (e.g. birds expecting a particular mating dance performed well and this sensibility shared among the local group of birds), and finally what we observe in our own development as a species, higher religion, but that is merely iteratively developed from layering these simple behaviors onto simple behaviors until the whole is quite elaborate in fact. In that sense I think getting caught up in “religion bad for tech” zeitgeist misses the point of what religion actually is. Collectively shared ritual. Belief in God, and specific shades of that, is just the step of the dance the bird does in this case. Taking a step back, plenty of atheists engage in collectively shared ritual too. Belief in the 9-5, the bludgeon that is the four years to specialize vs lifelong apprentanceships towards true mastery, economics constraining choice rather than pure skill. Do these rituals not also hold our species and technological development back? If we talk about religion, it is worth also considering the mountain of other blockers towards progress we have built for ourselves in this collectively agreed upon daily society ritual we all partake upon.
latexr 8mo ago
Religions may outsource opinions on morality, but no one went to their spiritual leader to ask about the Pythagorean theorem or the population of Zimbabwe.
- cwnyth 8mo ago
  Well, now, that's not actually true: [1] https://plato.stanford.edu/entries/pythagoreanism/ https://plato.stanford.edu/entries/pythagoreanism/ [2] https://en.wikipedia.org/wiki/Pythia https://en.wikipedia.org/wiki/Pythia
  - latexr 8mo ago
    Obviously I was using the Pythagorean theorem as a random not literal example. But I’m also curious about what you mean. Mind linking to the specific relevant parts? Linking to humongous articles doesn’t help much.
    - cwnyth 8mo ago
      I was linking it partially tongue in cheek, but oracles and the auspices in antiquity were specifically not about morality. They were about predicting the future. If you wanted to know if you should invade Carthage on a certain day, you'd check the chickens. Literally. And plenty of medical practices were steeped in religious fare, too. If you go back further, a lot of shamanistic practices divine the facts about the present reality. In the words of Terrence McKenna, "[Shamans] cure disease (and another way of putting that is: they have a remarkable facility for choosing patients who will recover), they predict weather (very important), they tell where game has gone, the movement of game, and they seem to have a paranormal ability to look into questions, as I mentioned, who’s sleeping with who, who stole the chicken, who—you know, social transgressions are an open book to them." All very much dealing with facts, not morality. With regards to Pythagoreanism, Pythagoras himself thought of mathematics in religious ways. From the entry on Pythagoras (https://plato.stanford.edu/entries/pythagoras/ https://plato.stanford.edu/entries/pythagoras/) in the SEP: > The cosmos of the acusmata, however, clearly shows a belief in a world structured according to mathematics, and some of the evidence for this belief may have been drawn from genuine mathematical truths such as those embodied in the “Pythagorean” theorem and the relation of whole number ratios to musical concords. There are numerous sections throughout both of these entries that discuss Pythagoras, mathematics, and religion. Plato too is another fruitful avenue, if you wanted to explore that further.

deleted 8mo ago

[deleted]

roenxi 8mo ago

That would have devastating consequences in the pre-LLM era, yes. What is less obvious is whether it'll be an advantage or disadvantage going forward. It is like observing that cars will make people fat and lazy and have devastating consequences on health outcomes - that is exactly what happened but the net impact was still positive because cars boost wealth, lifestyles and access to healthcare so much that the net impact is probably positive even if people get less exercise. It is unclear that a human thinking about things is going to be an advantage in 10, 20 years. Might be, might not be. In 50 years people will probably be outraged if a human makes an important decision without deferring to an LLM's opinion. I'm quite excited that we seem to be building scaleable superintelligences that can patiently and empathetically explain why people are making stupid political choices and what policy prescriptions would actually get a good outcome based on reading all the available statistical and theoretical literature. Screw people primarily thinking for themselves on that topic, the public has no idea.

gdulli 8mo ago
If you told me this was a verbatim cautionary sci-fi short story from 1953 I'd believe it.
- junofan 8mo ago
  Eh 1953 was more about what’s going to happen to the people left behind, e.g. Childhood’s End. The vast majority of people will be better off having the market-winning AI tell them what to do.
  - beedeebeedee 8mo ago
    Or how about that vast majority gets a decent education and higher standard of living so they can spend time learning and thinking on their own? You and a lot of folks seem to take for granted our unjust economy and its consequences, when we could easily change it.
    - roenxi 8mo ago

noduerme 8mo ago

I think you hit the nail on the head. Without years of learning by doing, experience in the saddle as you put it, who would be equipped to judge or edit the output of AI? And as knowledge workers with hands-on experience age out of the workforce, who will replace us? The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. We don't usually need to worry that a calculator might be giving us the wrong result, or an inferior result. It simply gives us an objective fact. Whereas the output of LLMs can be subjectively considered good or bad - even when it is accurate. So imagine teaching an architecture student to draw plans for a house, with a calculator that spit out incorrect values 20% of the time, or silently developed an opinion about the height of countertops. You'd not just have a structurally unsound plan, you'd also have a student who'd failed to learn anything useful.

MrDarcy 8mo ago
On the other hand the incorrect values may drive architects to think more critically about what their tools are producing.
- noduerme 8mo ago
  On the whole, not trusting one's own tools is a regression, not an advancement. The cognitive load it imposes on even the most capable and careful person can lead to all sorts of downstream effects.
hamasho 8mo ago
> The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. This really resonates with me. If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. We are using AI for a lot of small tasks inside big systems, or even for designing the entire architecture, and we still need to validate the answers by ourselves, at least for the foreseeable future. But outsourcing thinking reduces a lot of brain powers to do that, because it often requires understanding problems' detailed structure and internal thinking path. In current situation, by vibing and YOLOing most problems, we are losing the very ability we still need and can't replace with AI or other tools.

rco8786 8mo ago

I'll say that I'm still kinda on the fence here, but I will point out that your argument is exactly the same as the argument against calculators back in the 70s/80s, computers and the internet in the 90s, etc.

zephen 8mo ago
To some extent, the argument against calculators is perfectly valid. The cash register says you owe $16.23, you give the cashier $21.28, and all hell breaks loose.
- Joker_vD 8mo ago
  Um, no? The cashier punches your $21.28 into the register, and it tells her that she needs to give you $5.05 in change.
  - zephen 8mo ago
    At some places this is true. At other places, they take the $20, look confused, and fart around with the change for awhile.
- 1718627440 8mo ago
  My experience is more that you give €20.28 and the cashier asks you whether you have €1.
  - zephen 8mo ago
    I think Europe is a few years behind the US in many respects, including the dumbing down of the population.
    - 1718627440 8mo ago
      I wish we wouldn't be just behind, but would choose another path, but the fear is that we are really just behind.

benSaiyen 8mo ago

Too late. Outsourcing has already accomplished this. No one is making cool shit for themselves. Everyone is held hostage ensuring Wall Street growth. The "cross our fingers and hope for the best" position we find ourselves in politically is entirely due to labor capture. The US benefited from a social network topology of small businesses. No single business being a lynch pin that would implode everything. Now the economy is a handful of too big to fails eroding links between human nodes by capturing our agency. I argued as hard as I could against shipping electronics manufacturing overseas so the next generation would learn real engineering skills. But 20 something me had no idea how far up the political tree the decision was made back then. I helped train a bunch of people's replacements before the telecom focused network hardware manufacturer I worked for then shut down. American tech workers are now primarily cloud configurators and that's being automated away. This is a decades long play on the part of aging leadership to ensure Americans feel their only choice is capitulate. What are we going to do, start our own manufacturing business? Muricans are fish in a barrel. And some pretty well connected people are hinting at similar sense of what's wrong: https://www.barchart.com/story/news/36862423/weve-done-our-country-a-great-disservice-by-offshoring-nvidias-jensen-huang-says-we-have-to-create-prosperity-for-all-not-just-phds https://www.barchart.com/story/news/36862423/weve-done-our-c...

jordanb 8mo ago

There's an Isaac Asimov story where people are "educated" by programming knowledge into their brains, Matrix style. A certain group of people have something wrong with their brain where they can't be "educated" and are forced to learn by studying and such. The protagonist of the story is one of these people and feels ashamed at his disability and how everyone around him effortlessly knows things he has to struggle to learn. He finds out (SPOILER) that he was actually selected for a "priesthood" of creative/problem solvers, because the education process gives knowledge without the ability to apply it creatively. It allows people to rapidly and easily be trained on some process but not the ability to reason it out.

kej 8mo ago
Do you remember the title of that story, by chance?
- nemobius 8mo ago
  Profession (1957) https://en.wikipedia.org/wiki/Profession_(novella) https://en.wikipedia.org/wiki/Profession_(novella)
- yaantc 8mo ago
  Profession as sibling said, available here: https://www.inf.ufpr.br/renato/profession.html https://www.inf.ufpr.br/renato/profession.html The wikipedia entry also has link to the text but the above is nicer IMHO, just the raw text. From a previous HN discussion some weeks ago!

godelski 8mo ago

I think the comparison to giving change is a good one, especially given how frequently the LLM hype crowd uses the fictitious "calculator in your pocket" story. I've been in the exact situation you've described, long before LLMs came out and cashiers have had calculators in front of them for longer than we've had smartphones. I'll add another analogy. I tell people when I tip I "round off to the nearest dollar, move the decimal place (10%), and multiply by 2" (generating a tip that will be in the ballpark of 18%), and am always told "that's too complicated". It's a 3 step process where the hardest thing is multiplying a number by 2 (and usually a 2 digit number...). It's always struck me as odd that the response is that this is too complicated rather than a nice tip (pun intended) for figuring out how much to tip quickly and with essentially zero thinking. If any of those three steps appear difficult to you then your math skills are below that of elementary school. I also see a problem with how we look at math and coding. I hear so often "abstraction is bad" yet, that is all coding (and math) is. It is fundamentally abstraction. The ability to abstract is what makes humans human. All creatures abstract, it is a necessary component of intelligence, but humans certainly have a unique capacity for it. Abstraction is no doubt hard, but when in life was anything worth doing easy? I think we unfortunately are willing to put significantly more effort into justifying our laziness than we will to be not lazy. My fear is that we will abdicate doing worthwhile things because they are hard. It's a thing people do every day. So many people love to outsource their thinking. Be it to a calculator, Google, "the algorithm", their favorite political pundit, religion, or anything else. Anything to abdicate responsibility. Anything to abdicate effort. So I think AI is going to be no different from calculators, as you suggest. They can be great tools to help people do so much. But it will be far more commonly used to outsource thinking, even by many people considered intelligent. Skills atrophy. It's as simple as that.

userbinator 8mo ago
I briefly taught a beginner CS course over a decade ago, and at the time it was already surprising and disappointing how many of my students would reach for a calculator to do single-digit arithmetic; something that was a requirement to be committed to memory when I was still in school. Not surprisingly, teaching them binary and hex was extremely frustrating. I tell people when I tip I "round off to the nearest dollar, move the decimal place (10%), and multiply by 2" (generating a tip that will be in the ballpark of 18%), and am always told "that's too complicated". I would tell others to "shift right once, then divide by 2 and add" for 15%, and get the same response. However, I'm not so sure what you mean by a problem with thinking that abstraction is bad. Yes, abstraction is bad --- because it is a way to hide and obscure the actual details, and one could argue that such dependence on opaque things, just like a calculator or AI, is the actual problem.

polyrenn 8mo ago

> Can you audit/review/identify issues in a codebase if you've never written code? Actual knowledge about systems work much better more often than not, LLMs are not sentient and still need to be driven to get decent results.