3 ms·

I think you hit the nail on the head. Without years of learning by doing, experience in the saddle as you put it, who would be equipped to judge or edit the out

I think you hit the nail on the head. Without years of learning by doing, experience in the saddle as you put it, who would be equipped to judge or edit the output of AI? And as knowledge workers with hands-on experience age out of the workforce, who will replace us? The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. We don't usually need to worry that a calculator might be giving us the wrong result, or an inferior result. It simply gives us an objective fact. Whereas the output of LLMs can be subjectively considered good or bad - even when it is accurate. So imagine teaching an architecture student to draw plans for a house, with a calculator that spit out incorrect values 20% of the time, or silently developed an opinion about the height of countertops. You'd not just have a structurally unsound plan, you'd also have a student who'd failed to learn anything useful.

MrDarcy 9mo ago
On the other hand the incorrect values may drive architects to think more critically about what their tools are producing.
- noduerme 9mo ago
  On the whole, not trusting one's own tools is a regression, not an advancement. The cognitive load it imposes on even the most capable and careful person can lead to all sorts of downstream effects.
hamasho 9mo ago
> The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. This really resonates with me. If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. We are using AI for a lot of small tasks inside big systems, or even for designing the entire architecture, and we still need to validate the answers by ourselves, at least for the foreseeable future. But outsourcing thinking reduces a lot of brain powers to do that, because it often requires understanding problems' detailed structure and internal thinking path. In current situation, by vibing and YOLOing most problems, we are losing the very ability we still need and can't replace with AI or other tools.
- zephen 9mo ago
  > If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. I think past successes have led to a category error in the thinking of a lot of people. For example, the internet, and many constituent parts of the internet, are built on a base of fallible hardware. But mitigated hardware errors, whether equipment failures, alpha particles, or other, are uncorrelated. If you had three uncorrelated calculators that each worked 99.99% of the time, and you used them to check each other, you'd be fine. But three seemingly uncorrelated LLMs? No fucking way.
  - firejake308 9mo ago
    The LLMs are not uncorrelated, though, they're all trained on the same dataset (the Internet) and subject to most of the same biases
    - zephen 9mo ago
      Agreed. This is why I differentiated "uncorrelated" from "seemingly uncorrelated." Sorry if that wasn't clear.
  - noduerme 9mo ago
    There's another category error compounding this issue: People think that because past revolutions in technology eventually led to higher living standards after periods of disruption, this one will too. I think this one is the exception for the reasons enumerated by the parent's blog post.
    - zephen 9mo ago
      Agreed. In point of fact, most technological revolutions have fairly immediately benefited a significant number of people in addition to those in the top 1% -- either by increasing demand for labor, or reducing the price of goods, or both. The promise of LLMs is that they benefit people in the top 1% (investors and highly paid specialists) by reducing the demand for labor to produce the same stuff that was already being produced. There is an incidental initial increase in (or perhaps just reallocation of) labor to build out infrastructure, but that is possibly quite short-lived, and simultaneously drives a huge increase in the cost of electricity, buildings, and computer-related goods. But the benefits of new technologies are never spread evenly. When the technology of travel made remote destinations more accessible, it created tourist traps. Some well placed individuals and companies do well out of this, but typically, most people living near tourist traps suffer from the crowds and increased prices. When power plants are built, neighbors suffer noise and pollution, but other people can turn their lights on. We haven't yet begun to be able to calculate all the negative externalities of LLMs. I would not be surpised if the best negative externality comparisons were to the work of Thomas Midgley, who gifted the world both leaded gasoline and CFC refrigerants.
knollimar 9mo ago
It's funny, I'm working on trying to get LLMs to place electrical devices, and it silently developed opinions that my switches above countertops should be at 4 feet and not the 3'10 I'm asking for (the top cannot be above 4')
- noduerme 9mo ago
  That's quite funny, and almost astonishing, because I'm not an architect, and that scenario just came out of my head randomly as I wrote it. It seemed like something an architect friend of mine who passed away recently, and was a big fan of Douglas Adams, would have joked about. Maybe I just channeled him from the afterlife, and maybe he's also laughing about it.
  - knollimar 9mo ago
    They tend to develop silent opinions based on rules of thumb, so it's not actually reasoning that my symbol is to the center not the top. I fear for trying to get it to unlearn code from the last building code cycle when there's changes

MrDarcy 9mo ago

On the other hand the incorrect values may drive architects to think more critically about what their tools are producing.

noduerme 9mo ago
On the whole, not trusting one's own tools is a regression, not an advancement. The cognitive load it imposes on even the most capable and careful person can lead to all sorts of downstream effects.

hamasho 9mo ago

> The critical difference between AI and a tool like a calculator, to me, is that a calculator's output is accurate, deterministic and provably true. This really resonates with me. If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. We are using AI for a lot of small tasks inside big systems, or even for designing the entire architecture, and we still need to validate the answers by ourselves, at least for the foreseeable future. But outsourcing thinking reduces a lot of brain powers to do that, because it often requires understanding problems' detailed structure and internal thinking path. In current situation, by vibing and YOLOing most problems, we are losing the very ability we still need and can't replace with AI or other tools.

zephen 9mo ago
> If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them. I think past successes have led to a category error in the thinking of a lot of people. For example, the internet, and many constituent parts of the internet, are built on a base of fallible hardware. But mitigated hardware errors, whether equipment failures, alpha particles, or other, are uncorrelated. If you had three uncorrelated calculators that each worked 99.99% of the time, and you used them to check each other, you'd be fine. But three seemingly uncorrelated LLMs? No fucking way.
- firejake308 9mo ago
  The LLMs are not uncorrelated, though, they're all trained on the same dataset (the Internet) and subject to most of the same biases
  - zephen 9mo ago
    Agreed. This is why I differentiated "uncorrelated" from "seemingly uncorrelated." Sorry if that wasn't clear.
- noduerme 9mo ago
  There's another category error compounding this issue: People think that because past revolutions in technology eventually led to higher living standards after periods of disruption, this one will too. I think this one is the exception for the reasons enumerated by the parent's blog post.
  - zephen 9mo ago
    Agreed. In point of fact, most technological revolutions have fairly immediately benefited a significant number of people in addition to those in the top 1% -- either by increasing demand for labor, or reducing the price of goods, or both. The promise of LLMs is that they benefit people in the top 1% (investors and highly paid specialists) by reducing the demand for labor to produce the same stuff that was already being produced. There is an incidental initial increase in (or perhaps just reallocation of) labor to build out infrastructure, but that is possibly quite short-lived, and simultaneously drives a huge increase in the cost of electricity, buildings, and computer-related goods. But the benefits of new technologies are never spread evenly. When the technology of travel made remote destinations more accessible, it created tourist traps. Some well placed individuals and companies do well out of this, but typically, most people living near tourist traps suffer from the crowds and increased prices. When power plants are built, neighbors suffer noise and pollution, but other people can turn their lights on. We haven't yet begun to be able to calculate all the negative externalities of LLMs. I would not be surpised if the best negative externality comparisons were to the work of Thomas Midgley, who gifted the world both leaded gasoline and CFC refrigerants.

knollimar 9mo ago

It's funny, I'm working on trying to get LLMs to place electrical devices, and it silently developed opinions that my switches above countertops should be at 4 feet and not the 3'10 I'm asking for (the top cannot be above 4')

noduerme 9mo ago
That's quite funny, and almost astonishing, because I'm not an architect, and that scenario just came out of my head randomly as I wrote it. It seemed like something an architect friend of mine who passed away recently, and was a big fan of Douglas Adams, would have joked about. Maybe I just channeled him from the afterlife, and maybe he's also laughing about it.
- knollimar 9mo ago
  They tend to develop silent opinions based on rules of thumb, so it's not actually reasoning that my symbol is to the center not the top. I fear for trying to get it to unlearn code from the last building code cycle when there's changes