5 ms·

Human in the Loop and the Missing Productivity Growth

tuatoru 2y ago
Productivity in knowledge work is not difficult to measure, just very laborious. A few aspects need to be untangled to do it, though. We can use an analogue of the Kaya Identity for CO2 production[1], the production identity:- Dollars of revenue per year equals dollars produced per keystroke, times keystrokes per hour, times hours at work per year. Revenue equals productivity times work intensity times worker utilisation. Worker utilisation is commonly measured and called "productivity". More rarely work intensity is also measured, and the product of intensity and utilisation is called "productivity". Processes are almost never analysed sufficiently deeply to measure true productivity (dollars per keystroke), though. 1. https://en.wikipedia.org/wiki/Kaya_identity https://en.wikipedia.org/wiki/Kaya_identity
- willturman 2y ago
  Productivity in knowledge work is not difficult to measure, just very laborious. A few aspects need to be untangled to do it, though. Goodhart's law [1] is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure". It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. 1. https://en.m.wikipedia.org/wiki/Goodhart's_law https://en.m.wikipedia.org/wiki/Goodhart's_law
- marcosdumay 2y ago
  > dollars per keystroke Economical productivity is almost universally measured in dollars per hour of salary. None of that applies.
  - tuatoru 2y ago
    Comprehensively failing to comprehend what I said. Well done.
cess11 2y ago
"Humans, by nature, do everything they can to avoid cognitive effort." This is why the prevalence of sudoku in human societies can only be explained as a result of harsh coercion.
- intellectronica 2y ago
  :D
- glitchc 2y ago
  Nature rewards certain types of effort with a dopamine rush. As a counterpoint, philosophical discussions are in serious decline across human societies. No dopamine, no interest.
  - cess11 2y ago
    Dopamine rush feels like anticipation and excitement, at best. It can also feel like horror or result in vomiting. Reward and gratification belongs to different neurological systems. Philosophy has always been a bourgeois pastime, and a distinctly occidental one at that.
voidhorse 2y ago
I have a somewhat different take. We do not spend time on something because it is the amount of time to reach some objective standard, rather, we spend the amount of time that we feel is necessary for our personal subjective standard. So, of course, even as tools improve, we'll spend the same amount of time. If it's something we didn't care about in the first place, AI tools won't save us any time because we probably already spent barely any time and effort. If it's something we do care about, sure we might use AI to automate so aspects of the job, but we'll still spend the same amount of time examining and tweaking outputs until we feel it is up to snuff. The real "gains" will only be realized once management eliminates the human operators and accepts the lower quality output that is a consequence of that. Also, writing about productivity in abstract terms is and will forever remain an example of complete stupidity. It is not an objective measurable universal like "amount of oxygen". The definition changes depending on what it is you are actually trying to do. To give an example, I might be trying to eliminate any and all use of AI in my workplace, under this goal, any use of AI is fundamentally counterproductive.
satisfice 2y ago
Any post about productivity, AI, and humans that does not put responsibility at the center is a waste of bandwidth. Responsibility is the ballgame. Irresponsible productivity is not productivity at all. Irresponsible productivity is how disasters like Bhopal and the 737 Max happened. Am I being responsible when I trust my Python interpreter? For most purposes, yes. Am I being responsible when I trust any LLM to do anything remotely important? For most purposes, no. Fanboys make outrageous and untested claims about what AI can do. My productivity hasn’t budged because I am behaving responsibility.
- jaybrendansmith 2y ago
  Indeed. Biggest issue is that even humans cannot always hit the mark in quality. Most process at organizations is focused on consistently hitting a standard of quality. We will therefor not accept a lower standard in order to remove the human operators any time soon. As we've learned with self-driving cars, AI cannot be simply as good as humans, it must be far better in order to replace them.
  - satisfice 2y ago
    The responsibility angle on the problem of competence is to remind people that the critical difference between an incompetent human and incompetent machine is that a human can be held accountable for his actions, whereas “accountability” has no meaning for a machine. A plane crashed in South Korea, recently. The pilots died in the crash. This is as it should be: the people directly responsible for safety put their own lives on the line. There may be other responsible people, and they must be found and prosecuted.

Productivity in knowledge work is not difficult to measure, just very laborious. A few aspects need to be untangled to do it, though. We can use an analogue of the Kaya Identity for CO2 production[1], the production identity:- Dollars of revenue per year equals dollars produced per keystroke, times keystrokes per hour, times hours at work per year. Revenue equals productivity times work intensity times worker utilisation. Worker utilisation is commonly measured and called "productivity". More rarely work intensity is also measured, and the product of intensity and utilisation is called "productivity". Processes are almost never analysed sufficiently deeply to measure true productivity (dollars per keystroke), though. 1. https://en.wikipedia.org/wiki/Kaya_identity https://en.wikipedia.org/wiki/Kaya_identity

willturman 2y ago
Productivity in knowledge work is not difficult to measure, just very laborious. A few aspects need to be untangled to do it, though. Goodhart's law [1] is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure". It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. 1. https://en.m.wikipedia.org/wiki/Goodhart's_law https://en.m.wikipedia.org/wiki/Goodhart's_law
marcosdumay 2y ago
> dollars per keystroke Economical productivity is almost universally measured in dollars per hour of salary. None of that applies.
- tuatoru 2y ago
  Comprehensively failing to comprehend what I said. Well done.

cess11 2y ago

"Humans, by nature, do everything they can to avoid cognitive effort." This is why the prevalence of sudoku in human societies can only be explained as a result of harsh coercion.

intellectronica 2y ago
:D
glitchc 2y ago
Nature rewards certain types of effort with a dopamine rush. As a counterpoint, philosophical discussions are in serious decline across human societies. No dopamine, no interest.
- cess11 2y ago
  Dopamine rush feels like anticipation and excitement, at best. It can also feel like horror or result in vomiting. Reward and gratification belongs to different neurological systems. Philosophy has always been a bourgeois pastime, and a distinctly occidental one at that.

voidhorse 2y ago

I have a somewhat different take. We do not spend time on something because it is the amount of time to reach some objective standard, rather, we spend the amount of time that we feel is necessary for our personal subjective standard. So, of course, even as tools improve, we'll spend the same amount of time. If it's something we didn't care about in the first place, AI tools won't save us any time because we probably already spent barely any time and effort. If it's something we do care about, sure we might use AI to automate so aspects of the job, but we'll still spend the same amount of time examining and tweaking outputs until we feel it is up to snuff. The real "gains" will only be realized once management eliminates the human operators and accepts the lower quality output that is a consequence of that. Also, writing about productivity in abstract terms is and will forever remain an example of complete stupidity. It is not an objective measurable universal like "amount of oxygen". The definition changes depending on what it is you are actually trying to do. To give an example, I might be trying to eliminate any and all use of AI in my workplace, under this goal, any use of AI is fundamentally counterproductive.

satisfice 2y ago

Any post about productivity, AI, and humans that does not put responsibility at the center is a waste of bandwidth. Responsibility is the ballgame. Irresponsible productivity is not productivity at all. Irresponsible productivity is how disasters like Bhopal and the 737 Max happened. Am I being responsible when I trust my Python interpreter? For most purposes, yes. Am I being responsible when I trust any LLM to do anything remotely important? For most purposes, no. Fanboys make outrageous and untested claims about what AI can do. My productivity hasn’t budged because I am behaving responsibility.

jaybrendansmith 2y ago
Indeed. Biggest issue is that even humans cannot always hit the mark in quality. Most process at organizations is focused on consistently hitting a standard of quality. We will therefor not accept a lower standard in order to remove the human operators any time soon. As we've learned with self-driving cars, AI cannot be simply as good as humans, it must be far better in order to replace them.
- satisfice 2y ago
  The responsibility angle on the problem of competence is to remind people that the critical difference between an incompetent human and incompetent machine is that a human can be held accountable for his actions, whereas “accountability” has no meaning for a machine. A plane crashed in South Korea, recently. The pilots died in the crash. This is as it should be: the people directly responsible for safety put their own lives on the line. There may be other responsible people, and they must be found and prosecuted.