19 ms·

Why Doesn't Software Show Up in Productivity?

218 points by Wildgoose 5y ago

roenxi 5y ago
That is a graph of US TFP. I expect that once Asia is included in the mix the change will be a bit more pronounced. Most of the hardware related change in the IT revolution is happening there. The S&P 500 rank 1 company (Apple) would look to an alien like an Asian company since all the actual manufacturing happens there. Also, while I don't think it is necessarily the major driving factor, the US has a capital misallocation problem. People keep sinking fortunes into companies with bad profit margins.
- ZeroGravitas 5y ago
  Also, Total Factor Productivity will include the effects of everything a society does that can be measured in monetary terms. Dumping poison in rivers would probably show up as an "efficiency gain" at least until it causes ecosystem collapse and widespread illness and death, similarly so would clearcutting forests. I would expect at least some attempt to ground the claim in something real before leaping to a headline grabbing "computers aren't productive" (except for that bit when the productivity went up faster than previously) conclusion. For starters, do we care about TFP (the article implies we do) and if so has whatever we predict it to provide as a benefit also followed a similar graph? If not, then who cares if computers or anything else makes the graph go up. Secondly, how can we tell if computers have made it shoot upwards, but some other unconnected change has mostly negated that impact. It all feels very shoddy. The first time this came up in economics neatly lines up with when the graph suddenly changes direction upwards: "You can see the computer age everywhere but in the productivity statistics." Solow, 1987
agomez314 5y ago
Interesting read that harkens back to my econ days. I agree with the author based on my experience that digitizing a process often requires the developer to know the system better than the person who operates it - due to the nature of programming. I wonder if AI is really the way to transform software development into a General-Purpose-Technology. Codex is showing the way, in a niche and gimmicky way, but such is the way that many great ideas start. It's really hard to know though if _this is it_ or if it's merely another invention in a long line of inventions that failed to make it. Sometimes it just seems like we are swimming in a sea of code with no apparent gain. Incredible to think that people managed the construction of Pyramids, Cathedrals and awesome constructions with nothing but papyrus, ink, leather straps and good ol' memory. I can't even remember the function arguments for fs.read!
the_laka 5y ago
Low Code and No Code platforms transform the whole premise though. They do make it easy to "show it how to attach a fastener, then walk away". Too bad, as developers, we scorn those platforms instead of improving them to the point we'd be obsolete.
- stdbrouw 5y ago
  But that's exactly what people used to think in the 60s and 70s: instead of requiring a bunch of electrical engineers to build some arcane contraption, now ordinary folks can just write something that almost looks like English and you can automate anything and do calculations in seconds that used to take months! If that didn't pan out even though it seemed so freaking obvious that it would, why will No Code be any different?
  - hvidgaard 5y ago
    I'm inclined to agree. One of the most successful "No Code" programs is Excel. Yet we still, time and time again, see people struggle with basic calculations in it. It's literally elementary school mathematics we're talking about. I think most "No Code" and especially RPA in general will fall into that. The required mindset to think programmatically is not something the majority of people have unfortunately. But "No Code" will enable those that is somewhat technically inclined and able to think sufficiently programmatically.
  - toolslive 5y ago
    Yes! SQL for example, was invented for business people to allow them to pull their own reports iso having to bother programmers to do it for them. We all know what really happened.
    - datavirtue 5y ago
      Millions of business people pulling their own reports? That is a thing.
  - bildung 5y ago
    To add an anecdote: No Code already was the hot new thing in the 90s when I studied CS. You could click together custom interfaces in Delphi and even do basic wiring with clicking alone, IIRC. Devs expected that laypeople click together the solution they want and developers do the remaining wiring. Yet no non-developer could actually use that thing. Nowadays I think the main hurdle is the transformation of a fluffy real world problem into something of an algorithm. Developers do this almost unconsciously, because they practice this all the time, and thus are usually not aware of it. Yet this process of quantification of the real world problem often is the actual problem, not writing it down as code.
neogodless 5y ago
This article makes me think of the woodworker's dilemma. You might start working with cutting, planing, joining and finishing wood because you want to make a chair or end table, and you like the idea of learning to do it yourself, maybe saving some money, or at least getting some extra tools out of the process, and having some pride in your work. But before you know it, you've spent 4 years accumulating tools, but more importantly, getting really good at making jigs, shelves, etc. to organize your tools and make your hobby easier and more enjoyable. In fact, you spend 90% of your time building tools. You do this not because you have to, but because you enjoy it. And because the brain makes it so easy to think of new ways you can use the skills you're building to make more tools in a virtuous cycle. When you actually build a chair or end table, you complete the project, and you do enjoy the fruits of that labor, but there's no real cycle there. It's just an ending. Software developers might fall into a similar trap, being so enthralled with building their own tools, writing libraries, designing and implementing frameworks, creating processes like CI/CD that obviously make the whole software development life cycle better... but of course it's largely an internal cycle that's more interesting than a lot of the end results of software that might actually benefit business (and measurable productivity.)
- LeSaucy 5y ago
  As a software engineer who loves this type of stuff I’m thrilled someone’s willing to pay me exceptionally well to do it. It’s unfathomable to think of a company having 3-4 woodworking teams each with different shops, tools, jigs etc.
  - scooble 5y ago
    > It’s unfathomable to think of a company having 3-4 woodworking teams each with different shops, tools, jigs etc. Not really, depending on the company. Building something complex, like a house, could require at least carpenters, joiners and cabinet makers, which are quite different jobs.
- swiley 5y ago
  Sometimes I feel like the complexity introduced by those tools results in less productivity. When that happens this weird cycle forms where more and more developers are hired to manage the ever changing interfaces used by internal tools which results in more churn.
jasode 5y ago
>Almost every recent A.I. advance has come from one tiny corner of the field, machine learning. Machine learning exposes a set of connected nodes, known as neural nets, to mass amounts of labeled real-world data in an attempt to give those nodes tacit knowledge. The breakthrough example was software that was able to identify cat pictures. >So far, these neural nets have given us some great demos but mostly niche real-world applications. We don't have self-driving cars quite yet! What's the author's threshold for "real-world applications"? - Google's Youtube algorithm for recommendations uses neural nets[1]. So ~2 billion viewers being affected by it doesn't seem like a "niche" application. - Google language translation uses neural net[2] - Apple Siri voice recognition uses neural net It doesn't seem like neural nets are analogous to the joke that "graphene is the wonder material that can do everything except escape the research lab". In contrast, deep learning neural nets have escaped the research lab and are widely used in production systems today. The author's blog post is recently dated August 2021 so it seems like he's not kept up-to-date on this topic since the experimental neural net winning ImageNet in 2012. Yes, that was an artificial contest but things have progressed quickly and there is real-world commercial deployment of NN trained models. [1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45530.pdf https://static.googleusercontent.com/media/research.google.c... [2] https://smerity.com/articles/2016/google_nmt_arch.html https://smerity.com/articles/2016/google_nmt_arch.html
- marcinzm 5y ago
  Not to mention Google search uses neural networks for language parsing [1], social networks use neural networks for ranking feeds, etc. I'd venture to guess the author interacts with the results of neural networks dozens if not hundreds of times per day but simply doesn't realize it. [1] https://searchengineland.com/google-bert-used-on-almost-every-english-query-342193 https://searchengineland.com/google-bert-used-on-almost-ever...
- orolle 5y ago
  I think the author wording is a little bit confusing. I think he want to point out, that narrow AI is used successfully only in (niche) applications with narrow purpose. Niche here is not about total market size.
- 5y ago
tibbetts 5y ago
The phenomenon of Bullshit Jobs also seems highly relevant: https://www.strike.coop/bullshit-jobs/ https://www.strike.coop/bullshit-jobs/ If software causes soft cost savings (reducing the number of required people) the savings may not actually be realized. The internal feudalism of large enterprises protected by monopolistic moats means people resist headcount reductions. And since these large enterprises still employ the majority of people, they are over represented in the statistics.
- barry-cotter 5y ago
  > Alienation Is Not ‘Bullshit’: An Empirical Critique of Graeber’s Theory of BS Jobs > David Graeber’s ‘bullshit jobs theory’ has generated a great deal of academic and public interest. This theory holds that a large and rapidly increasing number of workers are undertaking jobs that they themselves recognise as being useless and of no social value. Despite generating clear testable hypotheses, this theory is not based on robust empirical research. We, therefore, use representative data from the EU to test five of its core hypotheses. Although we find that the perception of doing useless work is strongly associated with poor wellbeing, our findings contradict the main propositions of Graeber’s theory. The proportion of employees describing their jobs as useless is low and declining and bears little relationship to Graeber’s predictions. Marx’s concept of alienation and a ‘Work Relations’ approach provide inspiration for an alternative account that highlights poor management and toxic workplace environments in explaining why workers perceive paid work as useless. https://journals.sagepub.com/doi/10.1177/09500170211015067 https://journals.sagepub.com/doi/10.1177/09500170211015067
  - thescriptkiddie 5y ago
    It should be noted that David Graeber was a marxist and 100% agreed that bullshit jobs are a symptom of alienation of the worker.
mensetmanusman 5y ago
Because productivity doesn’t care about how people value their time. Is it more productive that I can watch 20 hours of high-quality TV every week?
tintt 5y ago
I still think that software in general does show up in productivity trends. Take a look at nineties on the chart in the article, this productivity surge can easily be attributed to spreadsheets, text processor and other software innovations that became ubiquitous in that period of time. It’s also true that niche software is hard to make right, but then again, take a look at Amazon — its crazy efficient logistics is based on the custom software and it seems to work fine for them.
swalsh 5y ago
This is off topic a bit since the article was really talking about something tangently different, and it's probably a really unpopular thing to say, but it's been in my mind for a while. If you look at the worlds population growth rate, you can see that it is beginning to level off. That world of 10 billion that we were all preparing for several years ago is probably not going to come. If the population stabalizes, or even starts shrinking, how important is growth of productivity? Making "Stuff" is obviously important, but in a world with lowering demand, maybe quality and distribution are the metrics we should be concentrating on. I have 2 kids, I am the only one with kids in both mine and my wifes family. My 2 kids are the only grandkids between 3 sets of grandparents (wife's parents got divorced and remarried). They are inudated with LOADS of stuff. So much so that it's a real problem. I tell my parents to stop buying them stuff. They think i'm joking. I'm not. IT'S TOO MUCH STUFF. I wish they would all go in, and just get my kids 1 good high quality thing. They just don't need all this cheap low quality stuff. I bring this up, because thinking this way is a different paradigm. Agile is still very relavent to quality driven development. But scale less so.
- orolle 5y ago
  Its a different discussion, quite off topic. When the boomers go into retirement, our current economic system with GDP growth focus has a big problem. Our system is based on produce more stuff and buy more stuff. But when the boomers retire, they will buy less stuff. Who will buy supply overhang? Nobody. Even worse, who will pay for the debt incurred to buy overly inflated asset prices, like housing and factories, when boomers start downsizing and start selling? Look at Japan. They are 10 years ahead of us. It will be tough and depressing until the economic system adapted and prices normalized. On an opitmistic point, humanity will progress and when an economy cannot sell more stuff then it has to sell better stuff.
  - zie 5y ago
    I don't disagree it could definitely be a problem with the US, but so far, with immigration, it's not been a problem. As long as our immigration #'s stay up, the US should be fine. So far it's been a steady state, but it's unknown if that will continue into the future.
li2uR3ce 5y ago
> While software improves through better tooling and faster hardware In my experience faster hardware leads to worse software. I'm doing the same things I've always done on my "smart" phone but apparently the same sized text messages now need more phone. Good god, ICQ from 2000 had more user facing features than texting apps and that was running on Windows-swap-everything-unconditially. Yeah, much software development time is spent on invisible "features" that aren't relevant to the poor bastards that will have to use it. It makes the case for the more vertical in-house software development. There's much less push back to specialized features which often aren't nearly as specialized as the outside developer thinks it is because of absent understanding of the job fortified by arrogance. But even when the job is well understood... display fucking words on the screen... I mean, come on! Why am I ever waiting 3 seconds after unlock for that?
- svachalek 5y ago
  I've been in software development 28 years. Seems like we're writing the same stuff we always did, except now it's in a language virtual machine running in a container running in a hardware abstraction VM running on actual hardware in the cloud. Thanks to hardware advances, we can move exponentially sharper images, bigger files, more polygons, but the actual complexity of what software is doing is increasing at a slow linear pace. The biggest gains now are in horizontal scaling, we can now handle way more users in parallel than we ever did 10 years ago. But for most products that don't need to handle a billion users, it's at least as hard to produce a shipping product as it was 20 years ago.
Jensson 5y ago
Firstly, GDP is a bad measure of productivity. A pill that replaced all healthcare would reduce GDP by 15%, but I doubt anyone (even economists) would call that a catastrophe. Productivity has stalled mostly because people have already filled their needs, so it makes little sense to buy more. Basically everyone have the clothes they need, the food they need, the car they need, the computer they need, already. Screen entertainment and information is basically free nowadays. So no matter how much you increase productivity these sectors will remain mostly constant. What do people still buy? Housing, but that is mostly a competitive good, people spend as much as they have on housing and it is limited supply so prices just increases to whatever people can afford. Same thing with education, international flights and the free market healthcare with restricted supply you have in USA. Another thing is access to other peoples time. You can buy a person to clean your home or do your lawn or drive you somewhere or renovate your kitchen or provide a massage or other things. There is no way to significantly increase that productivity, it is mostly fixed. So personally I see no need to increase GDP (what he calls productivity) further. Not to mention that many things gets cheaper, a family buying a TV today gets a much better TV for the same amount of money as a family buying a TV 30 years ago. The main thing would be to automate tasks so you no longer need as much access to other peoples time, but that is mostly an unsolved issue for now. Automating information delivery worked great, but it didn't lead to increased GDP rather it lead to those products becoming essentially free to consume effectively making it useless from an economists perspective.
- strgcmc 5y ago
  > Productivity has stalled mostly because people have already filled their needs, so it makes little sense to buy more. Basically everyone have the clothes they need, the food they need, the car they need, the computer they need, already. Screen entertainment and information is basically free nowadays. So no matter how much you increase productivity these sectors will remain mostly constant. I know you are not being literal with your claims, but I just wanted to provide some data, so that others who read it will have some grounded context. Across several categories, there is a stubborn (and maybe surprisingly or not-so-surprisingly consistent) 10-15% of Americans who don't have many of those needs filled that you mentioned: - The official national poverty rate hovers around 10%, but there has always been much controversy about defining what that means and what the threshold should be: https://www.census.gov/library/publications/2020/demo/p60-270.html https://www.census.gov/library/publications/2020/demo/p60-27... - 32 million uninsured under 65 (the default age threshold for Medicare): https://www.cdc.gov/nchs/fastats/health-insurance.htm https://www.cdc.gov/nchs/fastats/health-insurance.htm - 42+ million Americans face food insecurity: https://www.feedingamerica.org/hunger-in-america https://www.feedingamerica.org/hunger-in-america - It should come as no surprise that many poor people cannot afford cars and hence rely on public transit more, but that public transit systems are woefully underfunded (or funded but misappropriated/delayed/etc.) in America. Because economic disparity is interwoven with racial inequality, this is not just an economic problem; suffice it to say that, no, not everyone in America has the car they need: https://www.urban.org/features/unequal-commute https://www.urban.org/features/unequal-commute - The FCC reports large gains in the past 5 years for broadband and mobile broadband access, but this is baselined against a paltry and outdated 25/3 and 10/3 Mbps standard definition for "broadband speed" (trying living in a 25/3 Mbps household while remote-working, video conferencing, streaming Netflix, etc.): https://docs.fcc.gov/public/attachments/FCC-21-18A1.pdf https://docs.fcc.gov/public/attachments/FCC-21-18A1.pdf
dalbasal 5y ago
Extremely worthwhile question, long overdue. Productivity, especially in relevant areas like administration, stagnated despite computers hitting every desk. I read the Cowen book (Complacent Class) at the same time I was reader Graeber's "Bullshit Jobs." Heterodox writers from both sides of the spectrum. Same observation. On the face of it, it doesn't make sense. How could, for example, a local college's administration not have become more efficient because of computers? A factory's productivity, which has legible inputs and outputs is really different to something which doesn't. Software is management technology, perhaps, but only in cases that management technology is pretty efficient already. Modern warehouses, ports and stuff are more productive because of software. But, they we already pretty efficient. They already had pretty well formalized, legible processes. That said, software is also a tool. Say your job is to receive applications, payments or such. You process them. File. Respond. Software is undeniably a good tool for such things. We can't abstract that away by looking at the top level trends. It is a productivity tool for administrative tasks. Top line trends don't suggest a productivity gain, but I'm not willing to conclude that software is not an administrative tool. On the face of it, banks, universities, government departments, the legal sector, accounting, perhaps the whole finance sector are bigger today, not smaller. They have computers now, which are productivity tools. WTF is going on? Do we have more justice, better records? What is "productivity" anyway, outside of legible productivity like a factory's?
- handrous 5y ago
  I've had a suspicion for some years that productivity increases from the addition of computers and software to an operation is very uneven, and that apparent great progress overall is because it's a 1000x improvement in select areas while being a 0.5-1.05x change in most areas, with somewhat negative change perhaps even being the norm.
  - dalbasal 5y ago
    Perhaps. Why though? Why aren't HR, accounting or other such tasks more productive with computers than without?
    - 5y ago
MetaWhirledPeas 5y ago
Before we can have a meaningful discussion about this we really need to understand what this Total Factor Productivity graph even means. "In the U.S." implies that it's using GDP or some other nationwide measurement. So we're talking about software's impact on productivity in a system (a large nation) with many other forces at play. That makes the entire discussion a bit narcissistic don't you think?
sammyloso 5y ago
It is because we are no longer improving our society's "hardware" (read: infrastructure). No matter how quickly i can design a BIM model using state of the art software; i am still constrained by the fact that it takes me an hour to drive to work, an hour to my client's house, and then an hour back to the office, and then an hour home.
pjmorris 5y ago
"VisiCalc took 20 hours of work per week for some people and turned it out in 15 minutes and let them become much more creative." - Dan Bricklin It's interesting to me that VisiCalc (1979) and its successors (Lotus 1-2-3 and Excel) undoubtedly made some key business jobs vastly more productive and yet software spreadsheets don't really make a dent in the productivity numbers. I'd argue that software spreadsheets are a 'management technology' as the article defines them, but that they are a counter to the article's claim that management technologies spread slowly. They've been widely adopted by businesses of all scales, starting from the introduction of VisiCalc. Because of this, I wonder whether we are measuring productivity properly [0] http://theinventors.org/library/weekly/aa010199.htm http://theinventors.org/library/weekly/aa010199.htm
qwerty456127 5y ago
Perhaps because mainstream software of today doesn't add much to what we had in 199x. There already were the same Word, Excel and e-mail those days - . (and I doubt live chats we have today can add much to productivity, well-thought and well-organized emails are better). It can be possible to boost productivity with something like Roam Research (especially used in collaborative mode) but it would require a lot of enhancements to it and a lot more work on teaching the people to use it the right way. Even people with skills to use Word the right way (i.e. use styles instead of ad-hoc manual font adjustments and extra CRs) or to use non-basic features of Excel are rare. Teaching (or even getting them interested) the masses something entirely new, requiring a new way of thinking and totally new workflow would probably require enormous effort.
- datavirtue 5y ago
  We had live chat in the 1990s.
- crazygringo 5y ago
  I disagree completely. I'm insanely more productive as part of a team than I was in 199x. Not even talking about things for software development specifically, but for general-purpose word-processing and spreadsheeting and scheduling, you've got: - Live collaborative cloud editing over mobile. The back-and-forth that previously might take a week can now be done in half an hour while you're in the back of an Uber in a different country - Googling how to accomplish spreadsheet tasks. Stuff that you'd just give up and not do before, or would take you days to figure out on your own, there are tons of blog posts and YouTube videos letting you get it done in half an hour - Tons of scheduling and information-gathering improvements. Looking at people's Google Calendars live to find a meeting that everyone can attend, sending a Google Form to collect lunch preferences rather than contacting people individually, and so on The productivity of modern administrative office tasks has skyrocketed with collaborative, mobile, cloud-based tools. Getting stuff done as a team with static Word and Excel files that were stuck on a physical computer at a physical office while you tried to decipher printed software manuals was slow.
  - asdff 5y ago
drdec 5y ago
If we counted goofing off as part of productivity, we might start to see the gains we expected. Put another way, while computers have made us more productive, the internet has made it much easier to not do our jobs while at work. I don't think it is a coincidence that the graph stops being as steep around 2005.
dexen 5y ago
I find the opening graph to undermine the central thesis a bit: the dot-com bubble burst in 2002, but is barely reflected on the graph; the growth levels off in 2005 or 2006. If the change in productivity was largely software-moderated, I would expect a lager change around the dot-com burst. Meanwhile the large change seems to be the 2007 Subprime mortgage crisis - and presumably the follow-up change in interest rate and investment patterns.
nitwit005 5y ago
You can't eliminate all the paperwork and archival jobs twice. Some of the improvements tied to communication require cultural changes that can be slow. Telemedicine has been possible for a long time, but the shift only picked up steam due to Covid.
amelius 5y ago
If productivity stalled it's because we now have the equivalent of a TV set/entertainment system on our desks.
nonameiguess 5y ago
Something I don't see addressed here is how much of software is an arms race. I think this reality is hidden a bit from people who have only ever worked on commercial software that exists for the purpose of creating economic value. A lot of software doesn't have that purpose and exists mainly for defensive reasons. I have spent most of my career working on fielding new software systems for the intelligence community and the DoD. We can't say we haven't seen productivity gains in the form of many processes being automated to the point we can scale them much larger and process much more data. But this isn't economic productivity. 60 years ago, satellite imagery involved dropping film from the satellite on a little parachute and intercepting it before it hit the ground, developing the film, and deploying any improvements in imaging capabilities by launching a new satellite. Now we can do almost all of that with radio and software and we have virtually the entire globe covered, a near non-stop stream of imagery constantly being turned into possibly useful and actionable intelligence depending on what the interest is in knowing what is happening in that region. But in terms of what we're doing, much of it is economically purely a sink. We're monitoring foreign ports, known locations of military units, missile silos, to maintain the strategic advantage of not being caught with our pants down if anyone out there ever decided to launch a large-scale conventional attack. A lot of people would probably argue what we're doing is pointless, fighting yesterday's wars while losing today's. Maybe. I'm not really trying to make an argument either way for whether this activity is useful or not. But it's not increasing American economic output, and it's not intended to. But it is an incredibly expensive and enormous scale application of deployed software technologies. It's effectively a new category of cost for the world's major military powers. They now need to spend on maintaining an enormous development pipeline and operational environment for software capabilities that bring no economic gain, but just keep them from being overtaken by their enemies. You do see some patterns like this in commercial software, especially in the real of information security. We may or may not be able to easily deploy huge force multipliers to make our workforces more productive, but then we find they have vulnerabilities in them and we've exposed ourselves to a new kind of criminal taking advantage of that and extracting some of that value. So we devote more and more resources to securing these systems, often making them less efficient and more difficult to use in the process. We have to do it, because the added security is at least some of the time ultimately worth it due to the enormous cost of a breach. But it's purely defensive spending. You're not making your system any more effective at producing whatever it is your company produces that creates economic value. Often, you're making it less effective at doing that.
Spooky23 5y ago
Short term focus within companies drives cost control and growth, not productivity gains. The growth is the gremlin. Companies achieve monopoly/cartel status and they optimize for rent collection, which is the opposite of productive.
wantsanagent 5y ago
I take issue with the basic assumption of the article. Sustained productivity growth is hard and would not have continued without the software revolution. Let's take a look at agriculture. https://www.ers.usda.gov/amber-waves/2018/march/agricultural-productivity-growth-in-the-united-states-1948-2015/ https://www.ers.usda.gov/amber-waves/2018/march/agricultural... Mechanization and the widespread adoption and improvement of mechanized farming has lead to staggering productivity / farmer growth over the last 70 years. But there is only so much you can do with "dumb" machines. Today growth is being driven by computerized information gathering, planning, monitoring, and precision planting / soil maintenance. To maintain a growth curve takes constant innovation. Just because the growth doesn't significantly alter its slope does not mean that there is a missing improvement bump. If you decomposed slopes like these you would see they are compound sigmoids where growth is driven by one technology and then another, or an adoption of a new process, etc. So IMO if "software doesn't show up in productivity" you're not looking hard enough.
- jldugger 5y ago
  And TFP is a real monster of a formula[1]. It's not just GDP / hours. There's like 20 variables going into it's calculation, including things like 'labor quality' and 'capital's share of income' (alpha). I'm not a smart man, but I think this suggests that a society that lays off factory workers and retrains them as software engineers will not register on this metric. And looking at alpha, there's a pretty clear phase change at 2000 -- it's hovering at 31-33 for 50 years, then marches up from 0.31 to 0.38. Sounds to me like you could tell a story that labor is more productive, but seeing less of the gains than before. edit: just to belabor the point, here's a random chart I googled for US productivity that _doesn't_ feature the same trendline: https://tradingeconomics.com/united-states/productivity https://tradingeconomics.com/united-states/productivity. If anything it looks like productivity has accellerated during the past 20 years. [1]: https://www.frbsf.org/economic-research/indicators-data/total-factor-productivity-tfp/ https://www.frbsf.org/economic-research/indicators-data/tota...
shakezula 5y ago
If software is process, then what does that say about the labor value of software? Related: I think that software engineering is going to be the new blue collar warehouse or factory job that a good chunk of boomers enjoyed.
AnimalMuppet 5y ago
Because GDP doesn't measure right. It measures dollars, not value. Take Google search, for instance. I can look up (approximately) all the information in the world, for free. That shows up in the GDP as $0, because it's free. But what is Google search actually worth? Would your business, say, actually pay for it as a tool? Probably, at least for those of us who need Stack Overflow answers. Real value is being produced, but it isn't being measured because it is being given away. (Yeah, I know, ads. Search itself is still being given away. So is Linux and gcc and...) And does Google search help productivity? Yes. Does Linux? Yes. Does gcc? Yes. The ability to get all these things for free greatly expands the things you can do. Software is producing value. But because so much is being given away, the value isn't showing up in the dollar-based metrics.
kazinator 5y ago
Software is mostly a management technology. But there is all sorts of automation: CNC machines, industrial robots making stuff. I think that a graph of "total factor productivity in the USA" is misleading without looking at factors like, say, how much manufacturing has disappeared from the USA and gone overseas in that period!" You have to look at how much you're producing with how many people; and that cannot be some per capita based on the population, but the actual head counts in those industries that are covered by that graph: what is the productivity with how many people?
beefman 5y ago
My family watched Miracle on 34th Street a couple years ago. Aside from being generally impressed with how well it's aged, I was particularly impressed with the office technology on display (pneumatic tubes etc). In Victorian London, mail could be posted up to 12 times per day.[1] That's about as often as e-mail can be turned around. Bronze Age merchants exchanged clay tablets with remarkable throughput.[2] On the consumer side... I live in Silicon Valley. My grandparents had better access to services than I do — fresh milk delivery, an MD that came to their bedside, and an electric trolley — in the 1930s in a town of 12k ppl. My grandfather was a driver for a laundry service, my grandmother taught piano. [3] But maybe the most fundamental issue here is that productivity is 'measured' by dividing GDP by hours worked. But work seems better characterized as a mechanism that distributes, rather than creates, GDP.[4] [1] https://www.mentalfloss.com/article/24089/victorian-mail-delivery-12-times-each-day https://www.mentalfloss.com/article/24089/victorian-mail-del... [2] https://www.youtube.com/watch?v=bRcu-ysocX4 https://www.youtube.com/watch?v=bRcu-ysocX4 [3] https://twitter.com/clumma/status/1297571626901331968 https://twitter.com/clumma/status/1297571626901331968 [4] https://twitter.com/clumma/status/835581829654536192 https://twitter.com/clumma/status/835581829654536192
- smoldesu 5y ago
  A lot of those jobs ultimately became redundant for people. Milk delivery makes no sense when you can choose your bottle down at the grocery store next time you're there. Bedside care became impractical as medical technology advanced, and electric trolleys are pretty cumbersome (especially alongside city streets). Eventually, we realized that we could cut out the milkman: we laid off a lot of people in the process, but I'm sure the ice deliverymen are thankful that they no longer need to haul 25 pound bricks up New York staircases anymore.
  - beefman 5y ago
    Ice delivery is the only valid example of a reduntdant job here. My grandparents bought milk at the grocery store, too. And I get milk delivered with Instacart. Medical technology did not make bedside care impractical. Quite the opposite. More can now be done from the bedside. Not that a typical doctor's visit involves any meaningful use of technology. Of course there are many other reasons why bedside care is superior.
3pt14159 5y ago
I'm not sure that we're measuring productivity correctly these days. Take AI, for example. Going from barely being able to play chess, to winning at Starcraft and Go and basically any game Google thinks is worth cracking is a huge, huge leap in technological capacity. What was the impact of the people that worked on these and related technologies on "productivity"? You can count apples. You can't really quantify DynamoDB or Rust on the economy. Now, median worker productivity growth seems like it's drastically slowed and I think we have real problems in the economy, but as everything started merging with tech it gets harder to see the full picture.
- mytailorisrich 5y ago
  Oh we are measuring productivity correctly. How much more productive is an software engineer that uses Rust compared to one that uses, say, C/C++? That is to say, do I need fewer software engineers to deliver the same product if they use Rust rather than another language? If the answer is "not really", then Rust does not increase productivity (I don't know the answer, btw). In general, I don't think that software as a tool made much difference to people's productivity in the last 20 years or so. The boost enabled by internet connectivity was probably over by ~2005. Since then not much has happened. Uber, Deliveroo, etc. are great for consumers but they don't increase the productivity of drivers: Drivers cannot drive faster or service more customers per hour, really, these are bounded by physical constraints.
  - Jweb_Guru 5y ago
    > How much more productive is an software engineer that uses Rust compared to one that uses, say, C/C++? That is to say, do I need fewer software engineers to deliver the same product if they use Rust rather than another language? Anyone who's used both in production for any length of time can tell you that you will not deliver the same product in C++ unless you're willing to put in JPL levels of investment. So yes, it's far more productive for "the same product." The question is what you mean by "the same product." Do you consider a project implementing a feature set with loads of memory errors to be functionally the same thing as one without? If you do, then Rust probably won't be that much more productive than C++ (IME still more productive, but within the realm of subjectivity). If you do not, then it's not even close. This is the calculation companies like Microsoft and Google have been doing when they invested in Rust for new OS development.
lifeisstillgood 5y ago
I am writing a book covering a lot of this ground (shout out to Roald Coase) - but I have a different (ish) conclusion. It's not going to be a few "software-friendly" companies like FAANG that eventually lead the charge and we see productivity - it's waaay longer term than that. My take is software is a form of literacy - and it will only be when managers code daily that we will see enough of the control layers (model, monitor mentor) being actually software that software will show up in productivity stats If you like an analogy - steam engines used to power factories but there was one central engine and you spread out the power to other areas via bands / chains. Electricity came along but mostly replaced the central engine - it was not till people experimented with having power sent to many engines did the modern (Fordist) factory layout become feasible In short - everyone needs to learn to code or - if an SRE is what you get when you ask a coder to design a software development process, a programmable company is what you get when you ask a coder to design a company
- jeffreyrogers 5y ago
  Coding is hard though. I remember helping a smart friend of mine with his homework in college for an intro CS course and even basic fizz-buzz type stuff was very hard for him. This is a guy who has had a highly successful career post-college and did well in his (non-technical) major. I'm skeptical that more than a small fraction of the population will ever learn to code at more than a superficial level.
  - lifeisstillgood 5y ago
    "They said you couldn't teach women to read" - Santa Claus' Grandfather in Arthur Christmas (Best Xmas kids film, not best source of political quotes but I am still asleep)
    - jeffreyrogers 5y ago
      I don't think anyone has seriously said that for a long, long time, if ever. Literacy is obviously pretty easy, much easier than algebra, for example, because in societies with universal schooling almost everyone learns how to do it.
michael1999 5y ago
Solid thinking. But I think the Jevons paradox deserves a mention. How much of our expanded capacity is spent on intangibles like extra complexity that doesn't show up in GDP?
deleted 5y ago
[deleted]
drivers99 5y ago
Cal Newport points out the lack of productivity increases in his book about email (which is actually about email and instant messaging), and how it's the default workflow tool and how badly it works for that.
fungiblecog 5y ago
In my experience (mostly as a consultant to the public sector) all the (potential) productivity gains brought by software are eaten up by two things: a) Layers of managers whose job appears to be to hold meetings to talk endlessly without ever making a decision or assigning an action, and b) a complete failure of the same managers to understand how to use the software at their disposal to its fullest extend (eg no enforcement of data quality, no idea how to report useful metrics). Both of these are the result of the people in positions of authority having no technical background or education and recruiting similar people. If they do think they need someone to analyse data they think that's a low-level position and hire based on a low-level salary with predictable results.

218 points by Wildgoose 5y ago

That is a graph of US TFP. I expect that once Asia is included in the mix the change will be a bit more pronounced. Most of the hardware related change in the IT revolution is happening there. The S&P 500 rank 1 company (Apple) would look to an alien like an Asian company since all the actual manufacturing happens there. Also, while I don't think it is necessarily the major driving factor, the US has a capital misallocation problem. People keep sinking fortunes into companies with bad profit margins.

ZeroGravitas 5y ago
Also, Total Factor Productivity will include the effects of everything a society does that can be measured in monetary terms. Dumping poison in rivers would probably show up as an "efficiency gain" at least until it causes ecosystem collapse and widespread illness and death, similarly so would clearcutting forests. I would expect at least some attempt to ground the claim in something real before leaping to a headline grabbing "computers aren't productive" (except for that bit when the productivity went up faster than previously) conclusion. For starters, do we care about TFP (the article implies we do) and if so has whatever we predict it to provide as a benefit also followed a similar graph? If not, then who cares if computers or anything else makes the graph go up. Secondly, how can we tell if computers have made it shoot upwards, but some other unconnected change has mostly negated that impact. It all feels very shoddy. The first time this came up in economics neatly lines up with when the graph suddenly changes direction upwards: "You can see the computer age everywhere but in the productivity statistics." Solow, 1987

agomez314 5y ago

Interesting read that harkens back to my econ days. I agree with the author based on my experience that digitizing a process often requires the developer to know the system better than the person who operates it - due to the nature of programming. I wonder if AI is really the way to transform software development into a General-Purpose-Technology. Codex is showing the way, in a niche and gimmicky way, but such is the way that many great ideas start. It's really hard to know though if _this is it_ or if it's merely another invention in a long line of inventions that failed to make it. Sometimes it just seems like we are swimming in a sea of code with no apparent gain. Incredible to think that people managed the construction of Pyramids, Cathedrals and awesome constructions with nothing but papyrus, ink, leather straps and good ol' memory. I can't even remember the function arguments for fs.read!

the_laka 5y ago

Low Code and No Code platforms transform the whole premise though. They do make it easy to "show it how to attach a fastener, then walk away". Too bad, as developers, we scorn those platforms instead of improving them to the point we'd be obsolete.

stdbrouw 5y ago
But that's exactly what people used to think in the 60s and 70s: instead of requiring a bunch of electrical engineers to build some arcane contraption, now ordinary folks can just write something that almost looks like English and you can automate anything and do calculations in seconds that used to take months! If that didn't pan out even though it seemed so freaking obvious that it would, why will No Code be any different?
- hvidgaard 5y ago
  I'm inclined to agree. One of the most successful "No Code" programs is Excel. Yet we still, time and time again, see people struggle with basic calculations in it. It's literally elementary school mathematics we're talking about. I think most "No Code" and especially RPA in general will fall into that. The required mindset to think programmatically is not something the majority of people have unfortunately. But "No Code" will enable those that is somewhat technically inclined and able to think sufficiently programmatically.
- toolslive 5y ago
  Yes! SQL for example, was invented for business people to allow them to pull their own reports iso having to bother programmers to do it for them. We all know what really happened.
  - datavirtue 5y ago
    Millions of business people pulling their own reports? That is a thing.
- bildung 5y ago
  To add an anecdote: No Code already was the hot new thing in the 90s when I studied CS. You could click together custom interfaces in Delphi and even do basic wiring with clicking alone, IIRC. Devs expected that laypeople click together the solution they want and developers do the remaining wiring. Yet no non-developer could actually use that thing. Nowadays I think the main hurdle is the transformation of a fluffy real world problem into something of an algorithm. Developers do this almost unconsciously, because they practice this all the time, and thus are usually not aware of it. Yet this process of quantification of the real world problem often is the actual problem, not writing it down as code.

neogodless 5y ago

This article makes me think of the woodworker's dilemma. You might start working with cutting, planing, joining and finishing wood because you want to make a chair or end table, and you like the idea of learning to do it yourself, maybe saving some money, or at least getting some extra tools out of the process, and having some pride in your work. But before you know it, you've spent 4 years accumulating tools, but more importantly, getting really good at making jigs, shelves, etc. to organize your tools and make your hobby easier and more enjoyable. In fact, you spend 90% of your time building tools. You do this not because you have to, but because you enjoy it. And because the brain makes it so easy to think of new ways you can use the skills you're building to make more tools in a virtuous cycle. When you actually build a chair or end table, you complete the project, and you do enjoy the fruits of that labor, but there's no real cycle there. It's just an ending. Software developers might fall into a similar trap, being so enthralled with building their own tools, writing libraries, designing and implementing frameworks, creating processes like CI/CD that obviously make the whole software development life cycle better... but of course it's largely an internal cycle that's more interesting than a lot of the end results of software that might actually benefit business (and measurable productivity.)

LeSaucy 5y ago
As a software engineer who loves this type of stuff I’m thrilled someone’s willing to pay me exceptionally well to do it. It’s unfathomable to think of a company having 3-4 woodworking teams each with different shops, tools, jigs etc.
- scooble 5y ago
  > It’s unfathomable to think of a company having 3-4 woodworking teams each with different shops, tools, jigs etc. Not really, depending on the company. Building something complex, like a house, could require at least carpenters, joiners and cabinet makers, which are quite different jobs.
swiley 5y ago
Sometimes I feel like the complexity introduced by those tools results in less productivity. When that happens this weird cycle forms where more and more developers are hired to manage the ever changing interfaces used by internal tools which results in more churn.

jasode 5y ago

>Almost every recent A.I. advance has come from one tiny corner of the field, machine learning. Machine learning exposes a set of connected nodes, known as neural nets, to mass amounts of labeled real-world data in an attempt to give those nodes tacit knowledge. The breakthrough example was software that was able to identify cat pictures. >So far, these neural nets have given us some great demos but mostly niche real-world applications. We don't have self-driving cars quite yet! What's the author's threshold for "real-world applications"? - Google's Youtube algorithm for recommendations uses neural nets[1]. So ~2 billion viewers being affected by it doesn't seem like a "niche" application. - Google language translation uses neural net[2] - Apple Siri voice recognition uses neural net It doesn't seem like neural nets are analogous to the joke that "graphene is the wonder material that can do everything except escape the research lab". In contrast, deep learning neural nets have escaped the research lab and are widely used in production systems today. The author's blog post is recently dated August 2021 so it seems like he's not kept up-to-date on this topic since the experimental neural net winning ImageNet in 2012. Yes, that was an artificial contest but things have progressed quickly and there is real-world commercial deployment of NN trained models. [1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45530.pdf https://static.googleusercontent.com/media/research.google.c... [2] https://smerity.com/articles/2016/google_nmt_arch.html https://smerity.com/articles/2016/google_nmt_arch.html

marcinzm 5y ago
Not to mention Google search uses neural networks for language parsing [1], social networks use neural networks for ranking feeds, etc. I'd venture to guess the author interacts with the results of neural networks dozens if not hundreds of times per day but simply doesn't realize it. [1] https://searchengineland.com/google-bert-used-on-almost-every-english-query-342193 https://searchengineland.com/google-bert-used-on-almost-ever...
orolle 5y ago
I think the author wording is a little bit confusing. I think he want to point out, that narrow AI is used successfully only in (niche) applications with narrow purpose. Niche here is not about total market size.
5y ago

tibbetts 5y ago

The phenomenon of Bullshit Jobs also seems highly relevant: https://www.strike.coop/bullshit-jobs/ https://www.strike.coop/bullshit-jobs/ If software causes soft cost savings (reducing the number of required people) the savings may not actually be realized. The internal feudalism of large enterprises protected by monopolistic moats means people resist headcount reductions. And since these large enterprises still employ the majority of people, they are over represented in the statistics.

barry-cotter 5y ago
> Alienation Is Not ‘Bullshit’: An Empirical Critique of Graeber’s Theory of BS Jobs > David Graeber’s ‘bullshit jobs theory’ has generated a great deal of academic and public interest. This theory holds that a large and rapidly increasing number of workers are undertaking jobs that they themselves recognise as being useless and of no social value. Despite generating clear testable hypotheses, this theory is not based on robust empirical research. We, therefore, use representative data from the EU to test five of its core hypotheses. Although we find that the perception of doing useless work is strongly associated with poor wellbeing, our findings contradict the main propositions of Graeber’s theory. The proportion of employees describing their jobs as useless is low and declining and bears little relationship to Graeber’s predictions. Marx’s concept of alienation and a ‘Work Relations’ approach provide inspiration for an alternative account that highlights poor management and toxic workplace environments in explaining why workers perceive paid work as useless. https://journals.sagepub.com/doi/10.1177/09500170211015067 https://journals.sagepub.com/doi/10.1177/09500170211015067
- thescriptkiddie 5y ago
  It should be noted that David Graeber was a marxist and 100% agreed that bullshit jobs are a symptom of alienation of the worker.

mensetmanusman 5y ago

Because productivity doesn’t care about how people value their time. Is it more productive that I can watch 20 hours of high-quality TV every week?

tintt 5y ago

I still think that software in general does show up in productivity trends. Take a look at nineties on the chart in the article, this productivity surge can easily be attributed to spreadsheets, text processor and other software innovations that became ubiquitous in that period of time. It’s also true that niche software is hard to make right, but then again, take a look at Amazon — its crazy efficient logistics is based on the custom software and it seems to work fine for them.

swalsh 5y ago

This is off topic a bit since the article was really talking about something tangently different, and it's probably a really unpopular thing to say, but it's been in my mind for a while. If you look at the worlds population growth rate, you can see that it is beginning to level off. That world of 10 billion that we were all preparing for several years ago is probably not going to come. If the population stabalizes, or even starts shrinking, how important is growth of productivity? Making "Stuff" is obviously important, but in a world with lowering demand, maybe quality and distribution are the metrics we should be concentrating on. I have 2 kids, I am the only one with kids in both mine and my wifes family. My 2 kids are the only grandkids between 3 sets of grandparents (wife's parents got divorced and remarried). They are inudated with LOADS of stuff. So much so that it's a real problem. I tell my parents to stop buying them stuff. They think i'm joking. I'm not. IT'S TOO MUCH STUFF. I wish they would all go in, and just get my kids 1 good high quality thing. They just don't need all this cheap low quality stuff. I bring this up, because thinking this way is a different paradigm. Agile is still very relavent to quality driven development. But scale less so.

orolle 5y ago
Its a different discussion, quite off topic. When the boomers go into retirement, our current economic system with GDP growth focus has a big problem. Our system is based on produce more stuff and buy more stuff. But when the boomers retire, they will buy less stuff. Who will buy supply overhang? Nobody. Even worse, who will pay for the debt incurred to buy overly inflated asset prices, like housing and factories, when boomers start downsizing and start selling? Look at Japan. They are 10 years ahead of us. It will be tough and depressing until the economic system adapted and prices normalized. On an opitmistic point, humanity will progress and when an economy cannot sell more stuff then it has to sell better stuff.
- zie 5y ago
  I don't disagree it could definitely be a problem with the US, but so far, with immigration, it's not been a problem. As long as our immigration #'s stay up, the US should be fine. So far it's been a steady state, but it's unknown if that will continue into the future.

li2uR3ce 5y ago

> While software improves through better tooling and faster hardware In my experience faster hardware leads to worse software. I'm doing the same things I've always done on my "smart" phone but apparently the same sized text messages now need more phone. Good god, ICQ from 2000 had more user facing features than texting apps and that was running on Windows-swap-everything-unconditially. Yeah, much software development time is spent on invisible "features" that aren't relevant to the poor bastards that will have to use it. It makes the case for the more vertical in-house software development. There's much less push back to specialized features which often aren't nearly as specialized as the outside developer thinks it is because of absent understanding of the job fortified by arrogance. But even when the job is well understood... display fucking words on the screen... I mean, come on! Why am I ever waiting 3 seconds after unlock for that?

svachalek 5y ago
I've been in software development 28 years. Seems like we're writing the same stuff we always did, except now it's in a language virtual machine running in a container running in a hardware abstraction VM running on actual hardware in the cloud. Thanks to hardware advances, we can move exponentially sharper images, bigger files, more polygons, but the actual complexity of what software is doing is increasing at a slow linear pace. The biggest gains now are in horizontal scaling, we can now handle way more users in parallel than we ever did 10 years ago. But for most products that don't need to handle a billion users, it's at least as hard to produce a shipping product as it was 20 years ago.

Jensson 5y ago

Firstly, GDP is a bad measure of productivity. A pill that replaced all healthcare would reduce GDP by 15%, but I doubt anyone (even economists) would call that a catastrophe. Productivity has stalled mostly because people have already filled their needs, so it makes little sense to buy more. Basically everyone have the clothes they need, the food they need, the car they need, the computer they need, already. Screen entertainment and information is basically free nowadays. So no matter how much you increase productivity these sectors will remain mostly constant. What do people still buy? Housing, but that is mostly a competitive good, people spend as much as they have on housing and it is limited supply so prices just increases to whatever people can afford. Same thing with education, international flights and the free market healthcare with restricted supply you have in USA. Another thing is access to other peoples time. You can buy a person to clean your home or do your lawn or drive you somewhere or renovate your kitchen or provide a massage or other things. There is no way to significantly increase that productivity, it is mostly fixed. So personally I see no need to increase GDP (what he calls productivity) further. Not to mention that many things gets cheaper, a family buying a TV today gets a much better TV for the same amount of money as a family buying a TV 30 years ago. The main thing would be to automate tasks so you no longer need as much access to other peoples time, but that is mostly an unsolved issue for now. Automating information delivery worked great, but it didn't lead to increased GDP rather it lead to those products becoming essentially free to consume effectively making it useless from an economists perspective.

strgcmc 5y ago
> Productivity has stalled mostly because people have already filled their needs, so it makes little sense to buy more. Basically everyone have the clothes they need, the food they need, the car they need, the computer they need, already. Screen entertainment and information is basically free nowadays. So no matter how much you increase productivity these sectors will remain mostly constant. I know you are not being literal with your claims, but I just wanted to provide some data, so that others who read it will have some grounded context. Across several categories, there is a stubborn (and maybe surprisingly or not-so-surprisingly consistent) 10-15% of Americans who don't have many of those needs filled that you mentioned: - The official national poverty rate hovers around 10%, but there has always been much controversy about defining what that means and what the threshold should be: https://www.census.gov/library/publications/2020/demo/p60-270.html https://www.census.gov/library/publications/2020/demo/p60-27... - 32 million uninsured under 65 (the default age threshold for Medicare): https://www.cdc.gov/nchs/fastats/health-insurance.htm https://www.cdc.gov/nchs/fastats/health-insurance.htm - 42+ million Americans face food insecurity: https://www.feedingamerica.org/hunger-in-america https://www.feedingamerica.org/hunger-in-america - It should come as no surprise that many poor people cannot afford cars and hence rely on public transit more, but that public transit systems are woefully underfunded (or funded but misappropriated/delayed/etc.) in America. Because economic disparity is interwoven with racial inequality, this is not just an economic problem; suffice it to say that, no, not everyone in America has the car they need: https://www.urban.org/features/unequal-commute https://www.urban.org/features/unequal-commute - The FCC reports large gains in the past 5 years for broadband and mobile broadband access, but this is baselined against a paltry and outdated 25/3 and 10/3 Mbps standard definition for "broadband speed" (trying living in a 25/3 Mbps household while remote-working, video conferencing, streaming Netflix, etc.): https://docs.fcc.gov/public/attachments/FCC-21-18A1.pdf https://docs.fcc.gov/public/attachments/FCC-21-18A1.pdf

dalbasal 5y ago

Extremely worthwhile question, long overdue. Productivity, especially in relevant areas like administration, stagnated despite computers hitting every desk. I read the Cowen book (Complacent Class) at the same time I was reader Graeber's "Bullshit Jobs." Heterodox writers from both sides of the spectrum. Same observation. On the face of it, it doesn't make sense. How could, for example, a local college's administration not have become more efficient because of computers? A factory's productivity, which has legible inputs and outputs is really different to something which doesn't. Software is management technology, perhaps, but only in cases that management technology is pretty efficient already. Modern warehouses, ports and stuff are more productive because of software. But, they we already pretty efficient. They already had pretty well formalized, legible processes. That said, software is also a tool. Say your job is to receive applications, payments or such. You process them. File. Respond. Software is undeniably a good tool for such things. We can't abstract that away by looking at the top level trends. It is a productivity tool for administrative tasks. Top line trends don't suggest a productivity gain, but I'm not willing to conclude that software is not an administrative tool. On the face of it, banks, universities, government departments, the legal sector, accounting, perhaps the whole finance sector are bigger today, not smaller. They have computers now, which are productivity tools. WTF is going on? Do we have more justice, better records? What is "productivity" anyway, outside of legible productivity like a factory's?

handrous 5y ago
I've had a suspicion for some years that productivity increases from the addition of computers and software to an operation is very uneven, and that apparent great progress overall is because it's a 1000x improvement in select areas while being a 0.5-1.05x change in most areas, with somewhat negative change perhaps even being the norm.
- dalbasal 5y ago
  Perhaps. Why though? Why aren't HR, accounting or other such tasks more productive with computers than without?
  - 5y ago

MetaWhirledPeas 5y ago

Before we can have a meaningful discussion about this we really need to understand what this Total Factor Productivity graph even means. "In the U.S." implies that it's using GDP or some other nationwide measurement. So we're talking about software's impact on productivity in a system (a large nation) with many other forces at play. That makes the entire discussion a bit narcissistic don't you think?

sammyloso 5y ago

It is because we are no longer improving our society's "hardware" (read: infrastructure). No matter how quickly i can design a BIM model using state of the art software; i am still constrained by the fact that it takes me an hour to drive to work, an hour to my client's house, and then an hour back to the office, and then an hour home.

pjmorris 5y ago

"VisiCalc took 20 hours of work per week for some people and turned it out in 15 minutes and let them become much more creative." - Dan Bricklin It's interesting to me that VisiCalc (1979) and its successors (Lotus 1-2-3 and Excel) undoubtedly made some key business jobs vastly more productive and yet software spreadsheets don't really make a dent in the productivity numbers. I'd argue that software spreadsheets are a 'management technology' as the article defines them, but that they are a counter to the article's claim that management technologies spread slowly. They've been widely adopted by businesses of all scales, starting from the introduction of VisiCalc. Because of this, I wonder whether we are measuring productivity properly [0] http://theinventors.org/library/weekly/aa010199.htm http://theinventors.org/library/weekly/aa010199.htm

qwerty456127 5y ago

Perhaps because mainstream software of today doesn't add much to what we had in 199x. There already were the same Word, Excel and e-mail those days - . (and I doubt live chats we have today can add much to productivity, well-thought and well-organized emails are better). It can be possible to boost productivity with something like Roam Research (especially used in collaborative mode) but it would require a lot of enhancements to it and a lot more work on teaching the people to use it the right way. Even people with skills to use Word the right way (i.e. use styles instead of ad-hoc manual font adjustments and extra CRs) or to use non-basic features of Excel are rare. Teaching (or even getting them interested) the masses something entirely new, requiring a new way of thinking and totally new workflow would probably require enormous effort.

datavirtue 5y ago
We had live chat in the 1990s.
crazygringo 5y ago
I disagree completely. I'm insanely more productive as part of a team than I was in 199x. Not even talking about things for software development specifically, but for general-purpose word-processing and spreadsheeting and scheduling, you've got: - Live collaborative cloud editing over mobile. The back-and-forth that previously might take a week can now be done in half an hour while you're in the back of an Uber in a different country - Googling how to accomplish spreadsheet tasks. Stuff that you'd just give up and not do before, or would take you days to figure out on your own, there are tons of blog posts and YouTube videos letting you get it done in half an hour - Tons of scheduling and information-gathering improvements. Looking at people's Google Calendars live to find a meeting that everyone can attend, sending a Google Form to collect lunch preferences rather than contacting people individually, and so on The productivity of modern administrative office tasks has skyrocketed with collaborative, mobile, cloud-based tools. Getting stuff done as a team with static Word and Excel files that were stuck on a physical computer at a physical office while you tried to decipher printed software manuals was slow.
- asdff 5y ago

drdec 5y ago

If we counted goofing off as part of productivity, we might start to see the gains we expected. Put another way, while computers have made us more productive, the internet has made it much easier to not do our jobs while at work. I don't think it is a coincidence that the graph stops being as steep around 2005.

dexen 5y ago

I find the opening graph to undermine the central thesis a bit: the dot-com bubble burst in 2002, but is barely reflected on the graph; the growth levels off in 2005 or 2006. If the change in productivity was largely software-moderated, I would expect a lager change around the dot-com burst. Meanwhile the large change seems to be the 2007 Subprime mortgage crisis - and presumably the follow-up change in interest rate and investment patterns.

nitwit005 5y ago

You can't eliminate all the paperwork and archival jobs twice. Some of the improvements tied to communication require cultural changes that can be slow. Telemedicine has been possible for a long time, but the shift only picked up steam due to Covid.

amelius 5y ago

If productivity stalled it's because we now have the equivalent of a TV set/entertainment system on our desks.

nonameiguess 5y ago

Something I don't see addressed here is how much of software is an arms race. I think this reality is hidden a bit from people who have only ever worked on commercial software that exists for the purpose of creating economic value. A lot of software doesn't have that purpose and exists mainly for defensive reasons. I have spent most of my career working on fielding new software systems for the intelligence community and the DoD. We can't say we haven't seen productivity gains in the form of many processes being automated to the point we can scale them much larger and process much more data. But this isn't economic productivity. 60 years ago, satellite imagery involved dropping film from the satellite on a little parachute and intercepting it before it hit the ground, developing the film, and deploying any improvements in imaging capabilities by launching a new satellite. Now we can do almost all of that with radio and software and we have virtually the entire globe covered, a near non-stop stream of imagery constantly being turned into possibly useful and actionable intelligence depending on what the interest is in knowing what is happening in that region. But in terms of what we're doing, much of it is economically purely a sink. We're monitoring foreign ports, known locations of military units, missile silos, to maintain the strategic advantage of not being caught with our pants down if anyone out there ever decided to launch a large-scale conventional attack. A lot of people would probably argue what we're doing is pointless, fighting yesterday's wars while losing today's. Maybe. I'm not really trying to make an argument either way for whether this activity is useful or not. But it's not increasing American economic output, and it's not intended to. But it is an incredibly expensive and enormous scale application of deployed software technologies. It's effectively a new category of cost for the world's major military powers. They now need to spend on maintaining an enormous development pipeline and operational environment for software capabilities that bring no economic gain, but just keep them from being overtaken by their enemies. You do see some patterns like this in commercial software, especially in the real of information security. We may or may not be able to easily deploy huge force multipliers to make our workforces more productive, but then we find they have vulnerabilities in them and we've exposed ourselves to a new kind of criminal taking advantage of that and extracting some of that value. So we devote more and more resources to securing these systems, often making them less efficient and more difficult to use in the process. We have to do it, because the added security is at least some of the time ultimately worth it due to the enormous cost of a breach. But it's purely defensive spending. You're not making your system any more effective at producing whatever it is your company produces that creates economic value. Often, you're making it less effective at doing that.

Spooky23 5y ago

Short term focus within companies drives cost control and growth, not productivity gains. The growth is the gremlin. Companies achieve monopoly/cartel status and they optimize for rent collection, which is the opposite of productive.

wantsanagent 5y ago

I take issue with the basic assumption of the article. Sustained productivity growth is hard and would not have continued without the software revolution. Let's take a look at agriculture. https://www.ers.usda.gov/amber-waves/2018/march/agricultural-productivity-growth-in-the-united-states-1948-2015/ https://www.ers.usda.gov/amber-waves/2018/march/agricultural... Mechanization and the widespread adoption and improvement of mechanized farming has lead to staggering productivity / farmer growth over the last 70 years. But there is only so much you can do with "dumb" machines. Today growth is being driven by computerized information gathering, planning, monitoring, and precision planting / soil maintenance. To maintain a growth curve takes constant innovation. Just because the growth doesn't significantly alter its slope does not mean that there is a missing improvement bump. If you decomposed slopes like these you would see they are compound sigmoids where growth is driven by one technology and then another, or an adoption of a new process, etc. So IMO if "software doesn't show up in productivity" you're not looking hard enough.

jldugger 5y ago
And TFP is a real monster of a formula[1]. It's not just GDP / hours. There's like 20 variables going into it's calculation, including things like 'labor quality' and 'capital's share of income' (alpha). I'm not a smart man, but I think this suggests that a society that lays off factory workers and retrains them as software engineers will not register on this metric. And looking at alpha, there's a pretty clear phase change at 2000 -- it's hovering at 31-33 for 50 years, then marches up from 0.31 to 0.38. Sounds to me like you could tell a story that labor is more productive, but seeing less of the gains than before. edit: just to belabor the point, here's a random chart I googled for US productivity that _doesn't_ feature the same trendline: https://tradingeconomics.com/united-states/productivity https://tradingeconomics.com/united-states/productivity. If anything it looks like productivity has accellerated during the past 20 years. [1]: https://www.frbsf.org/economic-research/indicators-data/total-factor-productivity-tfp/ https://www.frbsf.org/economic-research/indicators-data/tota...

shakezula 5y ago

If software is process, then what does that say about the labor value of software? Related: I think that software engineering is going to be the new blue collar warehouse or factory job that a good chunk of boomers enjoyed.

AnimalMuppet 5y ago

Because GDP doesn't measure right. It measures dollars, not value. Take Google search, for instance. I can look up (approximately) all the information in the world, for free. That shows up in the GDP as $0, because it's free. But what is Google search actually worth? Would your business, say, actually pay for it as a tool? Probably, at least for those of us who need Stack Overflow answers. Real value is being produced, but it isn't being measured because it is being given away. (Yeah, I know, ads. Search itself is still being given away. So is Linux and gcc and...) And does Google search help productivity? Yes. Does Linux? Yes. Does gcc? Yes. The ability to get all these things for free greatly expands the things you can do. Software is producing value. But because so much is being given away, the value isn't showing up in the dollar-based metrics.

kazinator 5y ago

Software is mostly a management technology. But there is all sorts of automation: CNC machines, industrial robots making stuff. I think that a graph of "total factor productivity in the USA" is misleading without looking at factors like, say, how much manufacturing has disappeared from the USA and gone overseas in that period!" You have to look at how much you're producing with how many people; and that cannot be some per capita based on the population, but the actual head counts in those industries that are covered by that graph: what is the productivity with how many people?

beefman 5y ago

My family watched Miracle on 34th Street a couple years ago. Aside from being generally impressed with how well it's aged, I was particularly impressed with the office technology on display (pneumatic tubes etc). In Victorian London, mail could be posted up to 12 times per day.[1] That's about as often as e-mail can be turned around. Bronze Age merchants exchanged clay tablets with remarkable throughput.[2] On the consumer side... I live in Silicon Valley. My grandparents had better access to services than I do — fresh milk delivery, an MD that came to their bedside, and an electric trolley — in the 1930s in a town of 12k ppl. My grandfather was a driver for a laundry service, my grandmother taught piano. [3] But maybe the most fundamental issue here is that productivity is 'measured' by dividing GDP by hours worked. But work seems better characterized as a mechanism that distributes, rather than creates, GDP.[4] [1] https://www.mentalfloss.com/article/24089/victorian-mail-delivery-12-times-each-day https://www.mentalfloss.com/article/24089/victorian-mail-del... [2] https://www.youtube.com/watch?v=bRcu-ysocX4 https://www.youtube.com/watch?v=bRcu-ysocX4 [3] https://twitter.com/clumma/status/1297571626901331968 https://twitter.com/clumma/status/1297571626901331968 [4] https://twitter.com/clumma/status/835581829654536192 https://twitter.com/clumma/status/835581829654536192

smoldesu 5y ago
A lot of those jobs ultimately became redundant for people. Milk delivery makes no sense when you can choose your bottle down at the grocery store next time you're there. Bedside care became impractical as medical technology advanced, and electric trolleys are pretty cumbersome (especially alongside city streets). Eventually, we realized that we could cut out the milkman: we laid off a lot of people in the process, but I'm sure the ice deliverymen are thankful that they no longer need to haul 25 pound bricks up New York staircases anymore.
- beefman 5y ago
  Ice delivery is the only valid example of a reduntdant job here. My grandparents bought milk at the grocery store, too. And I get milk delivered with Instacart. Medical technology did not make bedside care impractical. Quite the opposite. More can now be done from the bedside. Not that a typical doctor's visit involves any meaningful use of technology. Of course there are many other reasons why bedside care is superior.

3pt14159 5y ago

I'm not sure that we're measuring productivity correctly these days. Take AI, for example. Going from barely being able to play chess, to winning at Starcraft and Go and basically any game Google thinks is worth cracking is a huge, huge leap in technological capacity. What was the impact of the people that worked on these and related technologies on "productivity"? You can count apples. You can't really quantify DynamoDB or Rust on the economy. Now, median worker productivity growth seems like it's drastically slowed and I think we have real problems in the economy, but as everything started merging with tech it gets harder to see the full picture.

mytailorisrich 5y ago
Oh we are measuring productivity correctly. How much more productive is an software engineer that uses Rust compared to one that uses, say, C/C++? That is to say, do I need fewer software engineers to deliver the same product if they use Rust rather than another language? If the answer is "not really", then Rust does not increase productivity (I don't know the answer, btw). In general, I don't think that software as a tool made much difference to people's productivity in the last 20 years or so. The boost enabled by internet connectivity was probably over by ~2005. Since then not much has happened. Uber, Deliveroo, etc. are great for consumers but they don't increase the productivity of drivers: Drivers cannot drive faster or service more customers per hour, really, these are bounded by physical constraints.
- Jweb_Guru 5y ago
  > How much more productive is an software engineer that uses Rust compared to one that uses, say, C/C++? That is to say, do I need fewer software engineers to deliver the same product if they use Rust rather than another language? Anyone who's used both in production for any length of time can tell you that you will not deliver the same product in C++ unless you're willing to put in JPL levels of investment. So yes, it's far more productive for "the same product." The question is what you mean by "the same product." Do you consider a project implementing a feature set with loads of memory errors to be functionally the same thing as one without? If you do, then Rust probably won't be that much more productive than C++ (IME still more productive, but within the realm of subjectivity). If you do not, then it's not even close. This is the calculation companies like Microsoft and Google have been doing when they invested in Rust for new OS development.

lifeisstillgood 5y ago

I am writing a book covering a lot of this ground (shout out to Roald Coase) - but I have a different (ish) conclusion. It's not going to be a few "software-friendly" companies like FAANG that eventually lead the charge and we see productivity - it's waaay longer term than that. My take is software is a form of literacy - and it will only be when managers code daily that we will see enough of the control layers (model, monitor mentor) being actually software that software will show up in productivity stats If you like an analogy - steam engines used to power factories but there was one central engine and you spread out the power to other areas via bands / chains. Electricity came along but mostly replaced the central engine - it was not till people experimented with having power sent to many engines did the modern (Fordist) factory layout become feasible In short - everyone needs to learn to code or - if an SRE is what you get when you ask a coder to design a software development process, a programmable company is what you get when you ask a coder to design a company

jeffreyrogers 5y ago
Coding is hard though. I remember helping a smart friend of mine with his homework in college for an intro CS course and even basic fizz-buzz type stuff was very hard for him. This is a guy who has had a highly successful career post-college and did well in his (non-technical) major. I'm skeptical that more than a small fraction of the population will ever learn to code at more than a superficial level.
- lifeisstillgood 5y ago
  "They said you couldn't teach women to read" - Santa Claus' Grandfather in Arthur Christmas (Best Xmas kids film, not best source of political quotes but I am still asleep)
  - jeffreyrogers 5y ago
    I don't think anyone has seriously said that for a long, long time, if ever. Literacy is obviously pretty easy, much easier than algebra, for example, because in societies with universal schooling almost everyone learns how to do it.

michael1999 5y ago

Solid thinking. But I think the Jevons paradox deserves a mention. How much of our expanded capacity is spent on intangibles like extra complexity that doesn't show up in GDP?

deleted 5y ago

[deleted]

drivers99 5y ago

Cal Newport points out the lack of productivity increases in his book about email (which is actually about email and instant messaging), and how it's the default workflow tool and how badly it works for that.

fungiblecog 5y ago

In my experience (mostly as a consultant to the public sector) all the (potential) productivity gains brought by software are eaten up by two things: a) Layers of managers whose job appears to be to hold meetings to talk endlessly without ever making a decision or assigning an action, and b) a complete failure of the same managers to understand how to use the software at their disposal to its fullest extend (eg no enforcement of data quality, no idea how to report useful metrics). Both of these are the result of the people in positions of authority having no technical background or education and recruiting similar people. If they do think they need someone to analyse data they think that's a low-level position and hire based on a low-level salary with predictable results.