10 ms·

A Generation Lost in the Bazaar (2012)

108 points by zbentley 5y ago

Hokusai 5y ago
> That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. I think that all the article is self disqualifying. Maybe this was acceptable language in 2012, but it was still having a bad effect on people, arrogance cannot substitute argumentation. A Bazaar in conjunction with regulations is more sensible. As an architect at a medium sized organization my job is not too build pretty cathedrals, but to allow the teams in my department to row in the same direction. I set non functional requirements, I set direction, but I cannot know the systems that each team is developing as well as they do. Bazaars require rules, but they evolve dynamically with the needs of the organization. I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way.
- felipemnoa 5y ago
  >I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way. Well, they are not jus nailing two pieces of wood. Also, if I were to build a cathedral today I would use 3d printing and CNC milling machines as much as possible.
  - TheOtherHobbes 5y ago
    Like this? https://www.bbc.com/news/av/technology-31923259 https://www.bbc.com/news/av/technology-31923259
  - WJW 5y ago
    For something like a cathedral, having it take a long time is a feature. Of course you could slap something of the same size together from prefab concrete blocks and some 3d printed stuff, but you would "just" get a soulless building with no effort from the faithful invested in it. A real cathedral is a monument for the ages, bigger than any individual who worked on it. Something for entire generations to pour their effort into so that at the end of construction you can say: "my grandfather started building this, my parents worked on it, I worked on it and during the time of my great-grandchildren this building will still be as magnificent as it is now". Trying to build it as quick and cheap as possible is simply not the point for a cathedral.
- analognoise 5y ago
  I think if everyone is simply rowing in one direction, you end up with a favela - functional, but not great in an earthquake or flood.
  - deleted 5y ago
    [deleted]
  - tempodox 5y ago
    I once talked to a CEO who expected and wanted an important piece of software to be rewritten every few months so it could use the app framework du jour. I wonder how many other shops there are where implementations aren't designed to last long, especially in web development.
- joshz404 5y ago
  Agreed. People don't flourish inside cathedrals, no matter how attractive they look.
  - mihaic 5y ago
    > Agreed. People don't flourish inside cathedrals, no matter how attractive they look. Nor do they flourish inside supermarkets, these buildings are designed for something else. Walking in St. Peter's in Roma gives a sensation of awe even to devout atheists, something I can't imagine the local Walmart doing.
    - lmm 5y ago
      Didn't Yeltsin experience something akin to a religious conversion on seeking an American supermarket? We're only numb to the miracle because we see it every day.
    - simiones 5y ago
      Off-topic, but all that the St Peter's Cathedral in Rome gave me was of obscene wealth and displays of wealth. For awe, I would much more recommend the Pantheon, especially knowing that you're stepping into a building for which 500 years is barely more than a quarter of its history.
      - dragonwriter 5y ago
        > Off-topic, but all that the St Peter's Cathedral in Rome gave me was of obscene wealth and displays of wealth. Even further off-topic, but if you are referring to the big church in the Vatican adjacent to St. Peter’s Square, it is Saint Peter’s Basilica; it is the second-ranking of the four major basilicas but it is not (though many tourist-focussed lists of cathedrals include it) a cathedral. The cathedral that is the seat of the Bishop of Rome (and also the first-ranking of the four major basilicas) is the Archbasilica of St. John Lateran.
        simiones 5y ago
- TheOtherHobbes 5y ago
  Barring apocalypse, Sagrada Familia will still be standing five hundred years from now. UNIX will be mostly forgotten by then - in the same way blacksmithing and carriage building are mostly forgotten today. That's the whole point of cathedral projects - make it beautiful and build it right because you're building for the future. Not to scratch your own personal boredom itch, or to nail a bit of wood to another bit of wood because why not.
  - dnautics 5y ago
    That has zero to do with development style, and everything to do with medium. There have been plenty of cathedral software projects that are already lost, much less lost 500 years from now.
    - dnautics 5y ago
      btw, the (purported) world's oldest running continuous market -- happens to also be called a bazaar -- was constructed in 1455, so it's been running in excess of 500 years.
  - the_af 5y ago
    > UNIX will be mostly forgotten by then UNIX may not be the best example for your argument: it's often criticized because of how long its legacy has endured, with some people arguing it's time to move on to more modern visions of what an operating system should be. So UNIX, for good or bad, is indeed the software equivalent of an ancient cathedral which endures from ages past.
    - lioeters 5y ago
      Indeed, UNIX will definitely not be forgotten 5 centuries from now (if humanity still exists by then). It will at least be in the history books, taught about in schools, and some quirky people will for sure be running it as a "retro computing" hobby.
  - simiones 5y ago
    Still, looking at ancient constructions, 150 years is a huge amount of time. Most roman constructions that have survived to today were built at most in a single life time - at an extreme end, it seems that the Hagia Sophia was built in 5 years. The more than 4500 years old Great Pyramid of Giza was built in a single pharaoh's lifetime. Which is to say, engineering culture and goals can absolutely degrade over time in terms of productivity, even with great increases in construction technology.
- simiones 5y ago
  > I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way. Interestingly, this may also be a sign of declining engineering. The Hagia Sophia, the largest church in the world for 1000 years, was apparently built in 5 years (532- 537). In general, it seems Rome was able to build fantastically persistent buildings in fractions of the time required later. Even today I wonder if we actually have the engineering and logistics practices required to build buildings that would survive thousands of years in a under a decade - of which the Romans had several.
  - felipemnoa 5y ago
    >>Interestingly, this may also be a sign of declining engineering. Or it could simply mean that not enough resources were committed to the project.
    - JetSpiegel 5y ago
      500 AD, that's about the time the Western Roman Empire collapsed, of course the Byzantines needed a gigantic project to show they were the real Roman empire now. It's probably more similar to the Manhattan Project.
- titzer 5y ago
  > Maybe this was acceptable language in 2012, but it was still having a bad effect on people, arrogance cannot substitute argumentation. I find it refreshingly honest, vivid, cathartic and spot on. Complaints from the peanut gallery just make me love it even more. In 9 or 90 years, maybe they'll think all of our 2021 eggshell tiptoeing to be completely useless drivel devoid of insight and dump it down the memory hole. Sensibilities change!
mihaic 5y ago
Things seem to be getting even worse, and as I hate to iterate on all the points often repeated on HN (the loss of older industry best practices, CV-driven development, bloated FAANG departments, the issues of funding OSS, etc.), I just want to notice one thing: Most developers and architects seem to have lower and lower confidence in what is technically achievable without some complex dependency web, and that becomes a self-fulfilling prophecy. As believing something is possible is a pre-requisite to actually approaching it, I expect things to continue getting worse.
- PragmaticPulp 5y ago
  > Most developers and architects seem to have lower and lower confidence in what is technically achievable without some complex dependency web, and that becomes a self-fulfilling prophecy One of the biggest problems I’ve had with hiring engineers out of big companies is that they’ve developed a sort of learned helplessness. The longer they spend in big, slow companies the more they’ve developed an idea that everything they do must adhere to mountains of process and approvals and documents. It can be a challenge to un-teach these ideas from people after hiring them into a company where they don’t need to put together presentations and seek multiples levels of approval before doing anything. One surprise that I didn’t account for is that many people come to like these process-heavy systems with approvals everywhere and no progress until the rules have been followed. It becomes a safety net to pull as many different people and process into a decision before action is taken. If something goes wrong, no one can blame you because you followed the process and got all of the approvals, right? I think this leads a lot of people down the road of preferring high-process, low-action environments because following the process is a type of low-effort, low-risk work whereas producing something new is high-effort, high-risk.
  - morpheos137 5y ago
    Complex process creates more work, sets barriers to entry for competitor workers, and reduces individual responsibility for something that goes wrong. So, from the employee perspective complex process means a lot of work to do (job security), higher pay (even if the domain problem is not too difficult process complication makes it difficult reducing the available supply of qualified labor) and less responsibility if something goes wrong. From the business point of view compexity means, being current, keeping up with the latest tech, signaling you are not a dinosaur. Keeping up with security because the shoddy earlier layers had holes because software engineers do not output a finished work product like other engineers. Even if some dinosaurs may do things cheaper and better. From the customer perspective the incentives are pretty often the same. Many customers want a touch screen instead of a knob or button because the latest tech all has touch screens. They want to use Slack or VS Code because that's what the cool kids do these days. From the outer space perspective it looks like humans are building a new tower of babble with their tech stacks. Ever higher layers of complexity not actually going any place or doing much new. Eventually it might all crumble into a pile of confusion.
deleted 5y ago
[deleted]
Communitivity 5y ago
There is truth in this article, but I do not feel as pessimistic as the author seems to feel. I think we need to push for more quality in Open Source software, and in commercial software. That quality can come from a team, though it seems more likely to be driven by a charismatic BDFL. The quality though comes from tests, attention to detail, QA, etc.
herio 5y ago
I do agree with PHK on his feelings about autotools. It is a cargo cult dumpsterfire, unfortunately it mostly works and nobody wants to go in and fix it. Also suffers from changes breaking stuff for somebody somewhere who insists that modern software should run on his ancient OS running on even older hardware. For me personally I have a bit of a dislike for GNU software in general because of the same type of things. It's all autotools and legacy code being dragged around. Don't get me wrong, I fully appreciate the GNU project and how much they have done; I just don't like their code very much. On a larger scale the Bazaar suffers from not having managers paying people to do boring things. This leads to boring things generally not being done as much, people on their own wants to work on interesting things.
- kryptiskt 5y ago
  Yes, there's a lot to be said for blowing away the cruft every now and then. At least autotools has been in a steady retreat since this was written and I don't see it too often now. It's perhaps noteworthy that this article coincided with autotools development winding down, after 2012 it didn't have a release until 2020 (https://lwn.net/Articles/834682/ https://lwn.net/Articles/834682/).
- taeric 5y ago
  Nothing particularly against managers, but that idea hits me a bad way. Managers have a tendency to want more and more in their management pool. Either people or products, if they can't get clients. So... I don't see that being a good path. Granted, developers often do this, too. If you can find a way to align incentives with users, I'm all for it.
- ho_schi 5y ago
  Take a look at Meson[1]. A lot of GNU, Gtk, GNOME and Freedesktops related projects have migrated or are migrating. The C, C++ and also Rust people use it. I've started reading the Autotools manual and turned away, to much bloat and hard to read code. Meson popped up and I immediately went that direction. [1] https://mesonbuild.com/ https://mesonbuild.com/
- the_gipsy 5y ago
joshuaellinger 5y ago
Hard to take the analysis seriously when it (a) take UNIX as an example and (b) pretends Linux doesn't exist. The Linux kernel looks to be a successful Bazaar project with a single person (Linus) having nominal responsibly. I recently return to UNIX after a 20 year break. Linux in 2020 is much better than Solaris in 1998. However, Windows made giant leaps as well. Both still have massive amounts of deadwood but Windows does a better job of hiding it.
- h2odragon 5y ago
  > Linux in 2020 is much better than Solaris in 1998 Depending on your purposes, Linux in 1998, was better than Solaris in 1998, even on Sun hardware.
- shadowgovt 5y ago
  Windows has the vested interest (customers and money) to make their deadwood compatible. There are some great stories about the engineering they've done to continue to support legacy applications (because those applications matter to customers).
- beprogrammed 5y ago
  Well he's a BSD guy, so he refers to the entire bunch as UNIX. "One of Brooks's many excellent points is that quality happens only if somebody has the responsibility for it" Linux is a prime example of this with Linus. I fear the day he's not around as a guiding force.
- tored 5y ago
  I don’t think the author says that a Bazaar project can’t be successful.
jrm4 5y ago
Overly dramatic headline is overly dramatic, and sub-headline is so obvious that it should be self-evident. I can't help but think all articles like this are fighting a very old and outdated strawman. I'd venture that literally no one is arguing that "Free Software is magic" anymore. Look, sometimes Free Software is the thing that works well, sometimes non-free but open, sometimes proprietary is what you have to use. There are a lot of things we can work on and do better, and concrete solutions and experiments are important to consider: but this sort of big picture whining isn't helping anyone.
ChrisMarshallNY 5y ago
I love that article, but it made me sad. It is just as relevant today, as it was, nine years ago. I'm a huge proponent of improving software Quality. I have found that complaining about it doesn't actually do much good. My complaints tend to get downvoted into the Mantle, and I just get pissed off. So, my approach, is to do the very best I can, in my own work, and limit the amount of other people's work on which I rely. When I do use it, I am quite careful, and vet the work. Even so, I have had to back out several highly-rated packages that I had thought were good, but my apps started crashing, as soon as I incorporated them. Just mentioning that, has often proved incredibly unpopular. It seems to be considered blasphemy. It does reduce the scope of what I can accomplish, but I am quite happy with what I do.
- yashap 5y ago
  Re: > I have found that complaining about it doesn't actually do much good. My complaints tend to get downvoted into the Mantle, and I just get pissed off. IMO creating real change in an organization is almost never done through complaining, it’s done through pushing practical solutions in a positive/constructive way. It might not be the content of your argument, but the way you present it. This comment, for example, instantly biased me against your argument, because the tone felt complain-y and self-righteous. I think regardless of what your argument is, if you present it with this tone, you won’t be persuasive.
  - ChrisMarshallNY 5y ago
    Oh, I wasn't talking about in my organization (which was actually famous for Quality). I was talking about making posts in tech communities, like this one. Software Quality is not a popular topic for discussion. HN isn't too bad, but some other communities react to Quality discussions, as if I was insulting God. I've learned to just keep my opinions to myself. Even then, when I talk about my own experiences, and my own process, it is not received well.
    - beprogrammed 5y ago
      The problem is, we get less of you and more of them, it damages us all. As long as you keep writing good software though, that's what really matters. We used to stand in a field where everywhere we looked we could see and hear another of us, now there's 10,000 fools between each of us, and it's rare for us to hear or see another over the crowd.
linuxhansl 5y ago
Yeah, yeah, autoconf sucks, so does maven (IMHO), software is complex, and, yes, we could do better... But, to reason from these examples about thousands of projects is disingenuous. The rumors of open source's demise have been greatly exaggerated. I use open source both privately (Linux, some open source games, FreeBSD a long time ago), and professionally (mostly Java and database stuff), and while sometimes I can't help rolling my eyes, on the whole open source works incredibly and surprisingly well; especially compared to the alternative of closed-only corporate software.
mgrennan 5y ago
If I could up vote this 100 times I would. I love open source. But 40 years in computing (hardware to apps) has left me with questions. Like, Why could I write an entire Point of Sale system in 64K in 1987 (Apple - The Store Manager) and now you couldn't do it in 64G.
vessenes 5y ago
Commenters here I think are missing what’s most interesting about this essay - it’s a sort of paean to an age we’ve now forgotten, an age that had ended in 2012, written by a high priest (published in the ACM!) Poul Henning-Kamp has been around long enough to see the end of his system and the rise of what Raymond talked about; high priests like Henning-Kamp just don’t matter as much as they used to, compared to the insane productivity of the entire world pumping out stuff whenever and however they want, gluing together whatever they need. There is a lot of shit being made, sold and given away in the Bazaar, but there are a lot of gems. A tech startup cost millions of dollars to start in say, 2008, the height of the dotcom one boom. For reference, discount hosting providers charged on the order of $30,000 a month for a single Sun server to serve your website. Today, a solid startup can glue together hundreds or thousands of open source libraries and launch product for free. Even if you ignore free time from the AWSes of the world, many a startup can structure their infra around serverless functions and be paying in the 10s of dollars a month. That’s the power of the bazaar, or whatever system we have now, which is more complex than Raymond described. I think the vast majority of users of this site do not want to return to a world where releasing products takes someone of Henning-Kamp’s intelligence and skill - there just aren’t that many of him in the world, and we would have far fewer awesome things in that world.
bsedlm 5y ago
as I see it, it comes down to ownership. Something about treating something which ain't yours (such as a software system with millions of lines of code) as if it were. Something about maintaining a complex software system which you are responsible for, but somehow are never allowed to make simpler for your own sake. You are only paid for work which increase the "value" (money, profit, whatever) of the real owners of the business. It's not really worth their money to pay you to make your own job easier. > Quality happens only when someone is responsible for it. The easy way for this to happen is to be able to own, really own what you do. This just doesn't happen in the 'software industry'.
- shadowgovt 5y ago
  Fully agree. This is one of the implicit advantages of most corporate structures; the ownership question has default answers. Who owns the product? The company. Who maintains the product? Whoever the company pays to maintain it. Usually, companies will build out trees of responsibility so some sub-group is responsible for smaller pieces of large products (with ultimate responsibility "rolling up" back to company ownership). How can I contribute? Your boss will tell you what needs maintenance or improvement. Do it or you're fired. If you have something else you want to work on, a good company will respond to your feedback and shuffle responsibilities around (you get better work when employees are satisfied), but ultimately, someone needs to maintain the build tools and if that's what they hired you to do, that's what you're doing until you decide to stop taking their money for it. There are lots of disadvantages to the corporate model, but (like the old chestnut from Thomas Paine's writings), it has the advantage of simple ownership.
fddddd 5y ago
this article is brought to you by the fat pigs that push opensourced code stealing at the top companies into closed source systems that gets rewritten (on top of mit/apache licensed code) again and again under the guise of "innovation" . what the cathedral gave us? wifi and usb standards that add literal nothing and just update the encapsulation frequencies every now and then with great fanfare. or maybe apple changing from bash to zsh? this thread is full of pundits complaining about gnu autotools but they are not aware of the move to different systems being discussed in very open forums. mostly people repeating propaganda they hear while being allowed to feel smugly supperior and still having no knowledge of the topic. gpl is not creating a perfect world because of fools who fall for this.
- dang 5y ago
  Would you please stop breaking the site guidelines by posting flamebait, calling names, and so on? We ban accounts that do those things. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
diegocg 5y ago
>The bazaar meme advocated by Raymond, "Just hack it," as opposed to the carefully designed cathedrals of the pre-dot-com The pre dot com days were not a golden age of software quality. He also has this idea that Unix is a "beautiful cathedral". No, the descriptions of the original environment where Unix was designed resembles to a internal AT&T version of a bazaar. Multics was the well architected, beautiful cathedral.
kazinator 5y ago
> The bazaar meme advocated by Raymond, "Just hack it," as opposed to the carefully designed cathedrals of the pre-dot-com years, unfortunately did, not die with the dot-com madness, and today Unix is rapidly sinking under its weight. A level-headed comparison of actual historic Unix sources, which are publicly available for study, reveals this to be false. Unix was always "just hack it", and later Unix is, overall, an improvement over earlier Unix. It seems I have not come across test cases in old Unix code bases. Can someone point to some circa 1980 Unix code base where there test cases? For anything: system calls, shell, yacc, cc, .. I think the idea may have been that if Unix rebuilds from scratch and you get to a system prompt where you can type "ls", the toolchain has been validated.
floatingatoll 5y ago
Is the book mentioned in this article worth reading?
bitwize 5y ago
Unix was invented long before modern dynamic linking schemes were. PHK seems to think it was possible, around 1990 or before, to wave a magic wand and have all vendors agree on a way to build and link dynamic libraries. There are a number of decisions to be made before you even get there. First, which binary format -- COFF? ELF? Something else? Secondly, how do you account for the fact that the library will be loaded into different programs' memory spaces at run time? Do you: 1) insist on PIC (what Linux does)? 2) replace all pointers with dummies and fix them up at load time (what Windows does)? 3) always load a given library to the same place in virtual memory (what Linux did in the a.out days)? Before you have a single compiler flag for dynamic library generation, you have to answer questions like this. And no single answer existed in the Unix environment of 1990, which wasn't a bazaar -- it was more like a struggle among competing cathedrals! Hence libtool, whose messiness reflects the messiness of the Unix scene at the time. GNU actually made monumental efforts to clean up the different Unixen and make them more consistent. Fortunately, today, all widely used Unixen these days seem to have settled on ELF format and option one -- all, that is, except macOS!
clankyclanker 5y ago
I occasionally contemplate this article (and am glad I found it again!), mostly because of this quote: > there is no standardized way to build a shared library in Unix. Instead of standardizing how to do that across all Unixen—something that would take just a single flag to the ld(1) command I've never looked into it, but I'd love to know what ld flag PHK was thinking about. Could the entirety of libtool still be replaced by that flag?
whatever_dude 5y ago
This is a great article that I find myself re-reading and referencing all the time. I think it's unfortunate that people only skim over it and then use it to complain/defend Linux/windows/m4/autoconf/etc. More important is the main point that "quality only happens when someone is responsible for it". This explains a mistake I see time and time again in all manners of software projects: anything by committee, anything overly democratic, can only aim to be mediocre at best. You need an owner/architect/champion/tyrant if you hope to achieve anything above average (not guaranteed though).

108 points by zbentley 5y ago

> That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. I think that all the article is self disqualifying. Maybe this was acceptable language in 2012, but it was still having a bad effect on people, arrogance cannot substitute argumentation. A Bazaar in conjunction with regulations is more sensible. As an architect at a medium sized organization my job is not too build pretty cathedrals, but to allow the teams in my department to row in the same direction. I set non functional requirements, I set direction, but I cannot know the systems that each team is developing as well as they do. Bazaars require rules, but they evolve dynamically with the needs of the organization. I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way.

felipemnoa 5y ago
>I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way. Well, they are not jus nailing two pieces of wood. Also, if I were to build a cathedral today I would use 3d printing and CNC milling machines as much as possible.
- TheOtherHobbes 5y ago
  Like this? https://www.bbc.com/news/av/technology-31923259 https://www.bbc.com/news/av/technology-31923259
- WJW 5y ago
  For something like a cathedral, having it take a long time is a feature. Of course you could slap something of the same size together from prefab concrete blocks and some 3d printed stuff, but you would "just" get a soulless building with no effort from the faithful invested in it. A real cathedral is a monument for the ages, bigger than any individual who worked on it. Something for entire generations to pour their effort into so that at the end of construction you can say: "my grandfather started building this, my parents worked on it, I worked on it and during the time of my great-grandchildren this building will still be as magnificent as it is now". Trying to build it as quick and cheap as possible is simply not the point for a cathedral.
analognoise 5y ago
I think if everyone is simply rowing in one direction, you end up with a favela - functional, but not great in an earthquake or flood.
- deleted 5y ago
  [deleted]
- tempodox 5y ago
  I once talked to a CEO who expected and wanted an important piece of software to be rewritten every few months so it could use the app framework du jour. I wonder how many other shops there are where implementations aren't designed to last long, especially in web development.
joshz404 5y ago
Agreed. People don't flourish inside cathedrals, no matter how attractive they look.
- mihaic 5y ago
  > Agreed. People don't flourish inside cathedrals, no matter how attractive they look. Nor do they flourish inside supermarkets, these buildings are designed for something else. Walking in St. Peter's in Roma gives a sensation of awe even to devout atheists, something I can't imagine the local Walmart doing.
  - lmm 5y ago
    Didn't Yeltsin experience something akin to a religious conversion on seeking an American supermarket? We're only numb to the miracle because we see it every day.
  - simiones 5y ago
    Off-topic, but all that the St Peter's Cathedral in Rome gave me was of obscene wealth and displays of wealth. For awe, I would much more recommend the Pantheon, especially knowing that you're stepping into a building for which 500 years is barely more than a quarter of its history.
    - dragonwriter 5y ago
      > Off-topic, but all that the St Peter's Cathedral in Rome gave me was of obscene wealth and displays of wealth. Even further off-topic, but if you are referring to the big church in the Vatican adjacent to St. Peter’s Square, it is Saint Peter’s Basilica; it is the second-ranking of the four major basilicas but it is not (though many tourist-focussed lists of cathedrals include it) a cathedral. The cathedral that is the seat of the Bishop of Rome (and also the first-ranking of the four major basilicas) is the Archbasilica of St. John Lateran.
      - simiones 5y ago
TheOtherHobbes 5y ago
Barring apocalypse, Sagrada Familia will still be standing five hundred years from now. UNIX will be mostly forgotten by then - in the same way blacksmithing and carriage building are mostly forgotten today. That's the whole point of cathedral projects - make it beautiful and build it right because you're building for the future. Not to scratch your own personal boredom itch, or to nail a bit of wood to another bit of wood because why not.
- dnautics 5y ago
  That has zero to do with development style, and everything to do with medium. There have been plenty of cathedral software projects that are already lost, much less lost 500 years from now.
  - dnautics 5y ago
    btw, the (purported) world's oldest running continuous market -- happens to also be called a bazaar -- was constructed in 1455, so it's been running in excess of 500 years.
- the_af 5y ago
  > UNIX will be mostly forgotten by then UNIX may not be the best example for your argument: it's often criticized because of how long its legacy has endured, with some people arguing it's time to move on to more modern visions of what an operating system should be. So UNIX, for good or bad, is indeed the software equivalent of an ancient cathedral which endures from ages past.
  - lioeters 5y ago
    Indeed, UNIX will definitely not be forgotten 5 centuries from now (if humanity still exists by then). It will at least be in the history books, taught about in schools, and some quirky people will for sure be running it as a "retro computing" hobby.
- simiones 5y ago
  Still, looking at ancient constructions, 150 years is a huge amount of time. Most roman constructions that have survived to today were built at most in a single life time - at an extreme end, it seems that the Hagia Sophia was built in 5 years. The more than 4500 years old Great Pyramid of Giza was built in a single pharaoh's lifetime. Which is to say, engineering culture and goals can absolutely degrade over time in terms of productivity, even with great increases in construction technology.
simiones 5y ago
> I love the sagrada familia un Barcelona, but it's still under construction after 150 years. Building cathedrals maybe not the way. Interestingly, this may also be a sign of declining engineering. The Hagia Sophia, the largest church in the world for 1000 years, was apparently built in 5 years (532- 537). In general, it seems Rome was able to build fantastically persistent buildings in fractions of the time required later. Even today I wonder if we actually have the engineering and logistics practices required to build buildings that would survive thousands of years in a under a decade - of which the Romans had several.
- felipemnoa 5y ago
  >>Interestingly, this may also be a sign of declining engineering. Or it could simply mean that not enough resources were committed to the project.
  - JetSpiegel 5y ago
    500 AD, that's about the time the Western Roman Empire collapsed, of course the Byzantines needed a gigantic project to show they were the real Roman empire now. It's probably more similar to the Manhattan Project.
titzer 5y ago
> Maybe this was acceptable language in 2012, but it was still having a bad effect on people, arrogance cannot substitute argumentation. I find it refreshingly honest, vivid, cathartic and spot on. Complaints from the peanut gallery just make me love it even more. In 9 or 90 years, maybe they'll think all of our 2021 eggshell tiptoeing to be completely useless drivel devoid of insight and dump it down the memory hole. Sensibilities change!

mihaic 5y ago

Things seem to be getting even worse, and as I hate to iterate on all the points often repeated on HN (the loss of older industry best practices, CV-driven development, bloated FAANG departments, the issues of funding OSS, etc.), I just want to notice one thing: Most developers and architects seem to have lower and lower confidence in what is technically achievable without some complex dependency web, and that becomes a self-fulfilling prophecy. As believing something is possible is a pre-requisite to actually approaching it, I expect things to continue getting worse.

PragmaticPulp 5y ago
> Most developers and architects seem to have lower and lower confidence in what is technically achievable without some complex dependency web, and that becomes a self-fulfilling prophecy One of the biggest problems I’ve had with hiring engineers out of big companies is that they’ve developed a sort of learned helplessness. The longer they spend in big, slow companies the more they’ve developed an idea that everything they do must adhere to mountains of process and approvals and documents. It can be a challenge to un-teach these ideas from people after hiring them into a company where they don’t need to put together presentations and seek multiples levels of approval before doing anything. One surprise that I didn’t account for is that many people come to like these process-heavy systems with approvals everywhere and no progress until the rules have been followed. It becomes a safety net to pull as many different people and process into a decision before action is taken. If something goes wrong, no one can blame you because you followed the process and got all of the approvals, right? I think this leads a lot of people down the road of preferring high-process, low-action environments because following the process is a type of low-effort, low-risk work whereas producing something new is high-effort, high-risk.
- morpheos137 5y ago
  Complex process creates more work, sets barriers to entry for competitor workers, and reduces individual responsibility for something that goes wrong. So, from the employee perspective complex process means a lot of work to do (job security), higher pay (even if the domain problem is not too difficult process complication makes it difficult reducing the available supply of qualified labor) and less responsibility if something goes wrong. From the business point of view compexity means, being current, keeping up with the latest tech, signaling you are not a dinosaur. Keeping up with security because the shoddy earlier layers had holes because software engineers do not output a finished work product like other engineers. Even if some dinosaurs may do things cheaper and better. From the customer perspective the incentives are pretty often the same. Many customers want a touch screen instead of a knob or button because the latest tech all has touch screens. They want to use Slack or VS Code because that's what the cool kids do these days. From the outer space perspective it looks like humans are building a new tower of babble with their tech stacks. Ever higher layers of complexity not actually going any place or doing much new. Eventually it might all crumble into a pile of confusion.

deleted 5y ago

[deleted]

Communitivity 5y ago

There is truth in this article, but I do not feel as pessimistic as the author seems to feel. I think we need to push for more quality in Open Source software, and in commercial software. That quality can come from a team, though it seems more likely to be driven by a charismatic BDFL. The quality though comes from tests, attention to detail, QA, etc.

herio 5y ago

I do agree with PHK on his feelings about autotools. It is a cargo cult dumpsterfire, unfortunately it mostly works and nobody wants to go in and fix it. Also suffers from changes breaking stuff for somebody somewhere who insists that modern software should run on his ancient OS running on even older hardware. For me personally I have a bit of a dislike for GNU software in general because of the same type of things. It's all autotools and legacy code being dragged around. Don't get me wrong, I fully appreciate the GNU project and how much they have done; I just don't like their code very much. On a larger scale the Bazaar suffers from not having managers paying people to do boring things. This leads to boring things generally not being done as much, people on their own wants to work on interesting things.

kryptiskt 5y ago
Yes, there's a lot to be said for blowing away the cruft every now and then. At least autotools has been in a steady retreat since this was written and I don't see it too often now. It's perhaps noteworthy that this article coincided with autotools development winding down, after 2012 it didn't have a release until 2020 (https://lwn.net/Articles/834682/ https://lwn.net/Articles/834682/).
taeric 5y ago
Nothing particularly against managers, but that idea hits me a bad way. Managers have a tendency to want more and more in their management pool. Either people or products, if they can't get clients. So... I don't see that being a good path. Granted, developers often do this, too. If you can find a way to align incentives with users, I'm all for it.
ho_schi 5y ago
Take a look at Meson[1]. A lot of GNU, Gtk, GNOME and Freedesktops related projects have migrated or are migrating. The C, C++ and also Rust people use it. I've started reading the Autotools manual and turned away, to much bloat and hard to read code. Meson popped up and I immediately went that direction. [1] https://mesonbuild.com/ https://mesonbuild.com/
the_gipsy 5y ago

joshuaellinger 5y ago

Hard to take the analysis seriously when it (a) take UNIX as an example and (b) pretends Linux doesn't exist. The Linux kernel looks to be a successful Bazaar project with a single person (Linus) having nominal responsibly. I recently return to UNIX after a 20 year break. Linux in 2020 is much better than Solaris in 1998. However, Windows made giant leaps as well. Both still have massive amounts of deadwood but Windows does a better job of hiding it.

h2odragon 5y ago
> Linux in 2020 is much better than Solaris in 1998 Depending on your purposes, Linux in 1998, was better than Solaris in 1998, even on Sun hardware.
shadowgovt 5y ago
Windows has the vested interest (customers and money) to make their deadwood compatible. There are some great stories about the engineering they've done to continue to support legacy applications (because those applications matter to customers).
beprogrammed 5y ago
Well he's a BSD guy, so he refers to the entire bunch as UNIX. "One of Brooks's many excellent points is that quality happens only if somebody has the responsibility for it" Linux is a prime example of this with Linus. I fear the day he's not around as a guiding force.
tored 5y ago
I don’t think the author says that a Bazaar project can’t be successful.

jrm4 5y ago

Overly dramatic headline is overly dramatic, and sub-headline is so obvious that it should be self-evident. I can't help but think all articles like this are fighting a very old and outdated strawman. I'd venture that literally no one is arguing that "Free Software is magic" anymore. Look, sometimes Free Software is the thing that works well, sometimes non-free but open, sometimes proprietary is what you have to use. There are a lot of things we can work on and do better, and concrete solutions and experiments are important to consider: but this sort of big picture whining isn't helping anyone.

ChrisMarshallNY 5y ago

I love that article, but it made me sad. It is just as relevant today, as it was, nine years ago. I'm a huge proponent of improving software Quality. I have found that complaining about it doesn't actually do much good. My complaints tend to get downvoted into the Mantle, and I just get pissed off. So, my approach, is to do the very best I can, in my own work, and limit the amount of other people's work on which I rely. When I do use it, I am quite careful, and vet the work. Even so, I have had to back out several highly-rated packages that I had thought were good, but my apps started crashing, as soon as I incorporated them. Just mentioning that, has often proved incredibly unpopular. It seems to be considered blasphemy. It does reduce the scope of what I can accomplish, but I am quite happy with what I do.

yashap 5y ago
Re: > I have found that complaining about it doesn't actually do much good. My complaints tend to get downvoted into the Mantle, and I just get pissed off. IMO creating real change in an organization is almost never done through complaining, it’s done through pushing practical solutions in a positive/constructive way. It might not be the content of your argument, but the way you present it. This comment, for example, instantly biased me against your argument, because the tone felt complain-y and self-righteous. I think regardless of what your argument is, if you present it with this tone, you won’t be persuasive.
- ChrisMarshallNY 5y ago
  Oh, I wasn't talking about in my organization (which was actually famous for Quality). I was talking about making posts in tech communities, like this one. Software Quality is not a popular topic for discussion. HN isn't too bad, but some other communities react to Quality discussions, as if I was insulting God. I've learned to just keep my opinions to myself. Even then, when I talk about my own experiences, and my own process, it is not received well.
  - beprogrammed 5y ago
    The problem is, we get less of you and more of them, it damages us all. As long as you keep writing good software though, that's what really matters. We used to stand in a field where everywhere we looked we could see and hear another of us, now there's 10,000 fools between each of us, and it's rare for us to hear or see another over the crowd.

linuxhansl 5y ago

Yeah, yeah, autoconf sucks, so does maven (IMHO), software is complex, and, yes, we could do better... But, to reason from these examples about thousands of projects is disingenuous. The rumors of open source's demise have been greatly exaggerated. I use open source both privately (Linux, some open source games, FreeBSD a long time ago), and professionally (mostly Java and database stuff), and while sometimes I can't help rolling my eyes, on the whole open source works incredibly and surprisingly well; especially compared to the alternative of closed-only corporate software.

mgrennan 5y ago

If I could up vote this 100 times I would. I love open source. But 40 years in computing (hardware to apps) has left me with questions. Like, Why could I write an entire Point of Sale system in 64K in 1987 (Apple - The Store Manager) and now you couldn't do it in 64G.

vessenes 5y ago

Commenters here I think are missing what’s most interesting about this essay - it’s a sort of paean to an age we’ve now forgotten, an age that had ended in 2012, written by a high priest (published in the ACM!) Poul Henning-Kamp has been around long enough to see the end of his system and the rise of what Raymond talked about; high priests like Henning-Kamp just don’t matter as much as they used to, compared to the insane productivity of the entire world pumping out stuff whenever and however they want, gluing together whatever they need. There is a lot of shit being made, sold and given away in the Bazaar, but there are a lot of gems. A tech startup cost millions of dollars to start in say, 2008, the height of the dotcom one boom. For reference, discount hosting providers charged on the order of $30,000 a month for a single Sun server to serve your website. Today, a solid startup can glue together hundreds or thousands of open source libraries and launch product for free. Even if you ignore free time from the AWSes of the world, many a startup can structure their infra around serverless functions and be paying in the 10s of dollars a month. That’s the power of the bazaar, or whatever system we have now, which is more complex than Raymond described. I think the vast majority of users of this site do not want to return to a world where releasing products takes someone of Henning-Kamp’s intelligence and skill - there just aren’t that many of him in the world, and we would have far fewer awesome things in that world.

bsedlm 5y ago

as I see it, it comes down to ownership. Something about treating something which ain't yours (such as a software system with millions of lines of code) as if it were. Something about maintaining a complex software system which you are responsible for, but somehow are never allowed to make simpler for your own sake. You are only paid for work which increase the "value" (money, profit, whatever) of the real owners of the business. It's not really worth their money to pay you to make your own job easier. > Quality happens only when someone is responsible for it. The easy way for this to happen is to be able to own, really own what you do. This just doesn't happen in the 'software industry'.

shadowgovt 5y ago
Fully agree. This is one of the implicit advantages of most corporate structures; the ownership question has default answers. Who owns the product? The company. Who maintains the product? Whoever the company pays to maintain it. Usually, companies will build out trees of responsibility so some sub-group is responsible for smaller pieces of large products (with ultimate responsibility "rolling up" back to company ownership). How can I contribute? Your boss will tell you what needs maintenance or improvement. Do it or you're fired. If you have something else you want to work on, a good company will respond to your feedback and shuffle responsibilities around (you get better work when employees are satisfied), but ultimately, someone needs to maintain the build tools and if that's what they hired you to do, that's what you're doing until you decide to stop taking their money for it. There are lots of disadvantages to the corporate model, but (like the old chestnut from Thomas Paine's writings), it has the advantage of simple ownership.

fddddd 5y ago

this article is brought to you by the fat pigs that push opensourced code stealing at the top companies into closed source systems that gets rewritten (on top of mit/apache licensed code) again and again under the guise of "innovation" . what the cathedral gave us? wifi and usb standards that add literal nothing and just update the encapsulation frequencies every now and then with great fanfare. or maybe apple changing from bash to zsh? this thread is full of pundits complaining about gnu autotools but they are not aware of the move to different systems being discussed in very open forums. mostly people repeating propaganda they hear while being allowed to feel smugly supperior and still having no knowledge of the topic. gpl is not creating a perfect world because of fools who fall for this.

dang 5y ago
Would you please stop breaking the site guidelines by posting flamebait, calling names, and so on? We ban accounts that do those things. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

diegocg 5y ago

>The bazaar meme advocated by Raymond, "Just hack it," as opposed to the carefully designed cathedrals of the pre-dot-com The pre dot com days were not a golden age of software quality. He also has this idea that Unix is a "beautiful cathedral". No, the descriptions of the original environment where Unix was designed resembles to a internal AT&T version of a bazaar. Multics was the well architected, beautiful cathedral.

kazinator 5y ago

> The bazaar meme advocated by Raymond, "Just hack it," as opposed to the carefully designed cathedrals of the pre-dot-com years, unfortunately did, not die with the dot-com madness, and today Unix is rapidly sinking under its weight. A level-headed comparison of actual historic Unix sources, which are publicly available for study, reveals this to be false. Unix was always "just hack it", and later Unix is, overall, an improvement over earlier Unix. It seems I have not come across test cases in old Unix code bases. Can someone point to some circa 1980 Unix code base where there test cases? For anything: system calls, shell, yacc, cc, .. I think the idea may have been that if Unix rebuilds from scratch and you get to a system prompt where you can type "ls", the toolchain has been validated.

floatingatoll 5y ago

Is the book mentioned in this article worth reading?

bitwize 5y ago

Unix was invented long before modern dynamic linking schemes were. PHK seems to think it was possible, around 1990 or before, to wave a magic wand and have all vendors agree on a way to build and link dynamic libraries. There are a number of decisions to be made before you even get there. First, which binary format -- COFF? ELF? Something else? Secondly, how do you account for the fact that the library will be loaded into different programs' memory spaces at run time? Do you: 1) insist on PIC (what Linux does)? 2) replace all pointers with dummies and fix them up at load time (what Windows does)? 3) always load a given library to the same place in virtual memory (what Linux did in the a.out days)? Before you have a single compiler flag for dynamic library generation, you have to answer questions like this. And no single answer existed in the Unix environment of 1990, which wasn't a bazaar -- it was more like a struggle among competing cathedrals! Hence libtool, whose messiness reflects the messiness of the Unix scene at the time. GNU actually made monumental efforts to clean up the different Unixen and make them more consistent. Fortunately, today, all widely used Unixen these days seem to have settled on ELF format and option one -- all, that is, except macOS!

clankyclanker 5y ago

I occasionally contemplate this article (and am glad I found it again!), mostly because of this quote: > there is no standardized way to build a shared library in Unix. Instead of standardizing how to do that across all Unixen—something that would take just a single flag to the ld(1) command I've never looked into it, but I'd love to know what ld flag PHK was thinking about. Could the entirety of libtool still be replaced by that flag?

whatever_dude 5y ago

This is a great article that I find myself re-reading and referencing all the time. I think it's unfortunate that people only skim over it and then use it to complain/defend Linux/windows/m4/autoconf/etc. More important is the main point that "quality only happens when someone is responsible for it". This explains a mistake I see time and time again in all manners of software projects: anything by committee, anything overly democratic, can only aim to be mediocre at best. You need an owner/architect/champion/tyrant if you hope to achieve anything above average (not guaranteed though).