17 ms·
The Unix Philosophy
- alfiedotwtf 11y agoPeople will hang shit on ESR, but The Art of Unix Programming is one of my all time favourite books. If you haven't read it, even if you're not a *nix developer, do yourself a favour and just skim the table of contents... something may pique your interest and you may learn a thing or two. It's also free online: http://www.catb.org/esr/writings/taoup/html/
- jumblesale 11y agoIt's a really great read and did a lot to inform my approach to writing all kinds of software. It's a shame that I cringe to recommend in now due to the politics of its author.
- yarou 11y agoThat's really bizarre. A person's political views have no bearing on the validity or usefulness of what they write.
- davidgerard 11y agoThat's a bizarre statement in itself. Of course it does.
- peteretep 11y agoUndoubtably a deep philosophical argument, but brighter minds than ours have written at length about it both in science and jurisprudence. "Research ethics" and "Exclusionary rule" would be good illustrating Wikipedia articles.
- arethuza 11y agoOut of interest, what are these political views that are so terrible that they should influence our views of his thoughts on operating system design? Edit: I really don't have any idea what they are...
- philh 11y agoAmong other unpopular opinions, ESR denies AGW, believes that race and IQ are correlated, and was strongly in favor of the Iraq war.
- harkyns_castle 11y agoThat's a surprise to me, I hadn't heard any of that. Not that I've followed things that closely. What I found so far is [1], in which he seems to just be doing a bit of a "show me the data" thing wrt global warming (its fairly old I guess in defense). In [2] he's definitely saying IQ is race-related, and gender related to a lesser extent, in my quick readings. I quite like that he doesn't mince his words, sugar coat things or seem to take any notice of popular opinion/political correctness. Not agreeing with him, but I find that refreshing. [1] http://esr.ibiblio.org/?p=1436 http://esr.ibiblio.org/?p=1436 [2] http://esr.ibiblio.org/?m=200311 http://esr.ibiblio.org/?m=200311
- laichzeit0 11y agoRegarding [2]. So I guess this quote is the problem: > And the part that, if you are a decent human being and not a racist bigot, you have been dreading: American blacks average a standard deviation lower in IQ than American whites at about 85. [...] And yes, it’s genetic; g seems to be about 85% heritable, and recent studies of effects like regression towards the mean suggest strongly that most of the heritability is DNA rather than nurturance effects. So is the problem with him saying this that (a) this is factually false or (b) that it's an inconvenient fact that should be glossed over? He seems to be saying that it's factually true since he obviously read it in some or other study. If it is factually true it's disingenuous to label him as a racist.
- philh 11y agoThose seem only tenuously linked to me. Research ethics forbids you from doing certain things when performing research, but it doesn't say anything about the political opinions of researchers. The exclusionary rule says that if evidence is obtained by breaking the rules, it can't be used in court. But again, it doesn't say anything about the political opinions of the person obtaining the evidence.
- peteretep 11y agoBoth are cases where the truth of the situation is put aside from a moral standpoint, which is analogous to suggesting that someone's political views could influence the reception of their engineering views.
- TeMPOraL 11y agoBoth exist mostly to disincentivize people from doing something immoral and/or unwanted, not to give light on the truth or false value of observations/evidence. So the question is, should one really refuse to read/recommend ESR's book because of his opinions? Frankly, this smells to me like Index Librorum Prohibitorum all over again.
- oneeyedpigeon 11y agoUnfortunately, you can't neatly compartmentalise one aspect of an individual's life and totally separate it from the others. Sometimes, the aspects with which you happen to agree might be used to indirectly (even invisibly) support the aspects you don't. Although it's difficult to judge that, it seems a reasonable condition for exercising caution. I say this as a huge fan of TAOUP.
- philh 11y agoIf someone writes something, it's "true" or "false" regardless of their political opinions. On the other hand, someone's political opinions change what they're likely to write, so if you don't know whether it's "true" or not, knowing their political opinions can justifiably alter your beliefs about the text. I think people tend to overweight the second consideration, and it seems particularly irrelevant in this case. (Scare quotes because it's rarely as simple as true-or-false, e.g. "literally true but horribly misleading".)
- arethuza 11y agoI don't think jumblesale is arguing that his writings are incorrect or not useful, just that he doesn't like popularizing his works because of the authors political views.
- fnordfnordfnord 11y agoIt's still a shame. I certainly don't feel like I have to love everything a person does in order to learn something from them, or even respect some of their work. Likewise, I don't require everyone to agree with my personal opinions in order to pass my courses. Things would really become cumbersome were I to ever change my mind about something.
- camperman 11y agoI agree. Seriously, who gives a toss about what anyone's opinions are on subjects unrelated to what you're reading by them? And even if they do creep in, what's to say you're right or they're wrong? I thought this was part of being an adult but I see it all too often: "so-and-so wrote an excellent technical manual on X but he's right wing so I won't buy it!" I don't understand that way of thinking since no human being will ever 100% agree with any other on everything.
- coldtea 11y agoExcept if he writes anything about politics, economics, society, etc -- in which case his political views have quite a bearing on the validity or usefulness of what they write.
- yummyfajitas 11y agoCan't you evaluate his claims on their own merits? Unless he's saying "trust me, I won't expose my reasoning but it's solid", or unless you are outsourcing your thinking, such things would seem to be irrelevant.
- coldtea 11y ago>Can't you evaluate his claims on their own merits? As with most things, it takes too much time to do that in detail for every argument one hears. Thankfully with the magic of brain's pattern matching and previous experience to BS arguments we don't have to. We can eliminate tons of opinions from the list of "potentially interesting to investigate" by their mere showing of certain characteristics we already know lead to bogus thinking. ("Hey man, I made a perpetual motion machine. Wait, where are you going? Don't you wanna hear how I made it? You're so close minded" -- or "I don't believe in climate change, it's all bogus. Here's what I think about educational reform..."). Sure, we might get a few false negatives (some good suggestions lost because their originator is a bigot etc), but the system overall works wonders for reducing the signal to noise ratio.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- yummyfajitas 11y agoYou are conflating a property of the claim itself (violating the laws of physics) with a straw-man property of the person making the claim. They aren't the same thing - one enables a simple proof by contradiction, the other is ad-hominem. Further, your "climate change, it's all bogus" ad hominem isn't even a real claim that skeptics make, and is more more indicative of your thinking than of reality. I get the impression you are just looking for ways to dismiss arguments which make you emotionally uncomfortable. Consider religion instead, it's a lot more unapologetic about simply declaring who the heretics are.
- navait 11y agoWhen they constantly mix technology and politics, yes it does matter. Open Source is a political movement, and that requires taking ESR's version of it with a grain of salt.
- angersock 11y agoWho did a lot of the popularization of that term again? Who cofounded the Open Source Initiative? It's in no small part due to the politicking and goodthink of ESR that we have Open Source as it is today.
- the_why_of_y 11y agoOpen Source is not a political movement; you're thinking of Free Software.
- tjr 11y agoFurther, I suspect the main reason there's any concern at all about recommending ESR's writings due to his politics is because he bothers to write a lot about his political views in the first place, presenting something to potentially disagree with. Many programmers / writers / programmer-writers may well have equally strong political views in one direction or another, with which others may strongly agree or disagree, but they just don't say much about them in public.
- angersock 11y agoUnfortunately, the general zeitgeist of the times seems to be that a person's politics are somehow a litmus test for whether or not they should be listened to at all, about anything. Normally this would just be a quirk, but the fact is that a lot of technical people here on HN and other places would happily throw the baby out with the bath water just because they disagree with somebody's politics. It's stupid and unprofessional. With so many companies focusing on such technically boring problems, image management is perhaps legitimately more of a business concern than having the best tech available. So, unfortunately, we have people with dissenting opinions but excellent work slandered or ostracized...even if their opinions are actually worth considering. Then again, that just means that those of us who are more genuinely tolerant will have an edge during hiring. :) Also, on ESR in particular: You have to understand that, rightly or wrongly, his worldview is long-term Culture War. Literally anything which prevents The Right People from breeding faster (homosexuality) or defending themselves (attacks on the 2nd amendment) or arguing (kafkatraps) is suspect. Because he's playing for keeps, he'll do whatever it takes (including, perhaps, being less than perfectly equal in presentations on things) to further his agenda. That's just how it is, and it doesn't reflect on his technical contriubtions or aptitude at all. Hell, the bitch of it is, he's even arguably correct on some of his cultural points, if he himself (much less his detractors) didn't spend so much time sounding so disagreeable and grumpy and wingnutty. Anyways, it's just a sign of the times, as I said. It seems that most people are unable to handle a mental model which accounts for biased or unreliable narrators while still allowing the work of those narrators to be taken advantage of.
- Frondo 11y agoThey absolutely do, but I get your point that that's not necessarily so in every single case. ESR's page on the rationalwiki gets it right, though, when they call him a "stopped clock"--right every once in a while (and by coincidence): http://rationalwiki.org/wiki/Eric_S._Raymond http://rationalwiki.org/wiki/Eric_S._Raymond
- tjr 11y agoHow then do you evaluate the validity of writings by an author whose political views you do not know?
- Frondo 11y agoI don't, but I also don't feel the need to. If, however, some other tech blogger I liked started writing about "kill the gays," well, I don't need to overlook that just because I like their tech writing. Like I said elsewhere, none of these folks has a monopoly on good engineering writing/technical thinking. I can read, and ultimately promote, writers whose writing I wouldn't be ashamed to share with all of my friends.
- deleted 11y ago[deleted]
- spiralpolitik 11y agoPersonally I found it disappointing as it seemed to take the view that the only UNIX in existence since 1992 has been Linux. A better title would have been the Art of Linux Programming or the Art of Open Source Programming.
- davidgerard 11y agoThis book is ESR's masterwork, and that's taking into account everything else he's done.
- tobik 11y agoThanks. PDF version is here: http://www.catb.org/esr/writings/taoup/html/graphics/taoup.pdf http://www.catb.org/esr/writings/taoup/html/graphics/taoup.p...
- bcg1 11y agoChapters 1 and 2 (Philosophy and History) are required reading for all junior devs and aspiring hackers.
- larrys 11y ago"People will hang shit on ESR" An example "Reading it, it looks like a total hack job by a poor programmer." https://news.ycombinator.com/item?id=923660 https://news.ycombinator.com/item?id=923660
- yummyfajitas 11y agoThat's not hanging shit on ESR, that's just a valid critique of poor code. A better example might be this post, where ESR is implied to be racist when his actual post is a reflective one about correcting irrational racist reactions: https://news.ycombinator.com/item?id=6884767 https://news.ycombinator.com/item?id=6884767
- nickpsecurity 11y agoThanks for the book. Unlike most UNIX fans, he's unusually honest in his critiques of it and even references material from UNIX Hater's Handbook haha. His comparisons to other OS's are fair except for the robustness of VMS: main reason many companies kept using it despite its uncertainty. Downloading it to re-read and see if I find some more enlightenment as a system designer.
- hitlin37 11y agoin case of mankind extinction, we should preserve the above text and pass it on to the next race.
- digi_owl 11y agothe rule of separation makes me wonder if the long term legacy of Wayland will be as a hardware accelerated backend for X.
- jkot 11y agoX as a protocol is long gone. Direct memory access, kernel drivers etc are not really compatible with network protocol. Wayland is just streamlining and formalizing current situation.
- umanwizard 11y agoAt my old job (which I left about ten days ago) I ran X servers and clients on different machines daily. Typical example was running my IDE on my desktop, connected to an X server running on my laptop.
- forgottenpass 11y agoThe X developers didn't break remoting, but that doesn't mean contemporary gui toolkits still use the features of X that made X "network transparent." As I understand it, most features of the protocol are largely ignored except for the parts needed to pump bitmaps over a network. Here's an LCA talk by a dude what works on X and Wayland: https://www.youtube.com/watch?v=RIctzAQOe44 https://www.youtube.com/watch?v=RIctzAQOe44
- digi_owl 11y agoThe main reason, as best i can tell, is that every DE out employ OpenGL compositing that effectively bypass X. Meaning that to draw the desktop they effectively pain a single large window inside X that is then filled with the output of the GPU. Thing is that Wayland seems more comparable to svgalib than X. This in that Wayland pretty much a lib/protocol for talking to the graphics hardware. Something else, be it their reference implementation Weston, GTK, Qt, or some other alternative, has to handle the handling of windows, desktops etc. Right now you can use Wayland as a driver for Xorg.
- jokoon 11y agoThis should be taught in any programming class. > Rule of Diversity: Distrust all claims for “one true way”. Although, does the python rule "There should be one-- and preferably only one --obvious way to do it." contradict this one ? > Rule 5. Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. I wonder what rob pike has to say about OOP or java, I wish I could listen to it. Also it says that text is a good representation of data, but I think he meant it as intermediary. I don't think xml or html are really good choices when you see all the CPU cycles spent parsing those. > Rule of Economy: Programmer time is expensive; conserve it in preference to machine time. I prefer this rules versus the "no premature optimization" rule.
- mtrn 11y ago> I wonder what rob pike has to say about OOP or java, I wish I could listen to it. 12 minutes where Pike talks about a few things, that seem to be wrong in our software world: https://www.youtube.com/watch?v=5kj5ApnhPAE https://www.youtube.com/watch?v=5kj5ApnhPAE
- olavk 11y agoPerl is arguable much closer to the Unix philosophy than Python, for better or worse. That said, the "One obvious way" in Python is not exactly the same as "Only one true way".
- deleted 11y ago[deleted]
- vmsisbetter 11y agoRob Pike doesn't know Smalltalk, Self, or Common Lisp/CLOS--why should you care what his opinions are concerning OO as a paradigm?
- fao_ 11y agoMost developers touting OOP don't know those languages; so why should we care what their opinions are concerning OOP as a paradigm. If you exclude one person because they disagree with X and don't know Y, then you must equally disregard all who agree with and disagree with X, because they don't know Y. Otherwise you're just cherrypicking :)
- jkot 11y agoHow debacles such as micro-kernels and XServer fits into Unix Philosophy?
- grymoire1 11y agoSun had the NeWS windowing system before XWindows. Be glad that did not become the standard. Be very glad.
- arethuza 11y agoI rather liked NeWS - certainly it was a lot more fun to develop for at that time than X and had some wonderful UI toolkits (particularly HyperNeWS). For example, being able to draw a shape in a a graphical editor and then paste the shape directly to a window as the shape that window should take was rather cool - and this was in the early 90s!
- nickpsecurity 11y agoEasy: a number of microkernel-based systems today actually have the properties UNIX pretends to. And to the core.
- el33th4xx0r 11y agoI wonder how relevant is it nowadays. Many good things we can't have, like systemd, if we have to follow strictly to the unix philosophy.
- userbinator 11y agoFor a lot of people, the set of "good things" does not include systemd.
- vezzy-fnord 11y agoNot to mention you can certainly get feature parity with systemd while still being Unix philosophy-robust. See the nosh service manager.
- TickleSteve 11y agoThe principle stands just as it always has. The Single responsibility Principle (SRP), interfaces, cohesion and coupling are all relevant patterns and techniques that also describe the same design ideas. The UNIX philosophy is simply one example of the above principles implemented at a user interface level, but they apply equally well in low-level embedded code, and device drivers just as well as in desktop and server land. Software complexity grows exponentially with the number of internal interactions. by reducing the scope of any given module, you greatly simplify it. Thinking about, designing and implementing fixed interfaces between small cohesive modules helps this process and you get much more simple (as in the Rich Hickey sense) software as a result. These same principles pop up all over the place in software, the Unix Philosophy, SRP, Microservices... they're all manifestations of the same thing.
- spiralpolitik 11y agoSure you can. See 'launchd' on OSX. The reason that 'systemd' is the ball of mud it is that Linux being just a kernel does not provide the basics for systemd to build on so it has to provide them itself.
- vezzy-fnord 11y ago
- zvrba 11y ago> The ‘Unix philosophy’ originated with Ken Thompson's early meditations on how to design a small but capable operating system with a clean service interface. Except that a program-to-program interface based on formatting and parsing text is anything but clean.
- mbrock 11y agoWhat do you mean by "text"?
- alkonaut 11y agoExactly. That's one of the biggest problems with text-only interfaces.
- pdkl95 11y agoMaintaining IPC/RPC as text has the significant advantage of keeping the programs involved honest, approachable, and easier to debug. Just being able to see the interface by running the program (or reading the config file) makes it a lot easier to learn[1], and bugs in text formats can often be seen visually (or marked in an editor). The alternative - binary structures - require complex data definitions where you have to care about stuff like exact byte lengths when you have to convert between big/little endian. Some people like to claim that binary is faster, but your bottleneck is not going to be strtol(3). As for advantages in parsing - you still have to parse the binary structure you just read. I suspect most people that think "parsing text" is difficult are confusing the parser with lexer; the latter is the only part that changes when you switch between text and binary formats. In the long run the initial investment in a text interface can be cheaper than wasting time debugging binary structures. Even more important when thinking long-term: it is a lot easier to inspect a text interface from the outside without the consent or availability of the original author. If you have an old program binary that was used in production for years and no source code or documentation, which would you rather try to debug? An opaque binary file? Or something in JSON or INI format? [1] however, this is not an excuse to skip proper documentation
- fauigerzigerk 11y agoI keep wondering about what seems to be the most important component of Unix philosophy: write many small programs that do one thing well and interface using text streams! Yes, modularity is important. However, in some cases, this philosophy has resulted in the "tangled mess held together by duct tape" kind of systems architecture that no one dares to touch for fear of breaking things. I think Unix philosophy is struggling with a fundamental dilemma: On one hand, creating systems from programs written by different people requires stronger formal guarantees in order to make interfaces more reliable, stronger guarantees than interfaces within one large program written by one person or a small team would require. On the other hand, creating systems from programms written by different people requires more flexibile interfaces that can deal with versioning, backward and forward compatibility, etc, something that is extremely difficult to do across programming languages without heaping on massive complexity (CORBA, WS-deathstar, ...) I think HTTP has shown that it can be done. But HTTP is also quite heavy weight. It doesn't exactly favor very small programs. Handling HTTP error codes is not something you'd want to do on every other function call. In any event, I think Unix philosophy is a good place to start but needs a refresh in light of a couple of decades worth of experience.
- lmm 11y agoThe big problem I've found is serialization of sum types. thrift (or protobuf or any number of similar systems) is very good at serializing most of the data one tends to work with, with declarations that are suitably strict but easy to write. But it doesn't have a good way to represent "this field is an A or a B".
- nly 11y agoThrift has unions, so does Apache Avro. Afaik Protocol Buffers is the only one where you have to string together one yourself with a required tag and a bunch of optionals.
- cfallin 11y agoProtobufs has unions now too (they're called "oneofs"), as of version 2.6.0 released last year: https://github.com/google/protobuf/releases/tag/v2.6.0 https://github.com/google/protobuf/releases/tag/v2.6.0
- sytse 11y agoWhen making a microservice application you have to choose between a text (JSON) or binary (Thrift) interface. You cloud argue that the unix way is to make it JSON until it becomes a performance problem.
- ajtulloch 11y agoThis is a false choice - you can serialize/deserialize Thrift objects to/from JSON. I think the choice here is between having a clearly-defined /schema/ or not.
- sytse 11y agoThanks for the insight.
- coldtea 11y ago>You could argue that the unix way is to make it JSON until it becomes a performance problem. I'd argue that the unix way is to make it JSON until it becomes a performance problem by which time the format is so entrenched in so many programs and since there's no real central coordination, you just get to live with it.
- coldtea 11y ago>Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features. Which means that no program is more un-UNIX-y than Emacs...
- rekado 11y agoEmacs is a collection of separate Elisp programmes that all cooperate and share data structures. You would not call Unix tools un-UNIX-y only because they all share procedures from the C library. Emacs also interfaces with external processes (e.g. using comint mode). It's the ultimate bottle of glue.
- agumonkey 11y agoBecause Emacs is a system of its own, the Lisp Machine legacy leaked too much. Inside of emacs many things are tiny components, after all it's a lisp, which had a stronger unixy feel than so many things since. I mean lambda, compose, map... hard to separate concerns more than that.
- brudgers 11y agoEmacs is an example of a complex application written consistent [more or less] with the Unix philosophy. One might argue [anything, it's the internet after all] that Emacs four decade shelf life demonstrates the robustness of its embedded Unix philosophy [at the risk of being a bit circular]. Where it butts against contemporary practice is in places like threads where it is still not proven that YAGNI doesn't apply.
- oldpond 11y agoCan't upvote this enough. My takeaway from this gem is that we need to keep thinking about our craft even as we evolve our technologies, and we need to have the courage to stand up for it. The hardest thing in the world to see is your own point of view because you have to step outside of it to see it.
- McElroy 11y agoAt my place of work, we have a client relationship to a company like ours in a neighbouring country. They develop a software that is of much use to us so we pay a license to them for it. They are nice people but god damnit, there is one thing they just never got right. They only support one platform, one that is certified UNIX, yet no matter how severe an error, their cli tools and scripts exit with 0 no matter what. I'm the guy who writes some smaller tools and scripts on our side integrating with their software so you probably understand why I get a bit upset about this at times. Still, I enjoy my work and as I said they are nice people and also they are a quite small team so I don't want to burden them with these concerns when there are other things that our company need from them more. Anyway, I've been with my company for a few years and soon my contract expires and I'm going to study the field our company is in and get a degree in that, then I'm going to apply for a position doing our core business. I would still like to be involved with the software my current position is touching on, though, if possible. (Our company has 1000+ employees and several different sub-sections, so even though I might get back into the company, it's not a given that I'll be working with the group of people I am now even though I'd like to.) I also sometimes think that if possible, perhaps I'd like to work for that other company in our neighbouring country for a few years and be on the dev team of the software. After all, I have experience from the user side which the dev team has not and the dev team has seen some of the tools I've made and a couple of the guys seemed to think that some of that stuff was pretty decent.
- JoeAltmaier 11y agoThis can be a very good fit. Smart companies hire their customers' best engineers with the most familiarity with their product. The best, smartest feedback comes with that package.
- thinkmoore 11y agoThe Unix Hater's Handbook (https://en.wikipedia.org/wiki/The_Unix-Haters_Handbook https://en.wikipedia.org/wiki/The_Unix-Haters_Handbook) has a few sections on of some of the ills of the Unix philosophy.
- marcosdumay 11y agoThe UNIX Haters Handbook shows its age, it's criticizing a system that has only a passing similarity with modern unixes. But yes, it's funny.
- Fenume 11y agoAgreed. Modern unixes are a lot worse.
- thinkmoore 11y agoAgreed, I posted mostly for the humor, though there are some grains of truth there still.
- nickpsecurity 11y agoNonetheless, the specific examples show the UNIX system contradicted many of the supposedly "UNIX" principles. The article seems a bit revisionist in a pro-UNIX way in that sense. Unsurprising. Saying that these are modern UNIX principles might be more fair. Yet, Kemp's article inspires doubt in even that: http://queue.acm.org/detail.cfm?id=2349257 http://queue.acm.org/detail.cfm?id=2349257