33 ms·

A significant amount of programming is done by superstition

304 points by mwcampbell 12y ago

boomshucka 12y ago
A significant amount of everything is done by superstition.
- swyman 12y ago
  Still, important to be occasionally reminded of your own assumptions and corner-cutting.
  - kremlin 12y ago
    Though as the author himself implies, it's not always appropriate to allocate the time necessary to do anything but corner-cutting. "If you say that you don't take this relatively fast road for Linux init scripts, I'll raise my eyebrows a lot. You've really read the LSB specification for init scripts and your distribution's distro-specific documentation?"
  - boomshucka 12y ago
    To be clear, the intention was to point out this (generally bad practice) is practiced in many fields. The intention wasn't to suggest it's ok/good.
nicpottier 12y ago
I actually really like the content of the article, but I'm not convinced superstition is the right word here. Yes, absolutely I have copied an init script / makefile or whatnot and if it worked after I tweaked it then I went on my business. But I didn't use it as is because of superstition, but rather because it now worked I believed it correct. Yes, quite often that came without a perfect knowledge of every single option, certainly usually without referring to the manual for every one of them. Perhaps this is semantics. But the definition of superstition is: a widely held but unjustified belief in supernatural causation leading to certain consequences of an action or event, or a practice based on such a belief Which ya, isn't quite the right thing. Anyways, agree with the sentiment of this 100%, just not the title. :)
- infogulch 12y ago
  The term "blind faith" works, but may be a bit harsh.
  - tjradcliffe 12y ago
    Except that it's not blind and it's not faith. We generally have some idea of what's in the black boxes we use, and we are always open to change our beliefs based on new evidence ("faith" is a belief that will stay the same no matter what evidence is presented: in Bayesian terms it is a belief with a prior of 0 or 1, neither of which can be changed by any evidence.) "Hopeful" or "optimistic" programming might be better.
  - Someone 12y ago
    Call it "trust", and it looks different. Do I trust my OS to have a memory allocator that works and to protect processes from reading and writing each other's memory? Yes, but not totally. Do I trust its more modern and complex APIs? Less so. Do I trust code I copied from stackoverflow that has 100+ points? Yes, but even less so than I trust the complex or new corners of my OS. So, I read every line I copy. On the other hand, I am lazy, so if this gets to 1000+ lines, chances are I won't read every line. I will think a bit more about the trust issue, though. Do I trust the documentation of the libraries shipping with my OS? More than stack overflow answers with 20-ish points, but certainly not completely. Do I make assumptions about APIs? I try not to, but it is hard. There is lots of documentation where it is hard or impossible to find out what the code claims to do in edge conditions. For example, in https://msdn.microsoft.com/en-us/library/352y4sff(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/352y4sff(v=vs.110)...., can one pass a null transaction? I wouldn't know, and the page doesn't tell me, so this afternoon, I wrote a conditional operator calling another constructor if transaction is null. I did consider a "let's try it and assume it always works if it works once" approach, though. I also think many people do that ('if it compiles and runs, it must be valid C')
- baddox 12y ago
  Yeah, I don't really get what definition of "superstition" the author is intending here. It seems like "laziness" or "lack of rigor" would be more appropriate terms.
ggchappell 12y ago
Maybe a better term than "superstition" might be "validated experience". We do what has worked in the past. It is true that there are standards that should be read and followed; the ones referred to in the article are probably among these. But other standards have a fair amount of wishful thinking in them. (How many of you JS developers have read the latest ECMAScript standard?) Sometimes, what has worked in the past is a better guide for practice than what ought to work.
frandroid 12y ago
The problem here is that if it works, how is it superstition? When I knock on wood or throw salt over my shoulder, I don't have a way of verifying that I've averted disaster. However, I can test the code I don't understand, and if it does what it says it does (or does what I want it to do, since that's why I've selected it in the first place), then we're a step removed from superstition. There might be unintended consequences if I truly don't understand the code, but that is something else than superstition.
- rbanffy 12y ago
  If you do not understand the code, you cannot say you tested it. It may have corner cases you did not anticipate. The test may be incomplete and only cover what you think the program should do.
  - tjradcliffe 12y ago
    Nonsense. We test stuff all the time we don't understand "completely" (which is some kind of magical standard, as we obviously don't understand the universe completely, so the very atoms and molecules that the machine is made of are in some sense not well understood.) Certainty is the alchemist's stone of philosophy. People have done all kinds of interesting stuff while attempting the impossible: turning knowledge into certainty. They can't, but that doesn't mean knowledge--which is inherently uncertain--is inferior. It means certainty is inferior, because it can't actually be achieved, regardless of what any pre-Bayesian thinker might have imagined. Knowledge is always uncertain. Tests are always incomplete. This does not mean "knowledge is impossible" or "you cannot say you tested the code" but rather "certainty is a chimera" and "compete test coverage is a futile goal." The whole point if good engineering is to adopt standards that are both achievable and useful, and the kind of test coverage that people run on code they are unfamiliar with to ensure it will do what they want it to is generally adequate for that. Evidence: most shipping code actually works.
    - sheepmullet 12y ago
      "Evidence: most shipping code actually works." The vast majority of time spent on software development is on bug fixing and on many projects the construction/coding defects make up more than half of all the defects.
rjbwork 12y ago
I think a better title might be "A vast majority of programming is done at a level of abstraction that this author is slightly uncomfortable with". If you work in Java or Ruby or Python or C#, or even C++ all day with say, an IoC container for argument's sake, you're certainly not constantly thinking "okay this interface will be resolved to this type, which has these dependencies which will be resovled to these types, which generate this byte code, which generates this JITed assembly code, which generates this machine code, which generates these executions of instructions on the processor and accesses these memories, which invokes these manipulations of these gates and flip flops on this particular part of the ALU". Doing so would be the mark of an unproductive madman.
- ggchappell 12y ago
  That's an interesting way of putting it. The issue is that we all have our own (more or less completely undocumented) abstractions that we mentally put on top of whatever language/library we're using. And sometimes these mental abstractions are out of sync with reality.
- dandelany 12y ago
  I don't think this is about abstraction. Abstraction implies having a deep understanding of the problem domain and encapsulating it in such a way that the implementation details can be ignored and the domain can be reasoned about at a higher level. I think a better title would be "A significant amount of (shipped, so-called production-quality) code is not fully understood by its authors".
  - rjbwork 12y ago
    Perhaps. One could argue that one of these init file templates is just a base class with parts of the implementation overridden. :D
  - tjradcliffe 12y ago
    So an even better alternative title would be "The Author of This Essay Does Not Understand Engineering, Software, Technology, Medicine, Or Any of the Other Practical Arts Which Have Always Involved Shipping Imperfectly Understood Machines".
anon4 12y ago
I'll have to admit something about myself -- I've never been able to learn how to do anything from just reading its first principles. I need to have a working example that I can then adapt and change and observe changes in the result. This has been especially true in programming. I've been writing programs in some capacity for 18 years now and not once in that time have I been able to simply read the reference manual of anything and following that write working code on my first try. Well, obviously I've been able to do it for trivial things, like after the Python 2->3 switch when map/filter/range started returning iterables instead of lists, simply being told "map/filter/range/etc. now return an iterable and not a list" is sufficient to explain the change in behaviour, but that's only because I know all the concepts behind it. You could say the explanation is only one level removed from my intuitive knowledge. Things start getting exponentially harder the more levels removed they get, i.e. if it's a new concept explained using new concepts previously defined, but for which I have no intuitive knowledge what they mean.
- GauntletWizard 12y ago
  Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'. Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key to science. You have to observe, change, guess, run, observe, change, observe until you can successfully predict what will happen for each change you make. The scientific method is taught to everyone, even if most ignore it and skip steps. 'Guess/check' is the correct way to do real-world engineering and design, not pure-logic simulation, because we still don't know all the first principles.
  - deleted 12y ago
    [deleted]
  - avmich 12y ago
    > Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However, both experimenting and reasoning can be flawed, and part of the trick is to learn how to do that properly. Computer science is difficult.
depsypher 12y ago
A nice example of this is yoda expressions in, say Java. The initial reasoning for them was to prevent accidental assignment, since a C/C++ compiler will complain on this: if (null = x) but not on this: if (x = null) In Java, neither is allowed, so the whole yoda construct is almost pointless. The exception is boolean assignment which does benefit from this idiom, but the loss in readability is a trade off to consider. In any case, I suspect most uses of this construct in languages like Java fall into the "superstition" category, where the people using it aren't really considering why.
- Peaker 12y ago
  A lot of us find Yoda conditions easier to read: if(0 == very_long_function_name_here(param1, param2)) { Tells me a lot after I've read a little of the line, whereas: if(very_long_function_name_here(param1, param2) == 0) { Makes the == 0 part easy to miss, and buried behind a lot of clutter.
  - mrspeaker 12y ago
    eh, you just end up having to parse if (VERY_LONG_AND_SPECIFIC_ENUM_NAME_HERE == func(param1)) { } Makes the function called easy to miss and buried behind a lot of clutter ;)
    - Peaker 12y ago
      That's why my rule is shorter first :-)
  - depsypher 12y ago
    Re: "buried behind a lot of clutter" If your code has a lot of clutter it's usually telling you something about your design. For readability a better way of handling it might be for the function could return a boolean instead of an int: if (is_valid(param1, param2)) { self documenting code and all that.
    - Peaker 12y ago
nightski 12y ago
I am sorry, the only reason I have been as successful as I have been is because I am not superstitious at all. I know that if something goes wrong, there is always a logical explanation for it. The quicker I can find that logical explanation the better I am doing. I don't think I am alone in this.
- lectrick 12y ago
  > The quicker I can find that logical explanation The quicker your test suite can find that logical explanation... ;)
- agumonkey 12y ago
  I often wish to have investigation classes. How to systematically reduce the search space in order to find meaning in explaining a phenomenon, good or bad.
- nullc 12y ago
  Is the reward for getting a comment posted so quickly so great that you don't bother reading the article even when the price of making yourself look like a fool?
  - nightski 12y ago
    I did read it, there were quite a few comments before I even posted. What he described was not superstition.
    - nullc 12y ago
      Indeed, what described was not superstition; there doesn't appear to be an exact word for what he was talking about (cargo cult is perhaps closer but not quite right), so it was a fuzzy match. What you responded to, however, was superstition and as a result your comment had basically nothing to do with the article. I hope you can forgive my misunderstanding. :)
johnny99 12y ago
It's true. All of my programs go from line 665 to 667. I've never made a commit on Friday the 13th, ever--and the one time I did it caused irretrievable data loss. Once I walked under a ladder, and it broke the build.
throwawayaway 12y ago
as a concerned cancerian, i am glad somebody is willing to admit it. i leave my sub conscious do the heavy lifting. "this is the general shape". making a list before starting is probably the extent of my preparation. if i can copy what's gone before without understanding it, all the better. only when i get stuck do i engage critical thinking. maybe this has been to my detriment. very often i notice myself and colleagues making up nonsense explanations for why c++ behaves the way it does without any real basis other than intuition and truthiness.
venomsnake 12y ago
Since I began programming I don't believe in miracles. I count on them.
dsugarman 12y ago
even with code I write myself, I can't remember the line by line implementation on the entire code base. To be honest I often forget implementation details on code I wrote last week.
- eric_h 12y ago
  Ditto. When a coworker asks me about implementation details of some code a wrote just a few weeks ago, my first answer is always "I haven't a fucking clue". Then I scan the code in question and it all comes rushing back.
  - eric_h 12y ago
    I should add that the extent to "it all comes rushing back" is directly and inversely correlated with how long ago I wrote it, and positively correlated with how interesting/difficult the problem was to solve at the time. Also, almost every time I go back to review code I wrote more than a year ago, my first reaction is "jesus, that's a stupid pattern/implementation. Why did I do it that way?". I've been a professional developer for about 12 years now and I suspect that will never change.
    - quicklyfrozen 12y ago
      It won't. And sometimes you'll start to change it, hit a wall, and then realize "oh that's why I did it that way."
mkramlich 12y ago
I think when I was very young and new to programming I sometimes did coding via superstition. But it was a long time ago, in say my first year or two, when I didn't yet have a good mental model of how the computer worked and how the language compiler or interpreters worked. Once you become solid at that I've found that 99.9999% of the time that computing is refeshingly rational and deterministic and amenable to logic.
benihana 12y ago
I realized that the entire piece is hinged upon this incorrect assumption: >This isn't something that we like to think about as programmers, because we'd really rather believe that we're always working from scratch and only writing the completely correct stuff that really has to be there; This is the issue I have with this article - the author assumes everyone works in exactly the same way they do and so are vulnerable in exactly the same way to their vulnerabilities. I don't know anyone who actually approaches programming in such a way that they think they're writing things from scratch and it's correct. I mean, defensive coding is a concept that has existed for decades that specifically addresses how incorrect the code we interact with is. The fact that I started coding in an environment where I had to manage memory myself and now I don't makes me very aware of the fact that everything I'm using is an abstraction built upon other abstractions. I dunno, the day I learned to program in Intro to C and Intro to Computer Science, defensive coding, not trusting user input, and the completely understandable at the time weirdness of the guy who came before you have been stressed as things to pay attention to.
- Retra 12y ago
  Usually when I'm writing code it is "Let's get the infrastructure up and come back to clean up after I know it works..." Sometimes I never come back. It doesn't mean I think the code is correct, only that it is not the biggest problem.
dblohm7 12y ago
This explains why so many websites don't validate email addresses properly.
- nulltype 12y ago
  How do you validate email addresses properly?
santosha 12y ago
I've always enjoyed what I call uncoding, which is taking a working piece of code and taking stuff away until things break, and examining how they break. It's really helped me understand what each piece does, and be more efficient when I'm writing it myself.
- eric_h 12y ago
  Amusingly, that's also more or less how we've learned to understand the various functions of the different parts of the human brain. We don't know what it does until it's been removed/damaged.
- forgueam 12y ago
  I love that term. You described exactly how I work: take an existing block of code, break it apart until I fully understand every line, then reassemble to meet my needs.
copsarebastards 12y ago
I'm surprised by the number of people who don't see this as a problem. I've worked pretty recently as the most senior developer on a team, and I saw tons of issues caused by copy/pasted code. It results in a lot of issues. Subtle bugs around `==` versus `===` in JavaScript, unecessary variables that obfuscate the code, Python that isn't Pythonic, C that was written as an example and therefore doesn't include the error handling that is constantly necessary in C (how many C examples check the return value of malloc?). The biggest problem, I think, is that often you end up with 10 different ways to solve the same problem in the code because instead of looking at the existing code, people Googled for a solution with subtly different keywords than the previous person and found a different blogger. This kind of duplication is very hard to identify and remove, and this does cause bugs. Of course a more experienced developer can work around these issues, but a more experienced developer also knows off the top of their head how to write the code themselves and usually does it.
- bentcorner 12y ago
  > The biggest problem, I think, is that often you end up with 10 different ways to solve the same problem in the code because instead of looking at the existing code, people Googled for a solution with subtly different keywords than the previous person and found a different blogger. This kind of duplication is very hard to identify and remove, and this does cause bugs. It's interesting to code in large code bases. They end up getting to the point where no single person can ever understand the entire thing. It is usually at this point that things like Hungarian and strict coding guidelines really help. If you need an array of pointers to some kind of object, you can be reasonable certain that search will be able to find it.
  - woah 12y ago
    IMO if your code base is large, you have already failed.
    - copsarebastards 12y ago
      If your code base is large, you've failed as a coder. If your code base never gets large, you've failed as a company.
Animats 12y ago
I've referred to this for years as "ritual-taboo programming". Ritual-taboo societies are ones in which things are done in certain ways because they worked in the past, but no one knows why. Too much of programming is like that. If you work from the specifications for a library, you'll probably find that some of the documented features don't work. If they weren't used by some "framework", or mentioned in a how-to book, they probably haven't been exercised well.
danschumann 12y ago
Such as, people are taught not to use global variables, but maybe they weren't taught the reasons why not. The reasons (some of) would be polluting the global namespace, having unexpected things defined in unexpected places, and so on. At some point, it could save the programmer oodles of time to just use global variables. It could be a project with limited scope where polluting and things won't be an issue. The reason for superstitions at all are stopping people from doing something when they can't understand the reasons yet. It's hard to explain to a child all of the reasons why they should be a good person, and much easier to tell them Santa will give them presents, until they can understand why being good is inherently its own reward.
thoughtsimple 12y ago
I've worked with developers who seem to code mostly by going to stack overflow, finding something that seems to fit their problem and then they copy code into the app. No attempt to find a more recent version or a more correct solution of the problem is made. I've seen JavaScript code copied from a stack overflow answer that was dated 2009. There is nothing better than a 6 year old answer to a common JavaScript problem.
mwsherman 12y ago
A significant amount of programming is done without articulating either the problem or the solution.
gee_totes 12y ago
What would be an example of good programming primary sources?
jrochkind1 12y ago
What that means is that if you're writing the system you want someone to do 'right' -- you better provide lots of good examples in the docs. A 'good' example is one that you've actually tested for real to make sure it works, that follows the intended principles for the system in question, that meets actual use cases developers using it will have, etc. A significant amount of software is released without good docs -- examples being just one part of good docs. So of course developers copy from the examples they can find.
rilita 12y ago
Article: Reinvent the wheel, because if you just mount whatever wheels fit the axle they will inevitably work poorly for your purposes. Wheels work just fine. Choose the best wheel from those available and tweak it as desired. Test it to make sure it doesn't shatter under too much weight. Apply and move on. This pervading notion of "if you don't understand you shouldn't do it" is silly. Sure it is nice to understand, but it doesn't put food on the table. Sometimes it's just handy to use the gun as is and go shoot your dinner in the woods. You don't need to understand the formulation of gunpowder or rifling in order to use a gun properly for it's intended use.
systematical 12y ago
Superstition...right I don't shave until a project is done and I have lucky underwear that I program in...
ff_ 12y ago
While the article is dramatically true (credits to StackOverflow for some of my production code), there's a factor that isn't taken into account here: time. Yes, because as engineers our time is more valuable than a minimal and perfectly-standard-compliant solution. I would happily read all the RFCs, and all the significant papers in CS, and all...etc. The reality is that we don't get paid to read, but to ship.
mpdehaan2 12y ago
A very sigifigant part of Ansible service module code was done to deal with misbehaving init scripts. They are very much cargo-culted just like RPM or debian package formats - borrow the ones from your previous project. They are hard to get right and many apps don't daemonize correctly or return OK before they are ready to be running - and then are actually running much later. Upstart/systemd didn't neccessarily make that more clear for people to understand either. While I haven't tried it, I recently encountered a reference to https://github.com/jordansissel/pleaserun https://github.com/jordansissel/pleaserun and it sounds promising. I don't know how much of this applies to programming in general, really, but init scripts are.. yes... special things.
- GauntletWizard 12y ago
  I dislike tools that claim to automatically generate your configuration for you. At Google, one of the worst abuses that anyone had to deal with was auto-generated configuration; a tool purported to deal with configuration for you, but really just blutrted a boilerplate mess into your directory, many of the properties and setting set by cargo-cult or overriding sane defaults with baked-in versions of the same. Facebook is proceeding down this path; There's a new tool that will do all of your service stuff for you! I look forward to what a boondoggle those services that use it will be. There was a discussion I recently stumbled upon about init systems in containers. Many processes spawned lots of zombies! You need a full init! Well, no; If your process is forking new processes, it should probably wait for them. If it's forking other processes that fork other processes, and they're not cleaning up zombies when they're done, that is a bug in those processes. If all else fails, your root-level process should probably implement a simple init on it's own (a wait() on loop in the background, dumping wait objects into a LRU map that other processes can do an equivalent of waitpid() on). Or just run under bash.
- stonith 12y ago
  I've used pleaserun a little in the past. When you're being quick and dirty and using fpm it fits into the workflow quite nicely. It can be a bit awkward if you're deploying things that need anything complex in their init script, like a bootstrap command or something, but if it's just a straight start|stop|restart it works quite well. It's probably not up to generating distro-guideline-compliant init scripts (and fpm obviously doesn't either for spec file) but so long as that's not the goal it's a useful tool.
Sevzinn 12y ago
Check .gitignore to see if it appears copy and pasted or actually thought through--this will invariably indicate the quality of the rest of the code.
haberman 12y ago
I am someone who tends to work from first principles, to a fault. It slows me down a lot. But it's like a compulsion. StackOverflow has improved my programming life dramatically, because it makes it a lot easier to "memoize" the search from what you're doing down to first principles. Whatever question you're trying to get clarity on, there is probably a StackOverflow answer out there that builds a positive case for a particular option, going all the way back to primary documents like standards.
danbruc 12y ago
This does not resonate with me at all. I can not stand having things in my code I don't fully understand. Without reading the source code of libraries, without reading relevant standards, without understanding the mathematics behind algorithms, there is always this feeling I may make really dumb mistakes. Really understanding all the bits and pieces obviously does not make your code bug free, but I am usually pretty confident that I did not do something horribly wrong.
- svisser 12y ago
  But what does it mean to fully understand your own code? Everyone relies on millions of lines of code that they didn't write themselves.
  - danbruc 12y ago
    For me that really depends on what I am dealing with. Most of the time the library source code is good enough. Sometimes - I am a C# developer - I will have to have a look at the .NET runtime specification. Sometimes I will have to learn details of Windows. In some cases I will even look into Intel's reference manuals to understand what is happening. I would say I stop at the level where I don't expect the layer below to be relevant to what I am actually dealing with.
- ackalker 12y ago
  This piques my interest. When was the last time anyone read the source code of (g)libc "cover to cover" before using it? Does anyone know the precise implications of (not) defining any of the myriad sensible permutations of its "feature test macros"? Is anyone even sure that those implications are actually implemented correctly before relying on them?
  - danbruc 12y ago
    I obviously do not and can not trace the entire code path from a print statement through the class library, the operating system, the display driver and the hardware, some things I will still take for granted. But if there is a really critical aspect to what I am doing, say some thread synchronization, I will definitely not just follow some sample code, but read the relevant parts of the memory model of the language specification detailing why this works as expected. If the underlying hardware matters, I will grab the reference manual for it, too. A lot of this is surly curiosity and you can get away without it most of the time, but at least as soon as I run into the first problem it really helps me a lot to have at least a basic understanding of what is going on in the layer of abstraction I am interacting with and maybe a layer or two below. And it gives me a really comforting feeling, that I know what is going on better than describing it as some black magic.
perlgeek 12y ago
sysv init scripts in particular are so pointless to write. In the end, the whole script usually revolves around four lines of actual code (starting a process, stopping it, querying the status, optionally sending it a SIGHUP to make it reload its configuration), but the result is a script with 50 lines of boilerplate. Which, of course, could nearly all be automated away by a sane, declarative format. Leaving all the political mess aside, that's the part that systemd got right. You write sometthing like [Unit] Description=some human-readable text here [Service] Type=simple User=... ExecStart=/path/to/deamon --with=options KillSignal=SIGKILL [Install] WantedBy=multi-user.target and that's it. Quick comparison of sysv init script lengths on Debian Wheezy vs. systemd service files on Debian Jessie: ssh: 162 vs. 15 lines, cron: 92 vs 11 lines, dbus: 122 vs 9 lines My point being: it's a bit too much to demand that people carefully engineer their sysv init files, when about 90% of those very same init files is simply boring, repetitive boiler plate code. Also, what happened to the "don't repeat yourself" principle? I tried writing a sysv init script from scratch. I stopped because of sheer boredom, and resorted to copy&paste to save both time and nerves.
- vezzy-fnord 12y ago
  The issue isn't so much in the medium (shell), but the underlying program itself (sysvinit) being overly primitive. See the daemontools family where runscripts are largely as short or shorter than systemd unit files: http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/run-scripts-and-service-units-side-by-side.html http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/ru... In addition, the BSDs have much shorter rc scripts mainly by putting all the boilerplate functions into a single library file that is then sourced. Gentoo's OpenRC framework, as well.
  - digi_owl 12y ago
    Franly i find myself thinking that moving the boilerplate elsewhere is also what systemd and upstart does. But those implement it in C rather than shell script.
- amyjess 12y ago
  Not just systemd, but... everything else that's trying to replace sysvinit, too. Upstart jobs are really simple, and OpenRC and modern BSD initscripts are also much simpler than SysV (and old BSD) initscripts. I think we're at the point where regardless of the systemd vs. everything else debate, we can all agree that SysV initscripts are shite. > I tried writing a sysv init script from scratch. I stopped because of sheer boredom, and resorted to copy&paste to save both time and nerves. Better yet, at my last job, I had some extra time, so I rewrote Ubuntu's qpidd initscript as an Upstart job, and it was so satisfying to see that giant script get condensed down to a few lines.
krick 12y ago
That's right, but I guess we're missing the real problem here. The real gold are these words: > You've really read the LSB specification for init scripts and your distribution's distro-specific documentation? If so, you're almost certainly a Debian Developer or the equivalent specialist for other distributions. Many of us would agree, that it's true. But why is it true? Shouldn't that make us suspicious in the first place? It's almost paradoxical, even funny: making the work done is easier, than finding out, how to do that work. You see, init scripts or makefiles aren't exactly rocket science. I mean, things we are willing to achieve with them are pretty simple: just start some process, just build some project (which means "execute bunch of shell commands in the right order", and not implementing out own superoptimizing compiler). So when we arrive to situation like one we've got today, the right message wouldn't be "come on guys, you see what we're doing because of being lazy? we should stop being so lazy!". That is, yeah, we all are lazy and maybe we should stop it, but that's not the point. What this situation actually means is that our tools for doing simple stuff are overcomplicated shit, and our documentation for these tools is shit. If anything, it's pretty obvious that a couple of lines of code, showing how to do the work is better documentation than complete list of possible flags and parameters with no examples whatsoever. So no wonder people are reading stackoverflow instead of man-pages. And so on.
- throwawayaway 12y ago
  > it's pretty obvious that a couple of lines of code, showing how to do the work is better documentation than complete list of possible flags and parameters with no examples whatsoever. So no wonder people are reading stackoverflow instead of man-pages. And so on. the best manpage is a man page with a nice EXAMPLES section you can grep to quickly. though the permutations offered by many programs prohibit all bases being covered.
  - mgob 12y ago
    http://bropages.org http://bropages.org to the rescue!
    - throwawayaway 12y ago
everyone 12y ago
I have a tendency to not use things I dont feel that I fully understand. This makes me stick to using the more basic elements of the language. In C# for example recently, there was a part of a game I am writing in my spare time for which 'events' would have suited. But I still dont feel I understand what events in C# is doing exactly. So instead I just put all the things I want the event to happen to in a list, and when the event happens everything in the list has a particular function called on it. I will probably get around to using C# events eventually when I have looked at them more and feel like I understand them. There is maybe a very small performance cost to what I did (according to some stuff I read, but I havent tested this so I dont know) But at least I have an architecture for my game and code that makes sense to me and that I feel like I understand completely (at least at the abstraction level of C#)
Sophistifunk 12y ago
If init scripts are mostly boilerplate that people keep copy-pasting, maybe the init process is the problem, rather than the poor chumps who have to write the scripts?
nickysielicki 12y ago
I'm a proud cargo culter. Truth of the matter is you can't just stare at a man page or textbook all day and try to understand, then code. You'll code some mess you don't fully understand, sure, but later on you'll come back and either realize it was wrong and fix it, or you'll understand why it was right. But you will eventually understand. The people trying to understand it will have gotten nowhere in the mean time.
anupshinde 12y ago
The author sometimes confuses the work "superstition" with "abstraction". That is how Humans work everywhere. And that is why my idle PC cannot create a Facebook/Twitter/or even a plain website by itself.
martininmelb 12y ago
I think this is what I refer to as SODD - or 'Stack Overflow Driven Development'. Somebody is not sure how to do something, so they look for an example on SO and they copy the code without really understanding what they're doing. (note: this is different to when they get an answer on SO and copy it knowing what they're doing).
jnem 12y ago
I hate to call myself out, but I envy most of you. It sounds as if many here get the opportunity to work on interesting problems from end to end. As such, first principles matter, and approaching something from scratch makes sense. I however have been stuck in a segment of the market that doesn't appreciate the right solution, but rather just wants a solution as fast as possible. To be clear, I'm talking about a more consumer audience. My clients expect things done quick, and quick means leveraging cms, ecommerce platforms (magento, woo commerce, etc.). To be frank I find such work unsatisfying, and often need to hack things together to meet the requirements set forth within the constraints of the budgets I'm working with. I'd love to be able to work in an environment where doing things right mattered, but the plain fact is the deliverables aren't measurable based on quality, but expedition. While I'm on a soap box, I'd like to point out to those in the positions where they're work is evaluated by professionals and peers how lucky they are. I'd kill for a job/client where my code is evaluated by other developers. This community sometimes forgets they are the 1% of the development world. Those who work for the googles, facebooks, and visionary startups with leaders who came from the same cloth. The rest majority of us are laboring away under management that have never produced a lick of code, or even design, for that matter. I guess the Tl;DR is: appreciate the fact that you even get to consider the finer points, and pray for the rest of us.
yason 12y ago
In my view, a lot of this is just trying to avoid learning yet another system. Learning systems is hard and takes time, and it's only worth it if you're going to be using the system a lot in the future. For example, creating a new init script or a Debian package description is something I don't do all the time. I don't want to learn all that stuff because by doing so I'd basically push out something else from my mind. Copying an existing configuration or script and making modifications without trying to create something original is an excellent pattern for things outside your immediate expertise target. The "avoiding creating something original" part does indeed carry a sort of "superstition". It's like relying on a new culture by merely mimicking it, because you don't really know it well enough to break the rules correctly. You don't know why things are done the way they are but your best bet is just to copy and adapt. This creates a sphere of fuzzy knowledge where intents and black magic seem to alternate.
lmm 12y ago
This is specific to shell scripting and to the history of nix standardization. No-one follows or cares about the LSB[1]; certification is expensive and based on buggy test programs that practically force you to "code to the test" and not to the specification. And when so many implementations ignore the spec, it would be more superstitious to code to the magical spec and assume it's going to work, rather than writing something and testing it empirically. When I'm programming (in Python or Scala) I can and do check the language specification if something is confusing, and I don't think I'm alone in this. It may be a "significant" amount of programming, but there are definitely areas of programming with more rigour. [1] I blame Debian, usually quixotic in its adherence to standards like the FHS - but when it came to the LSB they prefer to pretend that "alien" counts as RPM support. That set the tone for how much other distros tend to care about the spec.
- the_why_of_y 12y ago
  IMHO "alien" isn't much of an issue since LSB packages are allowed to use only a small subset of RPM features; the fancier features aren't portable between different RPM-based distros either. http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/pkgscripts.html http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-gener... http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/pkgdepend.html http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-gener... A bigger problem is Debian gratuitously using different SONAME than required by LSB for some libraries - your LSB binary can't use those system libraries if it wants to run on Debian. But at least it looks like they fixed that problem in Wheezy.
cel1ne 12y ago
The answer to "Why we can't build software the way we build bridges?" is that bridges and other physical works disintegrate over time. Code doesn't rot, so the mistake of 1980 can be freshly reused 40 years later.

304 points by mwcampbell 12y ago

boomshucka 12y ago

A significant amount of everything is done by superstition.

swyman 12y ago
Still, important to be occasionally reminded of your own assumptions and corner-cutting.
- kremlin 12y ago
  Though as the author himself implies, it's not always appropriate to allocate the time necessary to do anything but corner-cutting. "If you say that you don't take this relatively fast road for Linux init scripts, I'll raise my eyebrows a lot. You've really read the LSB specification for init scripts and your distribution's distro-specific documentation?"
- boomshucka 12y ago
  To be clear, the intention was to point out this (generally bad practice) is practiced in many fields. The intention wasn't to suggest it's ok/good.

nicpottier 12y ago

I actually really like the content of the article, but I'm not convinced superstition is the right word here. Yes, absolutely I have copied an init script / makefile or whatnot and if it worked after I tweaked it then I went on my business. But I didn't use it as is because of superstition, but rather because it now worked I believed it correct. Yes, quite often that came without a perfect knowledge of every single option, certainly usually without referring to the manual for every one of them. Perhaps this is semantics. But the definition of superstition is: a widely held but unjustified belief in supernatural causation leading to certain consequences of an action or event, or a practice based on such a belief Which ya, isn't quite the right thing. Anyways, agree with the sentiment of this 100%, just not the title. :)

infogulch 12y ago
The term "blind faith" works, but may be a bit harsh.
- tjradcliffe 12y ago
  Except that it's not blind and it's not faith. We generally have some idea of what's in the black boxes we use, and we are always open to change our beliefs based on new evidence ("faith" is a belief that will stay the same no matter what evidence is presented: in Bayesian terms it is a belief with a prior of 0 or 1, neither of which can be changed by any evidence.) "Hopeful" or "optimistic" programming might be better.
- Someone 12y ago
  Call it "trust", and it looks different. Do I trust my OS to have a memory allocator that works and to protect processes from reading and writing each other's memory? Yes, but not totally. Do I trust its more modern and complex APIs? Less so. Do I trust code I copied from stackoverflow that has 100+ points? Yes, but even less so than I trust the complex or new corners of my OS. So, I read every line I copy. On the other hand, I am lazy, so if this gets to 1000+ lines, chances are I won't read every line. I will think a bit more about the trust issue, though. Do I trust the documentation of the libraries shipping with my OS? More than stack overflow answers with 20-ish points, but certainly not completely. Do I make assumptions about APIs? I try not to, but it is hard. There is lots of documentation where it is hard or impossible to find out what the code claims to do in edge conditions. For example, in https://msdn.microsoft.com/en-us/library/352y4sff(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/352y4sff(v=vs.110)...., can one pass a null transaction? I wouldn't know, and the page doesn't tell me, so this afternoon, I wrote a conditional operator calling another constructor if transaction is null. I did consider a "let's try it and assume it always works if it works once" approach, though. I also think many people do that ('if it compiles and runs, it must be valid C')
baddox 12y ago
Yeah, I don't really get what definition of "superstition" the author is intending here. It seems like "laziness" or "lack of rigor" would be more appropriate terms.

ggchappell 12y ago

Maybe a better term than "superstition" might be "validated experience". We do what has worked in the past. It is true that there are standards that should be read and followed; the ones referred to in the article are probably among these. But other standards have a fair amount of wishful thinking in them. (How many of you JS developers have read the latest ECMAScript standard?) Sometimes, what has worked in the past is a better guide for practice than what ought to work.

frandroid 12y ago

The problem here is that if it works, how is it superstition? When I knock on wood or throw salt over my shoulder, I don't have a way of verifying that I've averted disaster. However, I can test the code I don't understand, and if it does what it says it does (or does what I want it to do, since that's why I've selected it in the first place), then we're a step removed from superstition. There might be unintended consequences if I truly don't understand the code, but that is something else than superstition.

rbanffy 12y ago
If you do not understand the code, you cannot say you tested it. It may have corner cases you did not anticipate. The test may be incomplete and only cover what you think the program should do.
- tjradcliffe 12y ago
  Nonsense. We test stuff all the time we don't understand "completely" (which is some kind of magical standard, as we obviously don't understand the universe completely, so the very atoms and molecules that the machine is made of are in some sense not well understood.) Certainty is the alchemist's stone of philosophy. People have done all kinds of interesting stuff while attempting the impossible: turning knowledge into certainty. They can't, but that doesn't mean knowledge--which is inherently uncertain--is inferior. It means certainty is inferior, because it can't actually be achieved, regardless of what any pre-Bayesian thinker might have imagined. Knowledge is always uncertain. Tests are always incomplete. This does not mean "knowledge is impossible" or "you cannot say you tested the code" but rather "certainty is a chimera" and "compete test coverage is a futile goal." The whole point if good engineering is to adopt standards that are both achievable and useful, and the kind of test coverage that people run on code they are unfamiliar with to ensure it will do what they want it to is generally adequate for that. Evidence: most shipping code actually works.
  - sheepmullet 12y ago
    "Evidence: most shipping code actually works." The vast majority of time spent on software development is on bug fixing and on many projects the construction/coding defects make up more than half of all the defects.

rjbwork 12y ago

I think a better title might be "A vast majority of programming is done at a level of abstraction that this author is slightly uncomfortable with". If you work in Java or Ruby or Python or C#, or even C++ all day with say, an IoC container for argument's sake, you're certainly not constantly thinking "okay this interface will be resolved to this type, which has these dependencies which will be resovled to these types, which generate this byte code, which generates this JITed assembly code, which generates this machine code, which generates these executions of instructions on the processor and accesses these memories, which invokes these manipulations of these gates and flip flops on this particular part of the ALU". Doing so would be the mark of an unproductive madman.

ggchappell 12y ago
That's an interesting way of putting it. The issue is that we all have our own (more or less completely undocumented) abstractions that we mentally put on top of whatever language/library we're using. And sometimes these mental abstractions are out of sync with reality.
dandelany 12y ago
I don't think this is about abstraction. Abstraction implies having a deep understanding of the problem domain and encapsulating it in such a way that the implementation details can be ignored and the domain can be reasoned about at a higher level. I think a better title would be "A significant amount of (shipped, so-called production-quality) code is not fully understood by its authors".
- rjbwork 12y ago
  Perhaps. One could argue that one of these init file templates is just a base class with parts of the implementation overridden. :D
- tjradcliffe 12y ago
  So an even better alternative title would be "The Author of This Essay Does Not Understand Engineering, Software, Technology, Medicine, Or Any of the Other Practical Arts Which Have Always Involved Shipping Imperfectly Understood Machines".

anon4 12y ago

I'll have to admit something about myself -- I've never been able to learn how to do anything from just reading its first principles. I need to have a working example that I can then adapt and change and observe changes in the result. This has been especially true in programming. I've been writing programs in some capacity for 18 years now and not once in that time have I been able to simply read the reference manual of anything and following that write working code on my first try. Well, obviously I've been able to do it for trivial things, like after the Python 2->3 switch when map/filter/range started returning iterables instead of lists, simply being told "map/filter/range/etc. now return an iterable and not a list" is sufficient to explain the change in behaviour, but that's only because I know all the concepts behind it. You could say the explanation is only one level removed from my intuitive knowledge. Things start getting exponentially harder the more levels removed they get, i.e. if it's a new concept explained using new concepts previously defined, but for which I have no intuitive knowledge what they mean.

GauntletWizard 12y ago
Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'. Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key to science. You have to observe, change, guess, run, observe, change, observe until you can successfully predict what will happen for each change you make. The scientific method is taught to everyone, even if most ignore it and skip steps. 'Guess/check' is the correct way to do real-world engineering and design, not pure-logic simulation, because we still don't know all the first principles.
- deleted 12y ago
  [deleted]
- avmich 12y ago
  > Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However, both experimenting and reasoning can be flawed, and part of the trick is to learn how to do that properly. Computer science is difficult.

depsypher 12y ago

A nice example of this is yoda expressions in, say Java. The initial reasoning for them was to prevent accidental assignment, since a C/C++ compiler will complain on this: if (null = x) but not on this: if (x = null) In Java, neither is allowed, so the whole yoda construct is almost pointless. The exception is boolean assignment which does benefit from this idiom, but the loss in readability is a trade off to consider. In any case, I suspect most uses of this construct in languages like Java fall into the "superstition" category, where the people using it aren't really considering why.

Peaker 12y ago
A lot of us find Yoda conditions easier to read: if(0 == very_long_function_name_here(param1, param2)) { Tells me a lot after I've read a little of the line, whereas: if(very_long_function_name_here(param1, param2) == 0) { Makes the == 0 part easy to miss, and buried behind a lot of clutter.
- mrspeaker 12y ago
  eh, you just end up having to parse if (VERY_LONG_AND_SPECIFIC_ENUM_NAME_HERE == func(param1)) { } Makes the function called easy to miss and buried behind a lot of clutter ;)
  - Peaker 12y ago
    That's why my rule is shorter first :-)
- depsypher 12y ago
  Re: "buried behind a lot of clutter" If your code has a lot of clutter it's usually telling you something about your design. For readability a better way of handling it might be for the function could return a boolean instead of an int: if (is_valid(param1, param2)) { self documenting code and all that.
  - Peaker 12y ago

nightski 12y ago

I am sorry, the only reason I have been as successful as I have been is because I am not superstitious at all. I know that if something goes wrong, there is always a logical explanation for it. The quicker I can find that logical explanation the better I am doing. I don't think I am alone in this.

lectrick 12y ago
> The quicker I can find that logical explanation The quicker your test suite can find that logical explanation... ;)
agumonkey 12y ago
I often wish to have investigation classes. How to systematically reduce the search space in order to find meaning in explaining a phenomenon, good or bad.
nullc 12y ago
Is the reward for getting a comment posted so quickly so great that you don't bother reading the article even when the price of making yourself look like a fool?
- nightski 12y ago
  I did read it, there were quite a few comments before I even posted. What he described was not superstition.
  - nullc 12y ago
    Indeed, what described was not superstition; there doesn't appear to be an exact word for what he was talking about (cargo cult is perhaps closer but not quite right), so it was a fuzzy match. What you responded to, however, was superstition and as a result your comment had basically nothing to do with the article. I hope you can forgive my misunderstanding. :)

johnny99 12y ago

It's true. All of my programs go from line 665 to 667. I've never made a commit on Friday the 13th, ever--and the one time I did it caused irretrievable data loss. Once I walked under a ladder, and it broke the build.

throwawayaway 12y ago

as a concerned cancerian, i am glad somebody is willing to admit it. i leave my sub conscious do the heavy lifting. "this is the general shape". making a list before starting is probably the extent of my preparation. if i can copy what's gone before without understanding it, all the better. only when i get stuck do i engage critical thinking. maybe this has been to my detriment. very often i notice myself and colleagues making up nonsense explanations for why c++ behaves the way it does without any real basis other than intuition and truthiness.

venomsnake 12y ago

Since I began programming I don't believe in miracles. I count on them.

dsugarman 12y ago

even with code I write myself, I can't remember the line by line implementation on the entire code base. To be honest I often forget implementation details on code I wrote last week.

eric_h 12y ago
Ditto. When a coworker asks me about implementation details of some code a wrote just a few weeks ago, my first answer is always "I haven't a fucking clue". Then I scan the code in question and it all comes rushing back.
- eric_h 12y ago
  I should add that the extent to "it all comes rushing back" is directly and inversely correlated with how long ago I wrote it, and positively correlated with how interesting/difficult the problem was to solve at the time. Also, almost every time I go back to review code I wrote more than a year ago, my first reaction is "jesus, that's a stupid pattern/implementation. Why did I do it that way?". I've been a professional developer for about 12 years now and I suspect that will never change.
  - quicklyfrozen 12y ago
    It won't. And sometimes you'll start to change it, hit a wall, and then realize "oh that's why I did it that way."

mkramlich 12y ago

I think when I was very young and new to programming I sometimes did coding via superstition. But it was a long time ago, in say my first year or two, when I didn't yet have a good mental model of how the computer worked and how the language compiler or interpreters worked. Once you become solid at that I've found that 99.9999% of the time that computing is refeshingly rational and deterministic and amenable to logic.

benihana 12y ago

I realized that the entire piece is hinged upon this incorrect assumption: >This isn't something that we like to think about as programmers, because we'd really rather believe that we're always working from scratch and only writing the completely correct stuff that really has to be there; This is the issue I have with this article - the author assumes everyone works in exactly the same way they do and so are vulnerable in exactly the same way to their vulnerabilities. I don't know anyone who actually approaches programming in such a way that they think they're writing things from scratch and it's correct. I mean, defensive coding is a concept that has existed for decades that specifically addresses how incorrect the code we interact with is. The fact that I started coding in an environment where I had to manage memory myself and now I don't makes me very aware of the fact that everything I'm using is an abstraction built upon other abstractions. I dunno, the day I learned to program in Intro to C and Intro to Computer Science, defensive coding, not trusting user input, and the completely understandable at the time weirdness of the guy who came before you have been stressed as things to pay attention to.

Retra 12y ago
Usually when I'm writing code it is "Let's get the infrastructure up and come back to clean up after I know it works..." Sometimes I never come back. It doesn't mean I think the code is correct, only that it is not the biggest problem.

dblohm7 12y ago

This explains why so many websites don't validate email addresses properly.

nulltype 12y ago
How do you validate email addresses properly?

santosha 12y ago

I've always enjoyed what I call uncoding, which is taking a working piece of code and taking stuff away until things break, and examining how they break. It's really helped me understand what each piece does, and be more efficient when I'm writing it myself.

eric_h 12y ago
Amusingly, that's also more or less how we've learned to understand the various functions of the different parts of the human brain. We don't know what it does until it's been removed/damaged.
forgueam 12y ago
I love that term. You described exactly how I work: take an existing block of code, break it apart until I fully understand every line, then reassemble to meet my needs.

copsarebastards 12y ago

I'm surprised by the number of people who don't see this as a problem. I've worked pretty recently as the most senior developer on a team, and I saw tons of issues caused by copy/pasted code. It results in a lot of issues. Subtle bugs around `==` versus `===` in JavaScript, unecessary variables that obfuscate the code, Python that isn't Pythonic, C that was written as an example and therefore doesn't include the error handling that is constantly necessary in C (how many C examples check the return value of malloc?). The biggest problem, I think, is that often you end up with 10 different ways to solve the same problem in the code because instead of looking at the existing code, people Googled for a solution with subtly different keywords than the previous person and found a different blogger. This kind of duplication is very hard to identify and remove, and this does cause bugs. Of course a more experienced developer can work around these issues, but a more experienced developer also knows off the top of their head how to write the code themselves and usually does it.

bentcorner 12y ago
> The biggest problem, I think, is that often you end up with 10 different ways to solve the same problem in the code because instead of looking at the existing code, people Googled for a solution with subtly different keywords than the previous person and found a different blogger. This kind of duplication is very hard to identify and remove, and this does cause bugs. It's interesting to code in large code bases. They end up getting to the point where no single person can ever understand the entire thing. It is usually at this point that things like Hungarian and strict coding guidelines really help. If you need an array of pointers to some kind of object, you can be reasonable certain that search will be able to find it.
- woah 12y ago
  IMO if your code base is large, you have already failed.
  - copsarebastards 12y ago
    If your code base is large, you've failed as a coder. If your code base never gets large, you've failed as a company.

Animats 12y ago

I've referred to this for years as "ritual-taboo programming". Ritual-taboo societies are ones in which things are done in certain ways because they worked in the past, but no one knows why. Too much of programming is like that. If you work from the specifications for a library, you'll probably find that some of the documented features don't work. If they weren't used by some "framework", or mentioned in a how-to book, they probably haven't been exercised well.

danschumann 12y ago

Such as, people are taught not to use global variables, but maybe they weren't taught the reasons why not. The reasons (some of) would be polluting the global namespace, having unexpected things defined in unexpected places, and so on. At some point, it could save the programmer oodles of time to just use global variables. It could be a project with limited scope where polluting and things won't be an issue. The reason for superstitions at all are stopping people from doing something when they can't understand the reasons yet. It's hard to explain to a child all of the reasons why they should be a good person, and much easier to tell them Santa will give them presents, until they can understand why being good is inherently its own reward.

thoughtsimple 12y ago

I've worked with developers who seem to code mostly by going to stack overflow, finding something that seems to fit their problem and then they copy code into the app. No attempt to find a more recent version or a more correct solution of the problem is made. I've seen JavaScript code copied from a stack overflow answer that was dated 2009. There is nothing better than a 6 year old answer to a common JavaScript problem.

mwsherman 12y ago

A significant amount of programming is done without articulating either the problem or the solution.

gee_totes 12y ago

What would be an example of good programming primary sources?

jrochkind1 12y ago

What that means is that if you're writing the system you want someone to do 'right' -- you better provide lots of good examples in the docs. A 'good' example is one that you've actually tested for real to make sure it works, that follows the intended principles for the system in question, that meets actual use cases developers using it will have, etc. A significant amount of software is released without good docs -- examples being just one part of good docs. So of course developers copy from the examples they can find.

rilita 12y ago

Article: Reinvent the wheel, because if you just mount whatever wheels fit the axle they will inevitably work poorly for your purposes. Wheels work just fine. Choose the best wheel from those available and tweak it as desired. Test it to make sure it doesn't shatter under too much weight. Apply and move on. This pervading notion of "if you don't understand you shouldn't do it" is silly. Sure it is nice to understand, but it doesn't put food on the table. Sometimes it's just handy to use the gun as is and go shoot your dinner in the woods. You don't need to understand the formulation of gunpowder or rifling in order to use a gun properly for it's intended use.

systematical 12y ago

Superstition...right I don't shave until a project is done and I have lucky underwear that I program in...

ff_ 12y ago

While the article is dramatically true (credits to StackOverflow for some of my production code), there's a factor that isn't taken into account here: time. Yes, because as engineers our time is more valuable than a minimal and perfectly-standard-compliant solution. I would happily read all the RFCs, and all the significant papers in CS, and all...etc. The reality is that we don't get paid to read, but to ship.

mpdehaan2 12y ago

A very sigifigant part of Ansible service module code was done to deal with misbehaving init scripts. They are very much cargo-culted just like RPM or debian package formats - borrow the ones from your previous project. They are hard to get right and many apps don't daemonize correctly or return OK before they are ready to be running - and then are actually running much later. Upstart/systemd didn't neccessarily make that more clear for people to understand either. While I haven't tried it, I recently encountered a reference to https://github.com/jordansissel/pleaserun https://github.com/jordansissel/pleaserun and it sounds promising. I don't know how much of this applies to programming in general, really, but init scripts are.. yes... special things.

GauntletWizard 12y ago
I dislike tools that claim to automatically generate your configuration for you. At Google, one of the worst abuses that anyone had to deal with was auto-generated configuration; a tool purported to deal with configuration for you, but really just blutrted a boilerplate mess into your directory, many of the properties and setting set by cargo-cult or overriding sane defaults with baked-in versions of the same. Facebook is proceeding down this path; There's a new tool that will do all of your service stuff for you! I look forward to what a boondoggle those services that use it will be. There was a discussion I recently stumbled upon about init systems in containers. Many processes spawned lots of zombies! You need a full init! Well, no; If your process is forking new processes, it should probably wait for them. If it's forking other processes that fork other processes, and they're not cleaning up zombies when they're done, that is a bug in those processes. If all else fails, your root-level process should probably implement a simple init on it's own (a wait() on loop in the background, dumping wait objects into a LRU map that other processes can do an equivalent of waitpid() on). Or just run under bash.
stonith 12y ago
I've used pleaserun a little in the past. When you're being quick and dirty and using fpm it fits into the workflow quite nicely. It can be a bit awkward if you're deploying things that need anything complex in their init script, like a bootstrap command or something, but if it's just a straight start|stop|restart it works quite well. It's probably not up to generating distro-guideline-compliant init scripts (and fpm obviously doesn't either for spec file) but so long as that's not the goal it's a useful tool.

Sevzinn 12y ago

Check .gitignore to see if it appears copy and pasted or actually thought through--this will invariably indicate the quality of the rest of the code.

haberman 12y ago

I am someone who tends to work from first principles, to a fault. It slows me down a lot. But it's like a compulsion. StackOverflow has improved my programming life dramatically, because it makes it a lot easier to "memoize" the search from what you're doing down to first principles. Whatever question you're trying to get clarity on, there is probably a StackOverflow answer out there that builds a positive case for a particular option, going all the way back to primary documents like standards.

danbruc 12y ago

This does not resonate with me at all. I can not stand having things in my code I don't fully understand. Without reading the source code of libraries, without reading relevant standards, without understanding the mathematics behind algorithms, there is always this feeling I may make really dumb mistakes. Really understanding all the bits and pieces obviously does not make your code bug free, but I am usually pretty confident that I did not do something horribly wrong.

svisser 12y ago
But what does it mean to fully understand your own code? Everyone relies on millions of lines of code that they didn't write themselves.
- danbruc 12y ago
  For me that really depends on what I am dealing with. Most of the time the library source code is good enough. Sometimes - I am a C# developer - I will have to have a look at the .NET runtime specification. Sometimes I will have to learn details of Windows. In some cases I will even look into Intel's reference manuals to understand what is happening. I would say I stop at the level where I don't expect the layer below to be relevant to what I am actually dealing with.
ackalker 12y ago
This piques my interest. When was the last time anyone read the source code of (g)libc "cover to cover" before using it? Does anyone know the precise implications of (not) defining any of the myriad sensible permutations of its "feature test macros"? Is anyone even sure that those implications are actually implemented correctly before relying on them?
- danbruc 12y ago
  I obviously do not and can not trace the entire code path from a print statement through the class library, the operating system, the display driver and the hardware, some things I will still take for granted. But if there is a really critical aspect to what I am doing, say some thread synchronization, I will definitely not just follow some sample code, but read the relevant parts of the memory model of the language specification detailing why this works as expected. If the underlying hardware matters, I will grab the reference manual for it, too. A lot of this is surly curiosity and you can get away without it most of the time, but at least as soon as I run into the first problem it really helps me a lot to have at least a basic understanding of what is going on in the layer of abstraction I am interacting with and maybe a layer or two below. And it gives me a really comforting feeling, that I know what is going on better than describing it as some black magic.

perlgeek 12y ago

sysv init scripts in particular are so pointless to write. In the end, the whole script usually revolves around four lines of actual code (starting a process, stopping it, querying the status, optionally sending it a SIGHUP to make it reload its configuration), but the result is a script with 50 lines of boilerplate. Which, of course, could nearly all be automated away by a sane, declarative format. Leaving all the political mess aside, that's the part that systemd got right. You write sometthing like [Unit] Description=some human-readable text here [Service] Type=simple User=... ExecStart=/path/to/deamon --with=options KillSignal=SIGKILL [Install] WantedBy=multi-user.target and that's it. Quick comparison of sysv init script lengths on Debian Wheezy vs. systemd service files on Debian Jessie: ssh: 162 vs. 15 lines, cron: 92 vs 11 lines, dbus: 122 vs 9 lines My point being: it's a bit too much to demand that people carefully engineer their sysv init files, when about 90% of those very same init files is simply boring, repetitive boiler plate code. Also, what happened to the "don't repeat yourself" principle? I tried writing a sysv init script from scratch. I stopped because of sheer boredom, and resorted to copy&paste to save both time and nerves.

vezzy-fnord 12y ago
The issue isn't so much in the medium (shell), but the underlying program itself (sysvinit) being overly primitive. See the daemontools family where runscripts are largely as short or shorter than systemd unit files: http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/run-scripts-and-service-units-side-by-side.html http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/ru... In addition, the BSDs have much shorter rc scripts mainly by putting all the boilerplate functions into a single library file that is then sourced. Gentoo's OpenRC framework, as well.
- digi_owl 12y ago
  Franly i find myself thinking that moving the boilerplate elsewhere is also what systemd and upstart does. But those implement it in C rather than shell script.
amyjess 12y ago
Not just systemd, but... everything else that's trying to replace sysvinit, too. Upstart jobs are really simple, and OpenRC and modern BSD initscripts are also much simpler than SysV (and old BSD) initscripts. I think we're at the point where regardless of the systemd vs. everything else debate, we can all agree that SysV initscripts are shite. > I tried writing a sysv init script from scratch. I stopped because of sheer boredom, and resorted to copy&paste to save both time and nerves. Better yet, at my last job, I had some extra time, so I rewrote Ubuntu's qpidd initscript as an Upstart job, and it was so satisfying to see that giant script get condensed down to a few lines.

krick 12y ago

That's right, but I guess we're missing the real problem here. The real gold are these words: > You've really read the LSB specification for init scripts and your distribution's distro-specific documentation? If so, you're almost certainly a Debian Developer or the equivalent specialist for other distributions. Many of us would agree, that it's true. But why is it true? Shouldn't that make us suspicious in the first place? It's almost paradoxical, even funny: making the work done is easier, than finding out, how to do that work. You see, init scripts or makefiles aren't exactly rocket science. I mean, things we are willing to achieve with them are pretty simple: just start some process, just build some project (which means "execute bunch of shell commands in the right order", and not implementing out own superoptimizing compiler). So when we arrive to situation like one we've got today, the right message wouldn't be "come on guys, you see what we're doing because of being lazy? we should stop being so lazy!". That is, yeah, we all are lazy and maybe we should stop it, but that's not the point. What this situation actually means is that our tools for doing simple stuff are overcomplicated shit, and our documentation for these tools is shit. If anything, it's pretty obvious that a couple of lines of code, showing how to do the work is better documentation than complete list of possible flags and parameters with no examples whatsoever. So no wonder people are reading stackoverflow instead of man-pages. And so on.

throwawayaway 12y ago
> it's pretty obvious that a couple of lines of code, showing how to do the work is better documentation than complete list of possible flags and parameters with no examples whatsoever. So no wonder people are reading stackoverflow instead of man-pages. And so on. the best manpage is a man page with a nice EXAMPLES section you can grep to quickly. though the permutations offered by many programs prohibit all bases being covered.
- mgob 12y ago
  http://bropages.org http://bropages.org to the rescue!
  - throwawayaway 12y ago

everyone 12y ago

I have a tendency to not use things I dont feel that I fully understand. This makes me stick to using the more basic elements of the language. In C# for example recently, there was a part of a game I am writing in my spare time for which 'events' would have suited. But I still dont feel I understand what events in C# is doing exactly. So instead I just put all the things I want the event to happen to in a list, and when the event happens everything in the list has a particular function called on it. I will probably get around to using C# events eventually when I have looked at them more and feel like I understand them. There is maybe a very small performance cost to what I did (according to some stuff I read, but I havent tested this so I dont know) But at least I have an architecture for my game and code that makes sense to me and that I feel like I understand completely (at least at the abstraction level of C#)

Sophistifunk 12y ago

If init scripts are mostly boilerplate that people keep copy-pasting, maybe the init process is the problem, rather than the poor chumps who have to write the scripts?

nickysielicki 12y ago

I'm a proud cargo culter. Truth of the matter is you can't just stare at a man page or textbook all day and try to understand, then code. You'll code some mess you don't fully understand, sure, but later on you'll come back and either realize it was wrong and fix it, or you'll understand why it was right. But you will eventually understand. The people trying to understand it will have gotten nowhere in the mean time.

anupshinde 12y ago

The author sometimes confuses the work "superstition" with "abstraction". That is how Humans work everywhere. And that is why my idle PC cannot create a Facebook/Twitter/or even a plain website by itself.

martininmelb 12y ago

I think this is what I refer to as SODD - or 'Stack Overflow Driven Development'. Somebody is not sure how to do something, so they look for an example on SO and they copy the code without really understanding what they're doing. (note: this is different to when they get an answer on SO and copy it knowing what they're doing).

jnem 12y ago

I hate to call myself out, but I envy most of you. It sounds as if many here get the opportunity to work on interesting problems from end to end. As such, first principles matter, and approaching something from scratch makes sense. I however have been stuck in a segment of the market that doesn't appreciate the right solution, but rather just wants a solution as fast as possible. To be clear, I'm talking about a more consumer audience. My clients expect things done quick, and quick means leveraging cms, ecommerce platforms (magento, woo commerce, etc.). To be frank I find such work unsatisfying, and often need to hack things together to meet the requirements set forth within the constraints of the budgets I'm working with. I'd love to be able to work in an environment where doing things right mattered, but the plain fact is the deliverables aren't measurable based on quality, but expedition. While I'm on a soap box, I'd like to point out to those in the positions where they're work is evaluated by professionals and peers how lucky they are. I'd kill for a job/client where my code is evaluated by other developers. This community sometimes forgets they are the 1% of the development world. Those who work for the googles, facebooks, and visionary startups with leaders who came from the same cloth. The rest majority of us are laboring away under management that have never produced a lick of code, or even design, for that matter. I guess the Tl;DR is: appreciate the fact that you even get to consider the finer points, and pray for the rest of us.

yason 12y ago

In my view, a lot of this is just trying to avoid learning yet another system. Learning systems is hard and takes time, and it's only worth it if you're going to be using the system a lot in the future. For example, creating a new init script or a Debian package description is something I don't do all the time. I don't want to learn all that stuff because by doing so I'd basically push out something else from my mind. Copying an existing configuration or script and making modifications without trying to create something original is an excellent pattern for things outside your immediate expertise target. The "avoiding creating something original" part does indeed carry a sort of "superstition". It's like relying on a new culture by merely mimicking it, because you don't really know it well enough to break the rules correctly. You don't know why things are done the way they are but your best bet is just to copy and adapt. This creates a sphere of fuzzy knowledge where intents and black magic seem to alternate.

lmm 12y ago

This is specific to shell scripting and to the history of nix standardization. No-one follows or cares about the LSB[1]; certification is expensive and based on buggy test programs that practically force you to "code to the test" and not to the specification. And when so many implementations ignore the spec, it would be more superstitious to code to the magical spec and assume it's going to work, rather than writing something and testing it empirically. When I'm programming (in Python or Scala) I can and do check the language specification if something is confusing, and I don't think I'm alone in this. It may be a "significant" amount of programming, but there are definitely areas of programming with more rigour. [1] I blame Debian, usually quixotic in its adherence to standards like the FHS - but when it came to the LSB they prefer to pretend that "alien" counts as RPM support. That set the tone for how much other distros tend to care about the spec.

the_why_of_y 12y ago
IMHO "alien" isn't much of an issue since LSB packages are allowed to use only a small subset of RPM features; the fancier features aren't portable between different RPM-based distros either. http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/pkgscripts.html http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-gener... http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/pkgdepend.html http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-gener... A bigger problem is Debian gratuitously using different SONAME than required by LSB for some libraries - your LSB binary can't use those system libraries if it wants to run on Debian. But at least it looks like they fixed that problem in Wheezy.

cel1ne 12y ago

The answer to "Why we can't build software the way we build bridges?" is that bridges and other physical works disintegrate over time. Code doesn't rot, so the mistake of 1980 can be freshly reused 40 years later.