4 ms·

This guy gets it wrong in his last post on reliability (but the other posts seem reasonable). Reliability is hard because we continue to move the goalpost. I

This guy gets it wrong in his last post on reliability (but the other posts seem reasonable). Reliability is hard because we continue to move the goalpost. I can write a pretty reliable program to compute sin/cos tables or to sort a phonebook or to do some basic projectile trajectory. But those aren't even table stakes anymore. Today a first year undergrad student might be expected to write a program that in 1970 the most advanced programmer in the world couldn't do with a large team and millions in funding. I hate the analogy to other fields of engineering (and this author doesn't fall down that trap, but plenty do). Bridges haven't changed in the past 10 years the way programs have (in terms of functionality or complexity). And lastly, this is IMO, a corollary for why ageism is so persistent in our field. Experience means very little. Why? Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it. I don't actually need you. There are few other fields where as much of your experience can be captured in something so reusable (despite our constant protests that code isn't reusable -- it's more reusable than any output from my plumber). Programming is terrible because we obsolete ourselves by moving technology out of our own grasp. The fact that newer programmers don't understand the foundation they build on is irrelevant, since few need to look behind the current. Unfortunately, I think it is simply the nature of the beast. It's why we love it and do it -- and probably why we'll all eventually stop (or at least move into management).

jiggy2011 14y ago
That last point rings quite true with my experiences. I've been writing PHP and some other languages off and on for about 10 years now. Looking at current jobs listings it's difficult to find things that I am particularly well suited for, even in web programming. Since I was using PHP well before the current crop of mature frameworks and CMS the majority of my experience is in trial and error development of my own database access patterns , anti XSS , anti SQL injection and all of that other stuff that you used to do yourself in the early 2000s. As a result I have handwritten countless numbers of CMS systems and frameworks and am comfortable taking a bunch of hardware , building a Linux server out of it and mounting it in a rack. But nowadays the conversation is not so much about that stuff, it's "what do you know about EC2?" , "What do you know about codeigniter?" , "What do you know about Wordpress?". So, my knowledge has been abstracted (generally by people who would do a better job it than I did). It's quite a hard thing to explain to your family when they say "But you've been doing programming for years! You must know everything about it now!"
- digisth 14y ago
  This is the nature of modern technology work (especially software-related work.) The ability and willingness of people to continue to learn throughout their lives is going to become more important with each passing year, particularly as things get more competitive, and technology continues to evolve (and even more so if the speed of that evolution increases, as some predict.) I'd argue that said willingness is now one of the most important things for people to remain competitive, right up there with past experience. Related article and discussion from not too long ago: https://news.ycombinator.com/item?id=4557816 https://news.ycombinator.com/item?id=4557816 It's a new era of autodidacticism.
- eru 14y ago
  That's one of the reasons to invest in lasting knowledge. Compiler construction is one of the evergreen topics.
- Tarential 14y ago
  I had to log in just to reply to this because it very well could be me writing this post. I taught myself PHP in grade 7 (around fifteen years ago, it must have been near the v3 release) and sold websites in high school. I hand built everything on web servers I compiled from source. Then I made the terrible mistake of taking a long break from professional programming through university (I worked as a tech writer instead, the money was good). When I came back two years ago I found that I had to learn a whole new set of tools. Javascript was now the norm instead of a disabled annoyance. Browsers supported incredible display features using stylesheets which made my table-and-images based designs obsolete. All the sites were built using some sort of CMS. I spent a year learning front end design, WordPress, Magento, Joomla, and Drupal theme design. Then I went back to my old do-it-yourself ways and wrote my own CMS [0]. Suffice to say that it wasn't necessary. I learned a lot about back end programming and MVC related design patterns while creating it, but the most valuable piece of knowledge I gained was "Don't reinvent and try to sell the wheel when other people are offering it for free." It should have been obvious, but it seems I'm much smarter in hindsight than I was in planning. I'm still learning today. Now it's the Ruby on Rails world and a whole plethora of new technologies come into play. I'm using Haml, SCSS and CoffeeScript on top of the RoR / PostgreSQL back end. My first web app will be ready for release soon. After that it's on to learning Haskell and RethinkDB. The irony of all this is back when I stopped programming I thought I had a pretty good grasp on everything I needed to know. I figured I'd step right back into web development and continue on the way it had been when I left it. I wouldn't have let my skills atrophy if I had known how important it was to stay up to date. I think after two years I'm finally catching up, and I don't intend to let them get rusty again. TL;DR: Don't get cozy because you think you know everything now. Even if you do, the state of the art changes quickly. [0] http://saintcms.com http://saintcms.com
angersock 14y ago
It's kind of funny, in a way--if you write a library correctly, documenting the behavior, run time characteristics, failure modes, and everything else, you should never be needed to help with the damned thing. Our best work is that which can stand alone, without us, across time.
kabdib 14y ago
> Experience means very little Perhaps. But I believe that good engineers have a number of cool failures in their past. > Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it You can capture /some/ of someone's experience this way. But don't confused the ability to _use_ the novel Moby Dick with the ability to _write_ Moby Dick. These are two very different things. Let's talk about this hypothetical library for a minute. 1. It's a finished product. You didn't experience the design process. You didn't see the things that were tried and which failed. All you see is the shiny object. Could you make another like it? 2. Since you missed out on the design interaction, you're missing out on the stuff that was left out for the next version. You weren't there for the discussions about alternatives (maybe better ones). 3. If you treat the library as a black box, you're going to be at the author's mercy for bug fixes, making improvements, or doing integration in environments where it doesn't exactly fit. 4. Ultimately, abstractions are lies. The best ones are white lies, the worst ones paper-over or ignore fundamental problems. (My favorite example is putting a database on top of a Unix file system: How do you know when data is stable on disk? What /is/ a commit, anyway?) How much of that library do you really believe? I've seen some really interesting product failures resulting from "unthinking isolation" -- perhaps a better term would be "false abstraction" -- of what was happening in the system at the hardware level. We're talking bricks and user data loss because someone thought that transactions weren't important. Performance is another area where it's hard to abstract (and some platforms, such as video game consoles, are all about performance). Good, reliable systems should be designed with all layers in mind. That's where experience starts to matter.
- chipsy 14y ago
  To add to the "layers" aspect, the young hotshot coders of the world are also some of the worst at making marketable products, because they've become used to making an 80% prototype and then moving on to the next thing, which means that they haven't experienced this yet: http://www.gocomics.com/calvinandhobbes/1986/11/26 http://www.gocomics.com/calvinandhobbes/1986/11/26 Learning software "in the large" still basically requires you to drive the bigger and bigger trucks over(more features) until the bridge(your architecture) breaks. It can't be properly conveyed with any weekend project or library.
deleted 14y ago
[deleted]
kstenerud 14y ago
"The fact that newer programmers don't understand the foundation they build on is irrelevant, since few need to look behind the current." Every time I think along these lines something comes along that pushes us closer to the metal again. 10 years ago I thought that the usefulness of my knowledge of assembly language had reached its end. After all, what use was it when I was writing enterprise applications in Java! And then our server apps started crashing (the whole JVM would core dump). So there I was, looking at a memory dump, x86 opcode table in hand as I tracked down what turned out to be a null pointer access in a native library from one of our vendors. And then came my stint writing software for wireless credit card/debit card systems. On a Z80. In 2001. Then came GBA development, counting every megabyte, worrying about image sizes and trying to make things look decent in a reduced palette and the eternally-too-small RAM. And now, here I am in 2013, counting the megabytes once again, fighting against eternally-too-small RAM (especially on iPad 1), trying to keep my iOS apps small enough, using the same old smoke & mirrors and dirty tricks to give the illusion that the app is doing what it actually isn't. I'm intercepting mach kernel messages. I'm poking around memory structures. And, of course, I once again find myself disassembling code from memory dumps, opcode table in hand. In 2013. I'm tempted to say that this will be the last wave of constrained systems, but I'd probably be wrong.
- kabdib 14y ago
  There are many classes of products that require what I like to call "fractal attention to detail." These include most consumer goods (which are usually cost and performance constrained), man-rated stuff that has to work (aerospace, medical), and products with stringent security requirements (game consoles, smart cards). By fractal I mean that you can have good abstractions, but you can't afford black boxes, and design has to happen at a whole-system level or you're leaving something on the table. It's surprising how often decisions at very low levels affect the high level user experience. Any time cost, reliability or security are on the line, there will be very few pieces of your product that are mysterious to you.
  - nnq 14y ago
    Why can't you have black box abstractions in things that "have to work"?? And even "has to work" can be relative, as if you can prove that something will work for a p value small enough to guarantee risk of failure is small enough in the next 100 years for a device/system that will at most stay in use for 10 it good enough... And if you're not constrained by power use or computing power, you can also use redundant subsystems: I imagine something like a satellite, receiving high radiation that consistently corrupts cache or RAM but runt 5 identical OSs on five identical subsystems and on every "high stakes" decision it does a "voting session", choosing the result of a function with the most out of 5 votes and resetting the state of the disagreeing systems with the winning ones, and a similar type of reliabilty by duplication for medical devices or aircraft systems... And going further, you can always replace an expensive reliable satellite with a swarm of cheaper ones that can fail. I think we'll choose the "swarm way" for more and more things and this will relax reliability requirements for individual components and at the whole swarm level we'll have "good enough" statistical reliability instead. And any "swarm" can conceptually be seen as a black box, once you accept its probabilistic nature and the low but measurable likelihood of it being wrong, and maybe using a "swarm of swarms" if one swarm's p value of being true is not good enough for your application. The "fractal attention to detail" seems needed when you need both reliability and you have serious power or computing resources constraints. And since power usage per flops is always decreasing, the number of such "dual constraint" cases should get smaller... EDIT: typos and + pre-last paragraph to clarify what I meant
eikenberry 14y ago
> And lastly, this is IMO, a corollary for why ageism is so persistent in our field. Out of curiosity, are you from the valley area? It has been my experience that ageism is really only that big a deal there. Everywhere else I've worked (PDX, ATL mostly) I have seen no sign of ageism. Also it fits with the valley's general culture and economic setup. Everyone I know in the valley is young and single and live with other young and single people. As soon as they get older, meet someone and think about a family they leave and go somewhere that they can get a house and situate themselves.
Confusion 14y ago
Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it. If you use my library, you have not captured my experience to write said library. If you need to write any higher level library of your own, the most problems you will face are exactly the ones I already solved once. I could write that library faster and more correctly than you. (the words 'you' and 'I' do not (necessarily) denote you and me)
fauigerzigerk 14y ago
>Experience means very little. Why? Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it. If you depend on my experience, it's going to make me money one way or another. And it's going to be money that you don't make, either because you have to share your revenue with me, or because it doesn't give you an edge as many others will use my library to compete with you. But the problem is that age doesn't automatically give me the kind of experience that others depend on. There's a lot of knowledge that becomes utterly useless or even misleading over time. And you can be quite sure of one thing. If your experience comes from gluing together a bunch of libraries to chase the fashions of the day, your experience will indeed mean very little down the road.

jiggy2011 14y ago

That last point rings quite true with my experiences. I've been writing PHP and some other languages off and on for about 10 years now. Looking at current jobs listings it's difficult to find things that I am particularly well suited for, even in web programming. Since I was using PHP well before the current crop of mature frameworks and CMS the majority of my experience is in trial and error development of my own database access patterns , anti XSS , anti SQL injection and all of that other stuff that you used to do yourself in the early 2000s. As a result I have handwritten countless numbers of CMS systems and frameworks and am comfortable taking a bunch of hardware , building a Linux server out of it and mounting it in a rack. But nowadays the conversation is not so much about that stuff, it's "what do you know about EC2?" , "What do you know about codeigniter?" , "What do you know about Wordpress?". So, my knowledge has been abstracted (generally by people who would do a better job it than I did). It's quite a hard thing to explain to your family when they say "But you've been doing programming for years! You must know everything about it now!"

digisth 14y ago
This is the nature of modern technology work (especially software-related work.) The ability and willingness of people to continue to learn throughout their lives is going to become more important with each passing year, particularly as things get more competitive, and technology continues to evolve (and even more so if the speed of that evolution increases, as some predict.) I'd argue that said willingness is now one of the most important things for people to remain competitive, right up there with past experience. Related article and discussion from not too long ago: https://news.ycombinator.com/item?id=4557816 https://news.ycombinator.com/item?id=4557816 It's a new era of autodidacticism.
eru 14y ago
That's one of the reasons to invest in lasting knowledge. Compiler construction is one of the evergreen topics.
Tarential 14y ago
I had to log in just to reply to this because it very well could be me writing this post. I taught myself PHP in grade 7 (around fifteen years ago, it must have been near the v3 release) and sold websites in high school. I hand built everything on web servers I compiled from source. Then I made the terrible mistake of taking a long break from professional programming through university (I worked as a tech writer instead, the money was good). When I came back two years ago I found that I had to learn a whole new set of tools. Javascript was now the norm instead of a disabled annoyance. Browsers supported incredible display features using stylesheets which made my table-and-images based designs obsolete. All the sites were built using some sort of CMS. I spent a year learning front end design, WordPress, Magento, Joomla, and Drupal theme design. Then I went back to my old do-it-yourself ways and wrote my own CMS [0]. Suffice to say that it wasn't necessary. I learned a lot about back end programming and MVC related design patterns while creating it, but the most valuable piece of knowledge I gained was "Don't reinvent and try to sell the wheel when other people are offering it for free." It should have been obvious, but it seems I'm much smarter in hindsight than I was in planning. I'm still learning today. Now it's the Ruby on Rails world and a whole plethora of new technologies come into play. I'm using Haml, SCSS and CoffeeScript on top of the RoR / PostgreSQL back end. My first web app will be ready for release soon. After that it's on to learning Haskell and RethinkDB. The irony of all this is back when I stopped programming I thought I had a pretty good grasp on everything I needed to know. I figured I'd step right back into web development and continue on the way it had been when I left it. I wouldn't have let my skills atrophy if I had known how important it was to stay up to date. I think after two years I'm finally catching up, and I don't intend to let them get rusty again. TL;DR: Don't get cozy because you think you know everything now. Even if you do, the state of the art changes quickly. [0] http://saintcms.com http://saintcms.com

angersock 14y ago

It's kind of funny, in a way--if you write a library correctly, documenting the behavior, run time characteristics, failure modes, and everything else, you should never be needed to help with the damned thing. Our best work is that which can stand alone, without us, across time.

kabdib 14y ago

> Experience means very little Perhaps. But I believe that good engineers have a number of cool failures in their past. > Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it You can capture /some/ of someone's experience this way. But don't confused the ability to _use_ the novel Moby Dick with the ability to _write_ Moby Dick. These are two very different things. Let's talk about this hypothetical library for a minute. 1. It's a finished product. You didn't experience the design process. You didn't see the things that were tried and which failed. All you see is the shiny object. Could you make another like it? 2. Since you missed out on the design interaction, you're missing out on the stuff that was left out for the next version. You weren't there for the discussions about alternatives (maybe better ones). 3. If you treat the library as a black box, you're going to be at the author's mercy for bug fixes, making improvements, or doing integration in environments where it doesn't exactly fit. 4. Ultimately, abstractions are lies. The best ones are white lies, the worst ones paper-over or ignore fundamental problems. (My favorite example is putting a database on top of a Unix file system: How do you know when data is stable on disk? What /is/ a commit, anyway?) How much of that library do you really believe? I've seen some really interesting product failures resulting from "unthinking isolation" -- perhaps a better term would be "false abstraction" -- of what was happening in the system at the hardware level. We're talking bricks and user data loss because someone thought that transactions weren't important. Performance is another area where it's hard to abstract (and some platforms, such as video game consoles, are all about performance). Good, reliable systems should be designed with all layers in mind. That's where experience starts to matter.

chipsy 14y ago
To add to the "layers" aspect, the young hotshot coders of the world are also some of the worst at making marketable products, because they've become used to making an 80% prototype and then moving on to the next thing, which means that they haven't experienced this yet: http://www.gocomics.com/calvinandhobbes/1986/11/26 http://www.gocomics.com/calvinandhobbes/1986/11/26 Learning software "in the large" still basically requires you to drive the bigger and bigger trucks over(more features) until the bridge(your architecture) breaks. It can't be properly conveyed with any weekend project or library.

deleted 14y ago

[deleted]

kstenerud 14y ago

"The fact that newer programmers don't understand the foundation they build on is irrelevant, since few need to look behind the current." Every time I think along these lines something comes along that pushes us closer to the metal again. 10 years ago I thought that the usefulness of my knowledge of assembly language had reached its end. After all, what use was it when I was writing enterprise applications in Java! And then our server apps started crashing (the whole JVM would core dump). So there I was, looking at a memory dump, x86 opcode table in hand as I tracked down what turned out to be a null pointer access in a native library from one of our vendors. And then came my stint writing software for wireless credit card/debit card systems. On a Z80. In 2001. Then came GBA development, counting every megabyte, worrying about image sizes and trying to make things look decent in a reduced palette and the eternally-too-small RAM. And now, here I am in 2013, counting the megabytes once again, fighting against eternally-too-small RAM (especially on iPad 1), trying to keep my iOS apps small enough, using the same old smoke & mirrors and dirty tricks to give the illusion that the app is doing what it actually isn't. I'm intercepting mach kernel messages. I'm poking around memory structures. And, of course, I once again find myself disassembling code from memory dumps, opcode table in hand. In 2013. I'm tempted to say that this will be the last wave of constrained systems, but I'd probably be wrong.

kabdib 14y ago
There are many classes of products that require what I like to call "fractal attention to detail." These include most consumer goods (which are usually cost and performance constrained), man-rated stuff that has to work (aerospace, medical), and products with stringent security requirements (game consoles, smart cards). By fractal I mean that you can have good abstractions, but you can't afford black boxes, and design has to happen at a whole-system level or you're leaving something on the table. It's surprising how often decisions at very low levels affect the high level user experience. Any time cost, reliability or security are on the line, there will be very few pieces of your product that are mysterious to you.
- nnq 14y ago
  Why can't you have black box abstractions in things that "have to work"?? And even "has to work" can be relative, as if you can prove that something will work for a p value small enough to guarantee risk of failure is small enough in the next 100 years for a device/system that will at most stay in use for 10 it good enough... And if you're not constrained by power use or computing power, you can also use redundant subsystems: I imagine something like a satellite, receiving high radiation that consistently corrupts cache or RAM but runt 5 identical OSs on five identical subsystems and on every "high stakes" decision it does a "voting session", choosing the result of a function with the most out of 5 votes and resetting the state of the disagreeing systems with the winning ones, and a similar type of reliabilty by duplication for medical devices or aircraft systems... And going further, you can always replace an expensive reliable satellite with a swarm of cheaper ones that can fail. I think we'll choose the "swarm way" for more and more things and this will relax reliability requirements for individual components and at the whole swarm level we'll have "good enough" statistical reliability instead. And any "swarm" can conceptually be seen as a black box, once you accept its probabilistic nature and the low but measurable likelihood of it being wrong, and maybe using a "swarm of swarms" if one swarm's p value of being true is not good enough for your application. The "fractal attention to detail" seems needed when you need both reliability and you have serious power or computing resources constraints. And since power usage per flops is always decreasing, the number of such "dual constraint" cases should get smaller... EDIT: typos and + pre-last paragraph to clarify what I meant

eikenberry 14y ago

> And lastly, this is IMO, a corollary for why ageism is so persistent in our field. Out of curiosity, are you from the valley area? It has been my experience that ageism is really only that big a deal there. Everywhere else I've worked (PDX, ATL mostly) I have seen no sign of ageism. Also it fits with the valley's general culture and economic setup. Everyone I know in the valley is young and single and live with other young and single people. As soon as they get older, meet someone and think about a family they leave and go somewhere that they can get a house and situate themselves.

Confusion 14y ago

Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it. If you use my library, you have not captured my experience to write said library. If you need to write any higher level library of your own, the most problems you will face are exactly the ones I already solved once. I could write that library faster and more correctly than you. (the words 'you' and 'I' do not (necessarily) denote you and me)

fauigerzigerk 14y ago

>Experience means very little. Why? Because I can capture 70% of your experience in your code -- in your library -- and build directly on top of it. If you depend on my experience, it's going to make me money one way or another. And it's going to be money that you don't make, either because you have to share your revenue with me, or because it doesn't give you an edge as many others will use my library to compete with you. But the problem is that age doesn't automatically give me the kind of experience that others depend on. There's a lot of knowledge that becomes utterly useless or even misleading over time. And you can be quite sure of one thing. If your experience comes from gluing together a bunch of libraries to chase the fashions of the day, your experience will indeed mean very little down the road.