14 ms·
You can see the Pacific Ocean on one side, and you can see the Atlantic on another. You know of something called a shovel, machetes, labourers and engineers. A
by lordnacho 10y ago
You can see the Pacific Ocean on one side, and you can see the Atlantic on another.
You know of something called a shovel, machetes, labourers and engineers. And you have a budget.
Of course, it must be possible to dig a trench connecting the two oceans.
When you start, you find your men get ill from the climate. And perhaps you don't quite know how to build a reliable lock, but you figure the engineers will find out somehow. And meanwhile there are constraints coming in from government.
Things take long because it's easy to see the big picture, and getting more detail (ie learning) normally means discovering that you need a bit more time to do fix some issue that you didn't see standing on the hill.
- jacquesm 10y agoExcept that in software 1000's of groups have already dug 1000's of canals and yet the next group that will build a canal will insist on doing it their way. If our profession would only be able to simply collect lessons learned and to pass those on to the next generation then we'd be moving faster on average and we'd produce better software. we don't need 500 crappy, slightly different ways of doing the same thing, we need maybe 5, battle tested and hardened bullet proof solutions that are well engineered. Fragmentation is going to be open source's death at some point if this continues, that's one area where closed source actually has an edge: focus. In part the problem is that software producers have managed to convince the buyers of their goods and services that they are never ever liable for their product (something no other industry has managed to do), in part it is because it seems superficially easy to re-invent until you reach the same level of complexity (which inevitably happens) as what those 'archaic' solutions were already addressing. I don't have any answers on how to solve this, unfortunately, but there are software eco-systems that really get this right and others that seem to be totally lost in ever expanding cycles of endless rewrites.
- meira 10y agoA man that never digged a canal don't really master it until he digs some. Software craftmanship isn't something you learn by osmosis.
- jacquesm 10y agoSoftware craftsmanship isn't something you learn by ignoring the state of the art either. And as for digging canals, the civil engineering profession is lightyears ahead of the software people because there are lives on the line and they do have a way to pass knowledge on in a way that lessons stay learned. (Most of the time, anyway, forget about the dark ages for a moment please.)
- croon 10y agoThat seems like a really unfair comparison. I'd argue that the software industry has evolved magnitudes more in the last 20 years than what you call civil engineering has evolved in the last 100. Sure, the latter has existed much much longer, but it's by no means evolving very quickly. Software development has traded safety for rapid evolution in areas where possible. This could be argued for or against, but I'd rather have a multiple generations more mature software industry at a point I'd need to develop "safe" software, than a safely evolved industry several generations behind.
- jacquesm 10y ago> I'd rather have a multiple generations more mature software industry at a point I'd need to develop "safe" software, than a safely evolved industry several generations behind. Yes, that's exactly the problem. So now we have an unsafe industry that is producing a terrible that powers a very large chunk of the worlds commerce. It's a house built on quicksand and our hope seems to lie in hoping that we can move to another house before this one collapses. The areas in which this is most visible: security, reliability, maintainability, documentation and testing, and finally a complete lack of warranty for suitability from the various manufacturers/service providers. Software is 'eating the world', but is it strong enough to support the world in the longer term if we continue down the road that we are on? I'm not bullish on that.
- croon 10y agoLet me start off with saying I completely agree with your points of the downsides of this. It is however extremely hard to imagine the world that would be, if we were to take a different route. What would be the effects of slower improvements. I assume (since I sometimes do) many people look at the software industry as narcissistic and self-agrandizing when "delivering expriences" and "solving global problems with scalable midware". But what third party industries would be held back if this one moved much slower? Could there be loss of medicinal advancements, monetary, safety, third world advancements? But as you say, I'm also very hesitant on the stability if we can't find some middleground between the two.
- kod 10y agoSurely it has nothing to do with the buyers of our goods and services insisting that the problem they need solving is unique, so trivial it should be done yesterday, and so clear that they dont really need detailed written specifications? To say nothing of IP concerns making it difficult to actually reuse solutions.
- TeMPOraL 10y agoYou hit the nail on the head. If customers/bosses actually figured out what they want before coming to us, it would be simple to write a perfect solution and/or reuse some existing general product to fulfill their needs. As my experience says, however, the requirements keep changing in parallel to the code. In other domains this tends not to happen. When you're in the middle of constructing a beam bridge, you don't get an architect suddenly deciding it should be a suspension bridge instead. Now I'm not saying iterative development and discovery is bad - it's not. But, with iterative development, you treat steps as throwaway, and you don't commit to deadlines. Something the bosses / customers don't understand. You can't have it both ways.
- ArkyBeagle 10y agoArchitects wield cost for requirements changes like a razor sharp sword. "...you treat steps as throwaway, and you don't commit to deadlines." Just that. For cost reasons. Prototypes are like football practice. Production code is like a football game. There is a difference.
- mobiuscog 10y agoClosed source often has the edge of 'focus' because it's working on a very specific type of 'canal'. It doesn't need to handle all types of boat. We used to have very well engineered software - bespoke for each customer. Then businessmen came along and said "Hey - we could save time (money) if we re-used existing things". Eventually businesses didn't have anyone who understood the things they were re-using, which were often designed for a slightly different purpose, and things started going wrong. There is no correct answer, other than do you want speed or quality (I leave the cost aspect out, as some things cannot be done faster or better irrespective of the money thrown at them).
- croon 10y agoSure, and evolution would be much more efficient if every mutation were positive and the species adapted that specific one instead of going out on their own creating meaningless mutations. Who creates the best/bulletproof solutions? You're arguing that we collectively should know which one is the best, among all our different opinions and habits. I'll instead argue that we move forward through natural selection and that everyone learns from trying, failing, trying, succeeding and learning what the best solutions are by experience. I'm not saying it's theoretically optimal, or even practically, but I would suggest it's only natural that we've landed here. That said, I'm all for trying to improve, if you or someone else every comes up with a better route.
- jacquesm 10y agoEvolution is nothing like engineering. Evolution by its very nature makes forward and backwards movements relative to what an outside observer would consider to be an optimum, and selection pressure is what eventually gives rise to a perceived movement in the 'right' direction. Software is - in principle - an engineering domain, but we tend to be much more liberal in applying engineering norms to software production schedules and reliability. This need not be the case. If on the other hand you wish to argue that software is produced in a random fashion and selection pressure is what decides what survives then I rest my case, that view definitely does explain a lot of what I'm seeing.
- croon 10y agoSure, your points on their differences are correct, I agree with that. But I still think my comparison holds. Basically the comparison was made to convery Sturgeon's Law; that 90% of everything made is crap. In that sense it is like evolution: A numbers game. While not explicitly random, distributing risk among groups going in different direction could be an efficient way forward, when counting for what would be lost with some central governance. You then select the optimal direction by results.
- jacquesm 10y ago90% of all bridges and buildings, powerstations, cranes and heavy machines are not crap. Engineering is a responsible job, it's nothing at all like making clay vases or plates where if you throw away 9 out of 10 of the things you make isn't going to cause anybody to suffer other than yourself.
- ap22213 10y agoDifferent canals to be used by different boats One can spend a lifetime trying to build the perfect canal that works for all boats and spans the longest distance over the most varied terrain. But, if one only has one boat to get through, and the distance is short enough, many people will just place a couple sticks of dynamite and be done with it. The most important thing is that the boat gets to port, not the canal. But, once in a while someone will notice all of the holes and try to build a more permanent canal. They'll save a lot of people a lot of time and may even make some money on it.
- naasking 10y ago> Except that in software 1000's of groups have already dug 1000's of canals and yet the next group that will build a canal will insist on doing it their way. Because the topography you're digging through is always different, and sometimes they want a Starbucks every two miles along the canal.
- Aaargh20318 10y agoNo, unlike e.g. civil engineering you're never doing the same thing twice and you're always doing something you've never done before because if you have, you could just use that code. You can't copy-paste a bridge but you can re-use software.
- jacquesm 10y agoYou possibly don't now much about civil engineering, let me start out by writing that every project in civil engineering stands on it's own, it is the knowledge and the practices that get re-cycled from one project to the next and on a higher level from one generation of civil engineers to the next. So it is extremely unlikely that civil engineering will be done twice for an exactly identical project. But no civil engineer is going to make his own table for bolt tensile or sheer strength just to scratch an itch. Then we come to software, in software the same problems appear over and over again and we have in the 100's of possible solutions for some problems that are all almost (but not quite, of course due to sloppy design and lack of standardization) interchangeable. Think web frameworks. They are themselves an attempt to abstract away some elements that are common, but in the meantime there are almost as many webframeworks as there are things that they were trying to abstract away to begin with (and all of them fail on one or more of the details). Or programming languages, another area in which we have re-invented the wheel so many times that the number of programming languages (1000+) will at some point probably exceed the number of human languages (about 6500 in active use, but about 2000 of those have < 1000 speakers). http://www.infoplease.com/askeds/many-spoken-languages.html http://www.infoplease.com/askeds/many-spoken-languages.html http://www.99-bottles-of-beer.net/ http://www.99-bottles-of-beer.net/
- Aaargh20318 10y ago> You possibly don't now much about civil engineering I don't, so maybe it's not the best example. But take houses for example, it's not common to see a dozen or more identical houses being built. After you've done 11, number 12 is not going to be much different.
- 10y ago
- cousin_it 10y agoSure, we could collect lessons and pass them on to the next generation. But as software developers, we can do better than that. We can write libraries that the next generation will reuse. That's why software development, which "takes so long" according to the OP, still goes much faster (in terms of economic growth) than your beloved civil engineering. The ease of making new stuff without studying too much existing stuff, which you complain about, is the whole reason why software is awesome and eats the world.
- jacquesm 10y ago> We can write libraries that the next generation will reuse. We could, but unfortunately we definitely do not do this as much as we could. > hat's why software development, which "takes so long" according to the OP, still manages to go much faster (in terms of economic growth) than your beloved civil engineering :-) If you took away the roads society would collapse, instantly. It is getting to the point where if you take away the internet (or even just a substantial fraction of it) for more than a few days that quite possibly the same will happen. That comes with a degree of responsibility that I do not see reflected in the quality of the product of our industry (including quite a few of those libraries). There are exceptions to this but not many. Economic growth is great, if we can keep that afloat in the long term. Given the stakes and the degree to which even the largest entities are failing to take responsibility for their products and services that growth might be the precursor to a very harsh lesson somewhere in our near future. For every crash, data breach, security issue, hack you have to look back and wonder whether the speed of delivery turned out to be worth the risk (and then you have to discount that even further to take into account that what you read about is only a very small fraction of what actually goes wrong...).
- deleted 10y ago[deleted]
- cousin_it 10y agoIt sounds like you want to hold software companies liable when their products get hacked. Is that really how other industries work? If a terrorist blows up a bridge, should the bridge builder be sued?
- wpietri 10y agoEmotionally, I agree with this point. I want to make reliable things: > we don't need 500 crappy, slightly different ways of doing the same thing, we need maybe 5, battle tested and hardened bullet proof solutions that are well engineered. As much as I'd like to do that, I think we're some ways off from the sorts of stability that allow for that. Sure, we're all neophiles, so there's a fair bit of people using new tools for the fun of it. But I think a much bigger problem is the extent to which our raw materials keep changing under us. For example, the smartphone isn't even 10 years old. We're all still replacing them every couple of years because the new ones keep getting better. Processing power, memory, power characteristics, sensors, network, ports: they all keep changing. The number of processors in my house is easily 5x what it was 10 years ago. The range of capabilities is larger, too: my phone is way smarter, but my lightbulbs are quite dumb. Even if that were stable, our conceptions of computing are still changing. Most of the code we write is in languages that are 20 years old, but we've learned a lot since those languages were created. We talk about "virtual servers", which as a mental model is much like "horseless carriage" or "radio with pictures": it's a sign that our understanding hasn't caught up with reality. We also have an enormous amount of business churn. A ten-year-old tech company seems old. Venture capitalists invested $58 billion last year, mostly in service of disrupting some existing order. Few will see a point in investing for the long term when they aren't sure if there will be one. Even given all that, I think we underinvest in the long term, for reasons I'm sure we'd agree on. But I am often forced to admit that the level of craftsmanship I personally want is at odds with the current practical reality of software. And I'm not alone. Back in 1992, Steve Wozniak was hoping for the end of Moore's Law. Bob Cringely writes: "But while the rest of the computing world waits worriedly for that moment when the lines etched on silicon wafers get so thin that they are equal to the wavelength of the light that traces them—the technical dead end for photolithography—Steve Wozniak looks forward to it. 'I can’t wait,' he said, 'because that’s when software tools can finally start to mature.'" [1] The rise of mobile and cloud computing ruined that plan, of course. But perhaps just about the time you and I retire, the technological base will be stable enough for people to do some serious, long-term building. [1] http://www.cringely.com/2013/04/03/accidental-empires-chapter-17-do-the-wave/ http://www.cringely.com/2013/04/03/accidental-empires-chapte...
- jacquesm 10y ago
- bradstewart 10y agoWe'll get there. Civil engineering has been around a hell of a longer than software engineering.
- up_and_up 10y ago> Except that in software 1000's of groups have already dug 1000's of canals and yet the next group that will build a canal will insist on doing it their way. This is why a prudent developer/team would attempt to leverage frameworks/plugins/existing API's and libraries vs reinventing the wheel. Leveraging well-vetted and popular resources should cut development time considerably.
- jacquesm 10y ago> This is why a prudent developer/team would attempt to leverage frameworks/plugins/existing API's and libraries vs reinventing the wheel. The problem here is that those frameworks/plugins/existing API's are all moving targets, their life-span is very likely substantially shorter than the life-span of your project. Well vetted and popular unfortunately does not equal 'will have staying power'.
- up_and_up 10y agoThat hasnt been my experience. Most of the frameworks I use last much longer than my tenure at a particular position. Are you saying you would really write your own django? jquery? react? chef? git? I would argue the reason some startups can now move blazingly fast with relatively few developers is by leveraging open-source in a smart way.
- jacquesm 10y ago> Are you saying you would really write your own django? jquery? react? chef? git? No, I think it should be fairly obvious that that would not be my choice, rather the opposite. But why django? Why not rails? Or symphony? Or Yii? Or Spring? and so on, why git? and not subversion, or tfs or mercurial? and so on. For every one of those choices there are 10's of possible choices and none of those will be something akin to an industry standard. And so we muddle on. Well, at least git seems to become slowly the standard in revision control, but because git was yet another fresh start project a lot of the lessons learned from other revision control systems that were not readily apparent to the git author(s) had to be re-learned the hard way, resulting in a lot of delay and frustration (and to this date the git command set strikes me as ill thought out). Leveraging open source in a smart way is exactly the right thing to do, now if only there would be some kind of mental penalty to start a new project when an alternative is already available which could use some TLC.
- ArkyBeagle 10y agoThe buyer of software has active disincentives - some of those to the level of national security - to not participate in some collective action to create a library of software for everyone to use. And at some point, you learn to make the software critically dependent not on lines of code doing their thing, but rather to push all the complexity out to the configuration. So now you have man-hours and company profits depending on maintaining a configuration. That's even more ephemeral than code, so it gets even worse.
- blhack 10y ago>the next group that will build a canal will insist on doing it their way. If only every business was identical! Then we wouldn't have to deal with the problem of creating custom software for each of them. In this canal-across-the-country analogy: imagine that every boat is a different size, different shape, different propulsion method, and is made of different materials. And some of them are actually trucks, airplanes, and vacuum tunnels. >Then just build the biggest canal that satisfies every requirement! This exists! It's called IBM.
- forgottenacc57 10y agoAnd then management, who have never seen and cannot visualise what you are building, decide that this is a damn awesome idea and the canal should now be extended to go straight up North America to Canada.
- hackits 10y agoI would say it's the complete lack of Software field being treated like any other major profession. Lack of empirical evidence or evidence based design. A mixture self appointed experts with opinions instead of facts. Sales people who over-sell a product when the engineering department is under-staffed. Over optimistic and out of touch management not knowing how to call developers out on their bull-shot.
- catwell 10y agoThis tweet was really spot on: https://twitter.com/jtolds/status/758716505320075266 https://twitter.com/jtolds/status/758716505320075266
- foobarrio 10y agoIf you could somehow graph the amount of work it takes to program something on a line, that line would be very rugged like a coastline. These "big picture" views and estimating programming work is like trying to estimate the length of a coast line: * https://en.wikipedia.org/wiki/Coastline_paradox
- majewsky 10y agoAnd then the Pointy-Haired Boss comes along and shouts at the engineers: "Why do you eggheads always have to make stuff so complicated? Let's just nuke our way through!" (If you don't get the reference, Ctrl-F for "Panama Canal" on https://en.wikipedia.org/wiki/Operation_Plowshare https://en.wikipedia.org/wiki/Operation_Plowshare .)
- douche 10y agoInteresting analogy. I've been reading The Path Between the Seas[1], about the construction of the Panama Canal, and whoo boy, now I won't be able to think about it except as a failed software startup. The hype. The outrageous valuations and fund-raising rounds. The charismatic founder. The blatant disregard of decades of accumulated knowledge. The pivots. The skirting, blindsiding, and lobbying to get around government regulation. The aqu-hire. The moment when the burn-rate overtook the runway, and further desperate rounds of funding were sought, at poisonous terms. Burn-out, layoffs, and attrition. The fire-sale exit. I'm just now getting up to the patent war-chest aquisition and the second-system effect stages of the canal... [1] http://amzn.to/2eSBq16 http://amzn.to/2eSBq16