5 ms·
My opinion? Because as much as the software industry loves to adopt the work "engineer", almost none of the sort of websites and software discussed here on HN
by bigiain 15y ago
My opinion?
Because as much as the software industry loves to adopt the work "engineer", almost none of the sort of websites and software discussed here on HN ever gets anything much like "engineering" in the sense of, say, "civil engineering" done.
I think in my ~20year career, I've had only 3 projects that were specified well enough up front that we just "built it according to the plans" and had a satisfied customer at the end. Overwhelmingy, some (or most) of the design gets "made up as we go along". Even on projects with several small forests worth of up-front documentation, there's almost always large areas of vagueness or outright contradictory requirements, which need decision making on the developers part halfway into the job.
There are _very_ few websites that couldn't be coded from the ground up in a few weeks with a few good programmers - _IF_ you had already thought through all of the things the site needs and all of the consequences and the contradictions in those consequences. Even the big ugly complex projects, like Google's search or Facebook or Wikipedia or Twitter - if you had a spec that answered all the details about how you wanted them, a small team of experienced web guys could get it up and running in a month or so, and let you now how much cloud/hardware/sysadmin/support you'd need to budget for as your customer acquisition kicked in.
It doesn't take very long to "build" the software or the website (obvious "large scale" projects like OSes excepted). What takes most of the time is identifying and solving the problem. (And, I suspect a lot of the reason software/websites have a reputation for taking "so long" is that they're _way_ too often rushed into the "building" phase way before all the things that need designing are even identified, then all the new solutions take longer 'cause we try to fit them around all the work that's already been done.)
- jurre 15y agoI think you might be underestimating how much custom code has been written by companies like twitter and Facebook, think php to c++ compilers, databases, rpc servers, log frameworks etc etc. even if all of the designs of all of that software was given upfront I doubt any small team could build it within a month or so.
- bigiain 15y agoI'll admit I'm exaggerating for effect, but… If the design and spec job was done with full knowledge of current state-of-the-art "web frameworks, databases, rpc servers, log frameworks, etc etc." - and requirements adjusted to suit "off the shelf" tested and reliable code, only speccing "custom code" where absolutely necessary, I think my claim is still supportable. I don't want a spec that says "you need to write this in php - therefore you need to write a php compiler to scale it". I want those problems solved by the spec. The "coders" just want APIs and datastructures and wireframes and finalised graphic design and content/content-inventory. I reckon I know people who could do it.
- thwarted 15y agoAnd the reason that custom code gets written is because it's specific to that environment. The wide majority of the software available is focused on either a very generic target or a very specific one. If you use something that is very generic, you need to customize it because one size does not fit all. And you can't use the very specific one because it doesn't match your use case. Or what's available is designed for a single webserver and a single database server, so it's really tempting to use that when you're small then you'll most likely need to replace that at scale. And by then, you're writing something that is specific for your use case, which you've discovered has grown dynamically and organically based on changing business and technical requirements. It's difficult to take technology or software from one place and transplant it into another. The best we can do is take what we learn on past projects and past companies and keep those lessons in mind when making new decisions.
- ExpiredLink 15y ago> I think in my ~20year career, I've had only 3 projects that were specified well enough up front that we just "built it according to the plans" and had a satisfied customer at the end. Overwhelmingy, some (or most) of the design gets "made up as we go along". Even on projects with several small forests worth of up-front documentation, there's almost always large areas of vagueness or outright contradictory requirements, which need decision making on the developers part halfway into the job. So the underlying development method, nowadays denounced as 'waterfall', wasn't sufficient? I am not an Agile zealot but change and 'knowing better' needs to be embraced during the development process and not ruled out.
- lechris48 15y agoAnd that is what actually differentiates software engineering from other engineering disciplines. You cant't simply embrace change when building a house or you'd have to tear it down multiple times during building...
- redthrowaway 15y agoI've worked in construction, and that's actually not the case at all. What happens is very similar to what happens in software development: someone realizes that the spec is borked; the subcontractor and general contractor get together to look it over, and come up with a kludge that fixes the problem as easily as possible. If the change is at the customer's request, they get charged a nice premium and the necessary changes are made. At the end of the day, everyone still gets paid regardless of how often the customer changes their mind. This, then, is perhaps what the trades do better than us: they're very upfront with the customer about added costs. "Yes, we can make your bathtub a jacuzzi. It'll cost you $2500 more and the job will take a couple days more. Do you want to go ahead with your change?" Because there is a clear and complete plan that the customer signs off on before construction begins, it's very easy to show them what a change in requirements entails. Everyone can envision what knocking a wall down means, and can see it will cost money and time. Many decide, upon reflection, that they really don't care enough to pay that price. Others go nuts and end up with expensive projects that take forever. Either way, the contractors get paid.
- mbenjaminsmith 15y ago"It doesn't take very long to 'build' the software or the website (obvious 'large scale' projects like OSes excepted)." A lot of assumptions have to be met for that to be true. It shouldn't take very long provided: 1. The programmers are very familiar with the tools. 2. The programmers are very familiar with all necessary 3rd party components. 3. The 3rd party components account for 99% of functionality. 4. The programmers have built that exact project before, preferably more than once. I usually work in a best-case scenario for development: I'm building projects with limited scope, solo and with no other decision makers involved in the project. I'm also experienced enough to know how complex things will generally be from a high level standpoint. And I have never once met my own estimate for complexity and the amount of time something will take. Exactly 100% of the time silly, seemingly inconsequential things add up to an extra 30% or more. It doesn't matter if you can write 1000 shippable lines of code in a day because tomorrow you'll spend most of your time hunting a bug in an external library. Suddenly your machine-like pace has been cut in half. This happens on every project I've worked on and I would assume all software projects. I agree that poor planning adds to the amount of time needed. Of course it does. But there really is no "best case" scenario where you and a couple of buddies could hammer out Facebook in three weeks.
- bigiain 15y ago"It doesn't matter if you can write 1000 shippable lines of code in a day because tomorrow you'll spend most of your time hunting a bug in an external library. Suddenly your machine-like pace has been cut in half. " I'm there right now - why doesn't this Concrete 5 ecommerece plugin's Paypal addon work this week, when nothing's changed since last week??? It's not just you…
- mbenjaminsmith 15y agoI was thinking a bit lower level than that but yeah, I guess you get my point. I spent a day with a bug that I found in OpenCV (a computer vision/machine learning library) that spooks me to this day. How it works internally is that for images to be processed you create a header (with usual image header info) which contains a pointer to the actual image data. There are two ways to do it: one you create the header and then create the image data separately and the other you create the image all in one go. I ended up in a situation where the image (it was going through a couple of different filtering algorithms) would get processed and be displayed regardless of whether or not I had set the pointer to the image data. That image data was already in memory since it was getting read from disk in a separate step, but there was really no way for the library to know its location in memory. I spent a day trying to sort out what was going on -- and threw my own schedule -- because I had to assume something was seriously wrong and would lead to other unintended consequences. I never did figure out what it was but after a lot of testing I decided it wouldn't cause any other problems (it never did, the library I was writing is still the core of a product I'm selling today). If you don't know it OpenCV is about as mature as a software project can get. It was most likely the best tool for the job I had to tackle. But even with the library's stability and widespread use there was just plain voodoo that set a project I was working on back. Other 3rd party tools can be much worse. I don't do any web stuff but I have had to integrate Facebook Connect into products. Interfacing with Facebook can be a nightmare. Their API changes fairly regularly and the documentation can be a mess. Even using a 4th party, regularly updated, helper library I've seen bugs that took ages to sort out. And that's just talking about integrating 3rd party, hopefully mature, components. I don't care how good you are, writing new code means a lot of testing and debugging. I've never seen new code that didn't introduce some voodoo of its own into a project.
- fridek 15y agoI was working as a freelancer for a long time and I had a quite simple solution for lack of documentation. Before the first meeting I'd send a customer 12-page form to fill which covered most of the "what and why" of the project. It quickly turned out that most of people have no idea whatsoever about the product they wanted to buy. I know that it's partly our job to guide people through rough path, but if all customer knows is "I want a website" it's better to quit early. Sometimes I waited for a filled form for few months or even more (not paying much attention), and some of these projects were very successful. If I'd have started building right away, I'd spent these few months struggling and waiting for decisions to be made. What is even worse, I'd probably estimate my time and salary based on wrong assumptions, so I would be mad and underpaid. The project would take way longer than I estimated, so customer would be angry too. Do yourself a favor and never start coding before final documentation arrives.
- smiler 15y agoCould you post that form somewhere?
- eric-hu 15y agoI, too would like to see this form. It would provide a lot of guidance for my company. We're knee deep in the consequences of scope creep right now.
- fridek 15y agoIt is in Polish, so I guess it won't be very useful. I may translate it and post somewhere if I have some free time. For now, try googling "website brief template", it's how I made my first version, then improved it over time. It's basically a compilation of most of the questions I had to ask during my previous projects. If you like to look like a funny guy try adding some "would you like a pictures of your cat on the website". It releases some tension associated with such formal document (which I usually attach to signed contract, just in case), but be prepared to hear "yes, I actually would like to have cats on my website". It's probably worth mentioning that this approach is correct only for small and medium projects, like things possible to deploy with Wordpress ninja in team. This may be a good place to start: http://www.methodandclass.com/article/write-a-web-site-brief/ http://www.methodandclass.com/article/write-a-web-site-brief...
- StavrosK 15y agoSure, but then, what's a program, other than an exhaustive spec?
- Tichy 15y agoAs I've written in my unknown blog, I think there is another misunderstanding here: software development is not the analogy to building something according to a plan. It is making the plan. http://blinker.net/2009/08/31/the-architecture-analogy-for-programming-is-wrong/ http://blinker.net/2009/08/31/the-architecture-analogy-for-p...
- lifeisstillgood 15y agoI have one of those invisi-blogs too http://ciocookbook.com/philosophy/thecodeisthedesign.html http://ciocookbook.com/philosophy/thecodeisthedesign.html Jack Reeves is the man here though - writing source code is designing the product. Before code can be written you need to take all the trade offs, the decisions and the research (or just make wild assumptions). Once you have source code, it has been designed. Which is why instead of sending out a 12 page form telling a client they need to think harder about something they don't understand, put out version 0.1, and ask them what they want changed.
- nahname 15y agoLook at your own experience, the building phase in the design phase. We will (almost) never get a complete spec before hand because people are in charge and people change their minds. The sad reality is that building something with developers is often a faster, cheaper way to discover what is actually needed. Most people cannot tell you what they want until it is right in front of them and even then the response is usually, "Not this".
- tgrass 15y agoI'm a civil engineer (five years) and have never been on a project that avoided the problems of a custom poorly specificied software build. Civil projects are a constant dialogue between client and engineer - but we get further monkey wrenched by one additional stakeholder that software doesn't have: local and at times federal government review. If you think software dev is difficult now, wait til your code is regulated.
- j-b 15y agoMy code is regulated. I develop retail point of sale technology which requires certification with various state agencies such as WIC. Each state likes to do things just a bit differently, which keeps things interesting to say the least.
- Czarnian 15y agoSoftware for banking and insurance is like this. The product has to accommodate different rules for all 50 states and federal oversight. Changes to just about anything need to be cleared by the relevant state and federal agencies. Another big difference between physical engineering and information engineering is that when an overpass is completed, the project is done. If you're lucky, someone will come by and look it over or maintain it over the years. There's no such thing as "done" software. When a program wraps up, everybody rushes back in and starts monkeying with it for the next iteration.
- OriginalSyn 15y agoIn Alberta, Canada, software developers cannot be have "Engineer" in their title, for this very reason, unless they fit a very specific set of requirements [1]. The Canadian Council of Professional Engineers defends the title, via The EGP Act, in order to preserve the credibility of the profession. [1] http://www.apega.ca/About/ACT/preface.htm http://www.apega.ca/About/ACT/preface.htm