4 ms·

So I know re-writing a technical system from scratch is always consider a big no-no. A couple of observations to counter that argument: 1. Re-writing code is t

by mbesto 13y ago

So I know re-writing a technical system from scratch is always consider a big no-no. A couple of observations to counter that argument: 1. Re-writing code is the Innovator's Solution to the Innovator's Dilemma.[0] 2. Mozilla wouldn't exist today if the Netscape team didn't decide to rewrite their browser engine.[1] 3. Isn't Basecamp New a rewrite?[2] [0]http://www.amazon.com/The-Innovators-Solution-Sustaining-Successful/dp/1578518520 http://www.amazon.com/The-Innovators-Solution-Sustaining-Suc... [1]http://en.wikipedia.org/wiki/Mozilla#History http://en.wikipedia.org/wiki/Mozilla#History [2]http://basecamp.com/new http://basecamp.com/new Thoughts?

casual_slacker 13y ago
I've spent two jobs rewriting software. The two main reasons for it: 1) the original was a prototype; 2) the requirements have changed so much that the original architecture doesn't make sense. If it's either of these situations, then a rewrite can make sense. Contrast that with developers wanting to rewrite because of vague frustrations or poorly-defined technical debt. Without a strict definition of the problem, a rewrite is doomed to repeat the same shortcoming as the original. I've seen the most success happen on teams which can figure out how to rewrite the app one module at a time, while keeping the existing parts functional. This reduces the risk, and allows you to compare the new effort against old code more directly.
lmm 13y ago
1. Without buying the book that sounds like a meaningless buzzword. 2. Mozilla wouldn't exist today if the Netscape team didn't destroy their company by deciding to rewrite their browser engine, no. Whether we'd be worse or better off is an open question - a living commercial netscape could have kept the browser wars and associated innovation alive longer. On the opensource side maybe the better-architectured KHTML would have gathered more mindshare and funding and evolved into something like today's WebKit faster without the distraction of gecko.
- mbesto 13y ago
  > that sounds like a meaningless buzzword. I can assure you they are not meaningless buzzwords. I'd recommend reading the book before commenting since your reply adds nothing to the discussion. > 2. Mozilla wouldn't exist today if the Netscape team didn't destroy their company Any source to back up the claim that the Netscape team maliciously destroyed the company? What would be their motive? From everything I've read the codebase was an absolute mess, hence the motivation for a re-write.
  - lmm 13y ago
    >I can assure you they are not meaningless buzzwords. I'd recommend reading the book before commenting since your reply adds nothing to the discussion. Perhaps naively, I assume you hadn't realised you were talking in what sounded like meaningless buzzwords. > Any source to back up the claim that the Netscape team maliciously destroyed the company? I never claimed malice. I'm sure the decision to rewrite the code was made with the best of intentions. It still destroyed the company.
  - rwallace 13y ago
    The code being a mess usually calls for refactoring not rewriting. As I understand it, the showstopper that made a rewrite necessary was that they had made the mistake (understandable since they were in a desperate hurry but mistake nonetheless) of using proprietary libraries.
    - 13b9f227ecf0 13y ago
      Technical debt can accrue to the point where a piece of software is "bankrupt." Any successful refactoring amounts to extracting a subset and redoing everything else, which can practically amount to a rewrite.
- ceejayoz 13y ago
  Would Netscape have survived if they hadn't rewritten the browser engine? I'd've thought they were fucked either way.
  - shortstuffsushi 13y ago
    +1 for spelling creativity. Really, though, I agree. Is the fact that the chose to rewrite their (failing) browser engine the reason that the company failed? Sounds more like an attempt to plug the hole in the company.
jerf 13y ago
It isn't always a big no-no, it's just that it's a known rationality failure on software engineers part, as they consider the benefits of a rewrite but brutally underestimate the costs. Also, the standard and IMHO valid riposte to the Mozilla rewrite is to observe that while it may all have turned out alright for the public in the end, Netscape didn't survive it. If you are in a business context, Netscape is still a warning about failure, not an example of success!
everyone 13y ago
"re-writing a technical system from scratch is always consider a big no-no" Why is this? (Apart from the fact that it takes time) Also what do you mean by "technical system" ?
- stonemetal 13y ago
  There should be a big for a business stuck on there. The primary reasons are loss of quality, practically by definition new code is less well tested than old code; loss of time, in the rewrite you spend time re-implementing features your system already had which doesn't provide new value for end users(it is hard to sell an upgrade of no new features.)
- mbesto 13y ago
  I guess I considered it common knowledge. I would say these two points illustrate my basis: 1. http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html 2. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. http://en.wikipedia.org/wiki/Systemantics#Elementary_Systems_functions http://en.wikipedia.org/wiki/Systemantics#Elementary_Systems... > "technical system" I could have said software, but software can't be very complex or very simple. I probably should have said "complex software" instead.
  - everyone 13y ago
    "A complex system designed from scratch never works" This is based on the work of John Gall who I've never read and cannot comment on. However he was a pediatrician who has never gone near software, I wonder how relevant the personal anecdotes and experience his books draw their conclusions from are. Also what is his definition of complex relating to the above quote? I'm assuming he could plot a graph with complexity along one axis and utility of starting from scratch along the other. At some point it might cross a certain threshold. Also is there much in software development that could really be considered 'complex'? or, how does complexity scale in software? Software could be described as "1000 layers of abstraction" if you are, say, re-writing a program in a high level language you are not starting your system from scratch. All the lower level elements of how it is assembled, run on the computer + all the protocols/ libraries drivers etc. used in the system are not changing.
InclinedPlane 13y ago
There are right ways and wrong ways to do a rewrite. Joel Spolsky says you should never do it but he's flat wrong, mostly, he's right in that it's almost never done correctly. Most rewrites are undertaken too lightly and with too much naivete. They are approached as projects with a lot of major unspoken assumptions. Often assumptions that are counter-factual. Assumptions such as: at code complete we will have a system that is as refined and defect free as a well-worn system that has been in real-world use for years. Or, there will be no problem with forcing every user to use the new system because it will be in every way objectively superior to the old system starting on day 1. Or, taking on the work of a rewrite won't cause a serious shortage of manpower needed to maintain the old system (or vice versa, the rewrite won't face a serious shortage due to the needs of maintaining the old system). And so on. Yet such obvious fallacies remain as latent assumptions behind the majority of rewrites. There are different ways to do rewrites effectively and none of them are easy. One way is to build a parallel product using an entirely separate team, and having that product compete against your older product directly in the marketplace. This is obviously an expensive route, which is why a lot of companies shy away from it, but sometimes it's the best way to go about things. It gives a chance for a product to mature before it replaces the original. Microsoft did this with Windows, having two parallel teams work on different kernels (NT and 9x) and it took over half a decade for the "rewrite" to win out. This is even more astounding when you consider that originally Microsoft had intended for Windows 95 to be simply Windows 4.0 based on the NT kernel. A goal they didn't really achieve until Windows XP was released 6 years later. An alternate way to do a rewrite is through refactoring style techniques. Start building the new system inside the old and slowly deprecate and migrate components piece by piece until the old system is entirely gone. This is a very effective way of doing things and has the advantage of being able to scale relative to the amount of development effort available but certainly has its fair share of limitations and downsides (which I won't enumerate). As far as the Netscape example, they ended up coding themselves into annihilation, as their constant rewrites and architectural buffoonery led them to create a very unstable and clunky browser that was markedly inferior to IE 4 when it hit the market, and they just were not able to respond with any degree of innovation software wise. Also, my understanding is that the seed code for the mozilla project was a horrendous mess that didn't even represent a functional browser at the time and required extensive effort to get to that point.

by mbesto 13y ago

casual_slacker 13y ago

I've spent two jobs rewriting software. The two main reasons for it: 1) the original was a prototype; 2) the requirements have changed so much that the original architecture doesn't make sense. If it's either of these situations, then a rewrite can make sense. Contrast that with developers wanting to rewrite because of vague frustrations or poorly-defined technical debt. Without a strict definition of the problem, a rewrite is doomed to repeat the same shortcoming as the original. I've seen the most success happen on teams which can figure out how to rewrite the app one module at a time, while keeping the existing parts functional. This reduces the risk, and allows you to compare the new effort against old code more directly.

lmm 13y ago

1. Without buying the book that sounds like a meaningless buzzword. 2. Mozilla wouldn't exist today if the Netscape team didn't destroy their company by deciding to rewrite their browser engine, no. Whether we'd be worse or better off is an open question - a living commercial netscape could have kept the browser wars and associated innovation alive longer. On the opensource side maybe the better-architectured KHTML would have gathered more mindshare and funding and evolved into something like today's WebKit faster without the distraction of gecko.

mbesto 13y ago
> that sounds like a meaningless buzzword. I can assure you they are not meaningless buzzwords. I'd recommend reading the book before commenting since your reply adds nothing to the discussion. > 2. Mozilla wouldn't exist today if the Netscape team didn't destroy their company Any source to back up the claim that the Netscape team maliciously destroyed the company? What would be their motive? From everything I've read the codebase was an absolute mess, hence the motivation for a re-write.
- lmm 13y ago
  >I can assure you they are not meaningless buzzwords. I'd recommend reading the book before commenting since your reply adds nothing to the discussion. Perhaps naively, I assume you hadn't realised you were talking in what sounded like meaningless buzzwords. > Any source to back up the claim that the Netscape team maliciously destroyed the company? I never claimed malice. I'm sure the decision to rewrite the code was made with the best of intentions. It still destroyed the company.
- rwallace 13y ago
  The code being a mess usually calls for refactoring not rewriting. As I understand it, the showstopper that made a rewrite necessary was that they had made the mistake (understandable since they were in a desperate hurry but mistake nonetheless) of using proprietary libraries.
  - 13b9f227ecf0 13y ago
    Technical debt can accrue to the point where a piece of software is "bankrupt." Any successful refactoring amounts to extracting a subset and redoing everything else, which can practically amount to a rewrite.
ceejayoz 13y ago
Would Netscape have survived if they hadn't rewritten the browser engine? I'd've thought they were fucked either way.
- shortstuffsushi 13y ago
  +1 for spelling creativity. Really, though, I agree. Is the fact that the chose to rewrite their (failing) browser engine the reason that the company failed? Sounds more like an attempt to plug the hole in the company.

jerf 13y ago

It isn't always a big no-no, it's just that it's a known rationality failure on software engineers part, as they consider the benefits of a rewrite but brutally underestimate the costs. Also, the standard and IMHO valid riposte to the Mozilla rewrite is to observe that while it may all have turned out alright for the public in the end, Netscape didn't survive it. If you are in a business context, Netscape is still a warning about failure, not an example of success!

everyone 13y ago

"re-writing a technical system from scratch is always consider a big no-no" Why is this? (Apart from the fact that it takes time) Also what do you mean by "technical system" ?

stonemetal 13y ago
There should be a big for a business stuck on there. The primary reasons are loss of quality, practically by definition new code is less well tested than old code; loss of time, in the rewrite you spend time re-implementing features your system already had which doesn't provide new value for end users(it is hard to sell an upgrade of no new features.)
mbesto 13y ago
I guess I considered it common knowledge. I would say these two points illustrate my basis: 1. http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html 2. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. http://en.wikipedia.org/wiki/Systemantics#Elementary_Systems_functions http://en.wikipedia.org/wiki/Systemantics#Elementary_Systems... > "technical system" I could have said software, but software can't be very complex or very simple. I probably should have said "complex software" instead.
- everyone 13y ago
  "A complex system designed from scratch never works" This is based on the work of John Gall who I've never read and cannot comment on. However he was a pediatrician who has never gone near software, I wonder how relevant the personal anecdotes and experience his books draw their conclusions from are. Also what is his definition of complex relating to the above quote? I'm assuming he could plot a graph with complexity along one axis and utility of starting from scratch along the other. At some point it might cross a certain threshold. Also is there much in software development that could really be considered 'complex'? or, how does complexity scale in software? Software could be described as "1000 layers of abstraction" if you are, say, re-writing a program in a high level language you are not starting your system from scratch. All the lower level elements of how it is assembled, run on the computer + all the protocols/ libraries drivers etc. used in the system are not changing.

InclinedPlane 13y ago

There are right ways and wrong ways to do a rewrite. Joel Spolsky says you should never do it but he's flat wrong, mostly, he's right in that it's almost never done correctly. Most rewrites are undertaken too lightly and with too much naivete. They are approached as projects with a lot of major unspoken assumptions. Often assumptions that are counter-factual. Assumptions such as: at code complete we will have a system that is as refined and defect free as a well-worn system that has been in real-world use for years. Or, there will be no problem with forcing every user to use the new system because it will be in every way objectively superior to the old system starting on day 1. Or, taking on the work of a rewrite won't cause a serious shortage of manpower needed to maintain the old system (or vice versa, the rewrite won't face a serious shortage due to the needs of maintaining the old system). And so on. Yet such obvious fallacies remain as latent assumptions behind the majority of rewrites. There are different ways to do rewrites effectively and none of them are easy. One way is to build a parallel product using an entirely separate team, and having that product compete against your older product directly in the marketplace. This is obviously an expensive route, which is why a lot of companies shy away from it, but sometimes it's the best way to go about things. It gives a chance for a product to mature before it replaces the original. Microsoft did this with Windows, having two parallel teams work on different kernels (NT and 9x) and it took over half a decade for the "rewrite" to win out. This is even more astounding when you consider that originally Microsoft had intended for Windows 95 to be simply Windows 4.0 based on the NT kernel. A goal they didn't really achieve until Windows XP was released 6 years later. An alternate way to do a rewrite is through refactoring style techniques. Start building the new system inside the old and slowly deprecate and migrate components piece by piece until the old system is entirely gone. This is a very effective way of doing things and has the advantage of being able to scale relative to the amount of development effort available but certainly has its fair share of limitations and downsides (which I won't enumerate). As far as the Netscape example, they ended up coding themselves into annihilation, as their constant rewrites and architectural buffoonery led them to create a very unstable and clunky browser that was markedly inferior to IE 4 when it hit the market, and they just were not able to respond with any degree of innovation software wise. Also, my understanding is that the seed code for the mozilla project was a horrendous mess that didn't even represent a functional browser at the time and required extensive effort to get to that point.