4 ms·

I was around for the Google Webserver rewrite and it was actually a rewrite done by refactoring not clean slate. It's amazing what you can do with disciplined r

by zaphar 7y ago

I was around for the Google Webserver rewrite and it was actually a rewrite done by refactoring not clean slate. It's amazing what you can do with disciplined refactoring, even moving to a new language.

BurningFrog 7y ago
If you have a decent test suite to lean on, you can work miracles this way!
- wpietri 7y ago
  And if you don't have a decent test suite, you might as well build one. So much of a rewrite is archaeology, figuring out what elements of a system are intentional versus accidental. A great way to do that archaeology is to start weaving a net of tests around the existing system. Sort of like Fowler's Strangler Fig pattern: https://martinfowler.com/bliki/StranglerFigApplication.html https://martinfowler.com/bliki/StranglerFigApplication.html
  - majormajor 7y ago
    Yeah, if you don't have a detailed test suite, and can't build one, you probably don't fully understand your requirements, and your rewrite is in trouble already.
    - AstralStorm 7y ago
      Mind you, maintenance is in trouble too. Just a different kind of trouble. If you have original code, you can always write a test suite based on it, unless management is averse to this. (Begging the question why they're not averse to a rewrite.) Like any test suite it is not going to be complete. Ultimately it's best not to get piled on by rewriting really and often in small parts. (Carmack way.)
pseudoramble 7y ago
This sounds really interesting to read about, especially refactoring into an entirely new language. Are there any articles about what that process looked like for Google at the time? Or any example that changed languages really.
- dpwm 7y ago
  Not the OP, and no experience with google, but I have personally worked in a way that sounds as if it could be similar: First, you start with a slow but correct prototype in python. This can then be refactored into python closer to C, thinking about the data structures and functions that will be needed in C. Then, you can refactor these bits into C. I've usually done this one module at a time. You then write python bindings to this new C code. There are a number of shortcuts available here with differing tradeoffs, but you can just use the Python/C API. In the end you're left with a C library, some bindings, and some python that just calls to the python bindings for the C library. It would not be a huge leap to go from that to a full C implementation. Nothing I've said here is C specific, so you can do the same with C++. There are upsides and downsides to this whole approach. For me, one of the downsides is the amount of stuff to remember, especially some of the quirks of extending Python. I find it easier to start with something correct but high-level in Python and work down to something easier to translate into C, even if I skip straight to the writing in C bit. I have found this usually leads to a significantly better design whilst it was still Python. You can also test the behaviour of the C and python modules against each other.

by zaphar 7y ago

BurningFrog 7y ago

If you have a decent test suite to lean on, you can work miracles this way!

wpietri 7y ago
And if you don't have a decent test suite, you might as well build one. So much of a rewrite is archaeology, figuring out what elements of a system are intentional versus accidental. A great way to do that archaeology is to start weaving a net of tests around the existing system. Sort of like Fowler's Strangler Fig pattern: https://martinfowler.com/bliki/StranglerFigApplication.html https://martinfowler.com/bliki/StranglerFigApplication.html
- majormajor 7y ago
  Yeah, if you don't have a detailed test suite, and can't build one, you probably don't fully understand your requirements, and your rewrite is in trouble already.
  - AstralStorm 7y ago
    Mind you, maintenance is in trouble too. Just a different kind of trouble. If you have original code, you can always write a test suite based on it, unless management is averse to this. (Begging the question why they're not averse to a rewrite.) Like any test suite it is not going to be complete. Ultimately it's best not to get piled on by rewriting really and often in small parts. (Carmack way.)

pseudoramble 7y ago

This sounds really interesting to read about, especially refactoring into an entirely new language. Are there any articles about what that process looked like for Google at the time? Or any example that changed languages really.

dpwm 7y ago
Not the OP, and no experience with google, but I have personally worked in a way that sounds as if it could be similar: First, you start with a slow but correct prototype in python. This can then be refactored into python closer to C, thinking about the data structures and functions that will be needed in C. Then, you can refactor these bits into C. I've usually done this one module at a time. You then write python bindings to this new C code. There are a number of shortcuts available here with differing tradeoffs, but you can just use the Python/C API. In the end you're left with a C library, some bindings, and some python that just calls to the python bindings for the C library. It would not be a huge leap to go from that to a full C implementation. Nothing I've said here is C specific, so you can do the same with C++. There are upsides and downsides to this whole approach. For me, one of the downsides is the amount of stuff to remember, especially some of the quirks of extending Python. I find it easier to start with something correct but high-level in Python and work down to something easier to translate into C, even if I skip straight to the writing in C bit. I have found this usually leads to a significantly better design whilst it was still Python. You can also test the behaviour of the C and python modules against each other.