6 ms·

LibreOffice: cleaning and re-factoring a giant code-base [video]

134 points by ExpiredLink 14y ago

raverbashing 14y ago
"Come and hear how we’ve built an international team of developers to tackle the problems of resurrecting a poorly-understood, gigantic code-base extensively commented in German, with no unit tests, a tangled build infrastructure, and twenty-five years of un-paid technical debt." Looks like a project I worked on. It's eerie how similar it is (and no, it was not an office app, it was something different). At least the comments weren't in German, but there were a couple of ones in Dutch. (Not that the comments were of any help) No unit tests: CHECK. Twenty-five year of technical debt, CHECK. And now we're supposed to make it work in a different platform So thank you, but I don't plan on touching a monolithical piece of old C++ ever again. Not even with gloves.
- jacquesm 14y ago
  It's doable. Start by adding integration tests, then (try to) identify the core of the project and isolate that in its own sub-project in the repo. Add unit tests for the functions in the core and then split the remainder into parts that are peripheral and that could be removed/disabled. (say a spellchecker). Next up re-factor the core until you're happy with the state of affairs, but keep the interface the same. If there was no interface to the rest of the code then you'll need to define one and implement that as well as changing the peripheral code to use that interface. Then look at each peripheral piece and decide if it is worth saving, needs rebuilding or refactor it as well. The basic trick is to reduce scope until a subsection of the work becomes tractable. With a really large project this is a multi-year effort with a team of seasoned programmers, it's hard and it means that while you do it there will be no honoring feature requests. You need management buy-in, patience and perseverance but that goes for any large effort. Building fresh and green stuff is obviously much more fun to do but taking an old codebase and making it nice is (to me, at least) just as rewarding.
  - cpressey 14y ago
    I agree, it's doable. You do need to put a lot more effort into coordination between team members than usual, because it's very easy for the developers (should I say the refactorers?) to step on each others' toes.
  - peteretep 14y ago
    Another side of the same approach is just starting to hack bits out in to their own services. Really the key point that you identified is trying to find the internal interfaces of the code, and fracturing the monolithic codebase along those lines.
  - raverbashing 14y ago
    It certainly is doable In my case there were added difficulties: - The project ran on embedded hw (hence it does not have a user interface, only interface is with hw devices) - There was a lack of problem domain understanding, so it's not - like in LibreOffice case, for example, 'this is a list of paragraphs', but 'we don't know what's this piece of data' because if refers to a specific domain of knowledge Not to mention unit tests (both writing and running) are more painful in C++ than in Python for example
    - jacquesm 14y ago
      I think acquiring domain knowledge is one of the most overlooked elements in software design. Lots of people write programs for things they essentially do not understand. This causes all kinds of misery. I'd see this being C++ over Python and embedded as advantages rather than disadvantages! Each to his own I guess :)
  - kami8845 14y ago
    I find this argument smells of rationalisation to me: You admit yourself that building new stuff is much more fun. Say making an old codebase is "just as rewarding" but in order to actually understand all the legacy you need to put in 2x (or whatever the amount) of work that you'd need to build a new thing. Wouldn't you want to build 2 new things instead? I could never imagine someone on their own accord, going by themselves into a 30-year-old codebase and refactoring it, just to make it "look nice". On the other hand people constantly build new stuff (too much of it really). I think it would be fine to admit that working on new code is more desirable than working on old code. Sometimes the latter needs to be done, but I think most programmers are happy to avoid maintenance work when they can.
- supar 14y ago
  I think that this scenario routinely happens at many companies. Only you don't hear public stories about it, because in the end what happens is the same for all software: continuous transition/refactoring. I know I've been working on many projects like in these conditions, though not of that size. With more and more experience of that kind and by working on stinking code-bases, I've come to the conclusion that while in the past I could have thought that trashing the code and starting from scratch would help, now I would probably approach most problems by pushing new code in the form that I want and transition the rest as changes are required. I had projects that I did myself with great design care, but after 5-6 years due to shifting requirements also started to look like you could have done a better job by starting from scratch again. Reality is, in retrospect all code is suboptimal.
pestaa 14y ago
I think all good developers have their connotations and anecdotes for words like 'poorly-understood', 'gigantic', 'no unit tests', 'tangled build infrastructure', let alone '25 years of technical debt'. What I was surprised to see is the word 'resurrect'. As if they wanted to imply LibreOffice was dead.
- BobbyBonsai 14y ago
  Well, I think many developers know the feeling when working with a codebase which just feels like riding a dead horse...uphill, both ways, in the cold...and when you manage to refactor some part it starts to feel like it gets up on one leg and so on.
tomelders 14y ago
I was talking to a friend about "legacy code" the other day and I realised: the company that's going to put you out of business won't have any legacy code. I think you need to have a really good excuse not to completely rebuild your product from scratch every now and again. Because if you don't, someone else will.
- jacques_chester 14y ago
  > I think you need to have a really good excuse not to completely rebuild your product from scratch every now and again. And I suppose we should also burn down factories and rebuild them every now and then.
  - Drakim 14y ago
    If the factories are operating with giant iron cog wheels older than your grandmother, then maybe yes, we should. The world of software is moving a lot faster than the physical world, so things can become outdated pretty quickly. Compare a 15 year old browser to a 15 year old factory.
    - cpressey 14y ago
      This attitude strikes me as wasteful. By all means, if a system has shortcomings, and you can afford to build a new system with fewer shortcomings, and migrate to it, then do that; but by itself, "it's old" is not a reason to rewrite something.
    - deleted 14y ago
      [deleted]
    - jacques_chester 14y ago
      I think that wasting money on solved problems is, essentially, bonkers. Sure, it's nice to have the latest and greatest. But I don't think it gives you magical powers. You still need to do all the other stuff. That, and the well-known problems with rewriting (the track record is notso hotso) mean that really, the onus for making a case to rewrite is on the rewriters.
cpursley 14y ago
LibreOffice needs some UX/UI volunteers, bad.
- qnk 14y ago
  Not only LibreOffice, this is somewhat a general sentiment I have about most Open Source Software. If I were a designer I'd love to be part of such an important project like this. I don't get why designers are always doing stuff like redesigning website X for free, without being asked to, but miss great opportunities to be in the spotlight by participating in projects like GNome, KDE or the likes. There are many examples of good OSS projects that could use some UX/UI love, one that comes to mind right now is Moodle, for example. Is it designers that don't like to participate in OSS? Are programmers making it harder for them to join these type of projects or is simply the fact that they don't bring to much excitement or challenge to the design community?
  - cpursley 14y ago
    Good point. Designers are always giving fre psd's away. Maybe they could do the same, but for Linux UI components. But you know, it might be that they work primarily in photoshop, which is runs on Windows / Mac OS??
  - IgorPartola 14y ago
    Maybe designers do that because it is a much more direct way to get hired or land a contract. Doing design for something like LibreOffice or Gnome has a really long turn-around time and your efforts are somewhat dependent on your teammates. Then, if your design is released, your potential employers/clients need to translate what you just did to how it is relevant to them. OTOH if you just redesign their website, that is a direct solicitation to them.
  - taligent 14y ago
    >I don't get why designers are always doing stuff like redesigning website X for free. Simple. Because if they did assist with FOSS projects much of the time they would be dealing with developers who (a) think they know what users want better (b) have no respect for designers and (c) believe obscure feature X is more important than UI. Remember this is the same group of people e.g. on Slashdot/Reddit who think the iPad is an overpriced toy and think people are sheep for buying it.
fiendsan 14y ago
they just need to focus on speed/performance thats the only thing missing on openoffice/libreoffice its slow... painfully slow!
- macalicious 14y ago
  It's not the only thing. The UX and UI itself is very 'old-school' and IMO very outdated. It's like using Windows 3.11 instead of Windows XP, let alone Windows 7/8.
- BobbyBonsai 14y ago
  Speed comes with refactoring (most of the time).
melling 14y ago
Are the LibreOffice and OpenOffice teams working completely independently? If not, it sounds like a lot of redundant effort is going to take place.
- stephen_g 14y ago
  Given their licenses, LibreOffice can incorporate any code written for OpenOffice into their project, but OpenOffice can't use any of LibreOffice's code.
zandorg 14y ago
I loaded a self-written novel (about 260 pages) into LibreOffice and went to Print Preview and it hung. Loaded it into Office and it worked fine.
- BobbyBonsai 14y ago
  So what?
  - zandorg 14y ago
    Obviously, because it simply doesn't work as it should.
- chris_wot 14y ago
  Have you filed a bug report?
Surio 14y ago
Going through some of the discussions and views and opinions on legacy code rewrites, I thought I should provide another relevant reading warstory: An oldie but a goldie: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html EDIT While I was reading the thread, it appears @jodrellblank had also posted it in the discussion elsewhere in the thread.
evandrix 14y ago
link to slides please?
jfb 14y ago
I wish this was available written instead of as low-bandwidth video.
largesse 14y ago
I tried watching the video, but it had no information. Not a drop of code or an ounce of strategy. It was just a status report.

134 points by ExpiredLink 14y ago

"Come and hear how we’ve built an international team of developers to tackle the problems of resurrecting a poorly-understood, gigantic code-base extensively commented in German, with no unit tests, a tangled build infrastructure, and twenty-five years of un-paid technical debt." Looks like a project I worked on. It's eerie how similar it is (and no, it was not an office app, it was something different). At least the comments weren't in German, but there were a couple of ones in Dutch. (Not that the comments were of any help) No unit tests: CHECK. Twenty-five year of technical debt, CHECK. And now we're supposed to make it work in a different platform So thank you, but I don't plan on touching a monolithical piece of old C++ ever again. Not even with gloves.

jacquesm 14y ago
It's doable. Start by adding integration tests, then (try to) identify the core of the project and isolate that in its own sub-project in the repo. Add unit tests for the functions in the core and then split the remainder into parts that are peripheral and that could be removed/disabled. (say a spellchecker). Next up re-factor the core until you're happy with the state of affairs, but keep the interface the same. If there was no interface to the rest of the code then you'll need to define one and implement that as well as changing the peripheral code to use that interface. Then look at each peripheral piece and decide if it is worth saving, needs rebuilding or refactor it as well. The basic trick is to reduce scope until a subsection of the work becomes tractable. With a really large project this is a multi-year effort with a team of seasoned programmers, it's hard and it means that while you do it there will be no honoring feature requests. You need management buy-in, patience and perseverance but that goes for any large effort. Building fresh and green stuff is obviously much more fun to do but taking an old codebase and making it nice is (to me, at least) just as rewarding.
- cpressey 14y ago
  I agree, it's doable. You do need to put a lot more effort into coordination between team members than usual, because it's very easy for the developers (should I say the refactorers?) to step on each others' toes.
- peteretep 14y ago
  Another side of the same approach is just starting to hack bits out in to their own services. Really the key point that you identified is trying to find the internal interfaces of the code, and fracturing the monolithic codebase along those lines.
- raverbashing 14y ago
  It certainly is doable In my case there were added difficulties: - The project ran on embedded hw (hence it does not have a user interface, only interface is with hw devices) - There was a lack of problem domain understanding, so it's not - like in LibreOffice case, for example, 'this is a list of paragraphs', but 'we don't know what's this piece of data' because if refers to a specific domain of knowledge Not to mention unit tests (both writing and running) are more painful in C++ than in Python for example
  - jacquesm 14y ago
    I think acquiring domain knowledge is one of the most overlooked elements in software design. Lots of people write programs for things they essentially do not understand. This causes all kinds of misery. I'd see this being C++ over Python and embedded as advantages rather than disadvantages! Each to his own I guess :)
- kami8845 14y ago
  I find this argument smells of rationalisation to me: You admit yourself that building new stuff is much more fun. Say making an old codebase is "just as rewarding" but in order to actually understand all the legacy you need to put in 2x (or whatever the amount) of work that you'd need to build a new thing. Wouldn't you want to build 2 new things instead? I could never imagine someone on their own accord, going by themselves into a 30-year-old codebase and refactoring it, just to make it "look nice". On the other hand people constantly build new stuff (too much of it really). I think it would be fine to admit that working on new code is more desirable than working on old code. Sometimes the latter needs to be done, but I think most programmers are happy to avoid maintenance work when they can.
supar 14y ago
I think that this scenario routinely happens at many companies. Only you don't hear public stories about it, because in the end what happens is the same for all software: continuous transition/refactoring. I know I've been working on many projects like in these conditions, though not of that size. With more and more experience of that kind and by working on stinking code-bases, I've come to the conclusion that while in the past I could have thought that trashing the code and starting from scratch would help, now I would probably approach most problems by pushing new code in the form that I want and transition the rest as changes are required. I had projects that I did myself with great design care, but after 5-6 years due to shifting requirements also started to look like you could have done a better job by starting from scratch again. Reality is, in retrospect all code is suboptimal.

pestaa 14y ago

I think all good developers have their connotations and anecdotes for words like 'poorly-understood', 'gigantic', 'no unit tests', 'tangled build infrastructure', let alone '25 years of technical debt'. What I was surprised to see is the word 'resurrect'. As if they wanted to imply LibreOffice was dead.

BobbyBonsai 14y ago
Well, I think many developers know the feeling when working with a codebase which just feels like riding a dead horse...uphill, both ways, in the cold...and when you manage to refactor some part it starts to feel like it gets up on one leg and so on.

tomelders 14y ago

I was talking to a friend about "legacy code" the other day and I realised: the company that's going to put you out of business won't have any legacy code. I think you need to have a really good excuse not to completely rebuild your product from scratch every now and again. Because if you don't, someone else will.

jacques_chester 14y ago
> I think you need to have a really good excuse not to completely rebuild your product from scratch every now and again. And I suppose we should also burn down factories and rebuild them every now and then.
- Drakim 14y ago
  If the factories are operating with giant iron cog wheels older than your grandmother, then maybe yes, we should. The world of software is moving a lot faster than the physical world, so things can become outdated pretty quickly. Compare a 15 year old browser to a 15 year old factory.
  - cpressey 14y ago
    This attitude strikes me as wasteful. By all means, if a system has shortcomings, and you can afford to build a new system with fewer shortcomings, and migrate to it, then do that; but by itself, "it's old" is not a reason to rewrite something.
  - deleted 14y ago
    [deleted]
  - jacques_chester 14y ago
    I think that wasting money on solved problems is, essentially, bonkers. Sure, it's nice to have the latest and greatest. But I don't think it gives you magical powers. You still need to do all the other stuff. That, and the well-known problems with rewriting (the track record is notso hotso) mean that really, the onus for making a case to rewrite is on the rewriters.

cpursley 14y ago

LibreOffice needs some UX/UI volunteers, bad.

qnk 14y ago
Not only LibreOffice, this is somewhat a general sentiment I have about most Open Source Software. If I were a designer I'd love to be part of such an important project like this. I don't get why designers are always doing stuff like redesigning website X for free, without being asked to, but miss great opportunities to be in the spotlight by participating in projects like GNome, KDE or the likes. There are many examples of good OSS projects that could use some UX/UI love, one that comes to mind right now is Moodle, for example. Is it designers that don't like to participate in OSS? Are programmers making it harder for them to join these type of projects or is simply the fact that they don't bring to much excitement or challenge to the design community?
- cpursley 14y ago
  Good point. Designers are always giving fre psd's away. Maybe they could do the same, but for Linux UI components. But you know, it might be that they work primarily in photoshop, which is runs on Windows / Mac OS??
- IgorPartola 14y ago
  Maybe designers do that because it is a much more direct way to get hired or land a contract. Doing design for something like LibreOffice or Gnome has a really long turn-around time and your efforts are somewhat dependent on your teammates. Then, if your design is released, your potential employers/clients need to translate what you just did to how it is relevant to them. OTOH if you just redesign their website, that is a direct solicitation to them.
- taligent 14y ago
  >I don't get why designers are always doing stuff like redesigning website X for free. Simple. Because if they did assist with FOSS projects much of the time they would be dealing with developers who (a) think they know what users want better (b) have no respect for designers and (c) believe obscure feature X is more important than UI. Remember this is the same group of people e.g. on Slashdot/Reddit who think the iPad is an overpriced toy and think people are sheep for buying it.

fiendsan 14y ago

they just need to focus on speed/performance thats the only thing missing on openoffice/libreoffice its slow... painfully slow!

macalicious 14y ago
It's not the only thing. The UX and UI itself is very 'old-school' and IMO very outdated. It's like using Windows 3.11 instead of Windows XP, let alone Windows 7/8.
BobbyBonsai 14y ago
Speed comes with refactoring (most of the time).

melling 14y ago

Are the LibreOffice and OpenOffice teams working completely independently? If not, it sounds like a lot of redundant effort is going to take place.

stephen_g 14y ago
Given their licenses, LibreOffice can incorporate any code written for OpenOffice into their project, but OpenOffice can't use any of LibreOffice's code.

zandorg 14y ago

I loaded a self-written novel (about 260 pages) into LibreOffice and went to Print Preview and it hung. Loaded it into Office and it worked fine.

BobbyBonsai 14y ago
So what?
- zandorg 14y ago
  Obviously, because it simply doesn't work as it should.
chris_wot 14y ago
Have you filed a bug report?

Surio 14y ago

Going through some of the discussions and views and opinions on legacy code rewrites, I thought I should provide another relevant reading warstory: An oldie but a goldie: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html EDIT While I was reading the thread, it appears @jodrellblank had also posted it in the discussion elsewhere in the thread.

evandrix 14y ago

link to slides please?

jfb 14y ago

I wish this was available written instead of as low-bandwidth video.

largesse 14y ago

I tried watching the video, but it had no information. Not a drop of code or an ounce of strategy. It was just a status report.