22 ms·

Restructuring a giant, ancient codebase to make LibreOffice work well everywhere

489 points by buovjaga 9y ago

eloy 9y ago
I'm quite surprised that the markup incompatibility with Microsoft Office is still an issue. Are they actively sabotaging the Open Document Foundation, or is it just really difficult to achieve?
- tadfisher 9y ago
  I'm not surprised at all that someone on HN replies to a positive story about some open-source product with unrelated bug complaints, then implies the bugs are due to malice on the part of the developers.
  - TomMarius 9y ago
    I think he meant Microsoft...
    - tadfisher 9y ago
      d'oh! I'll leave my comment up as a warning to others to reread carefully before getting annoyed.
  - cabaalis 9y ago
    My worst downvotes have been when I'm referring directly to the motivations or knowledge of the commenter. My second-worst downvotes are when I'm opining on something I know nothing about. My suggestion (to myself, and anyone else interested) is to avoid these two pitfalls.
- Lordarminius 9y ago
  ::Are they actively sabotaging the Open Document Foundation, or is it just really difficult to achieve? The former is a more likely scenario
- beamatronic 9y ago
  Is compatibility more of a problem with .doc or .docx? .doc shouldn't still be evolving, I'd imagine.
  - _delirium 9y ago
    .doc is more stable, but it's also much more onerous to be fully compatible with. .docx was at least designed as a reasonably modern, rational document format with a specification. .doc by contrast evolved from generations of MS Word that for efficiency saved files by doing raw dumps of internal data structures as binary blobs, which got more and more complicated as things accreted over the years and versions. And there was no public documentation of the very complex binary format until 2008. Joel Spolsky has some on that here: https://www.joelonsoftware.com/2008/02/19/why-are-the-microsoft-office-file-formats-so-complicated-and-some-workarounds/ https://www.joelonsoftware.com/2008/02/19/why-are-the-micros...
    - nradov 9y ago
      The JPEG file formats are nearly as bad, especially once you get into camera vendor extensions. It was common practice back then. Just take a bunch of C structs and dump them from memory to disk.
- buovjaga 9y ago
  I don't think they need to actively sabotage anything and I would not want to dwell on those lines of thinking. I think a large part of the open issues are in corner case territory or due to the freedom of users to go wild with their document layouts, creating a mess of nested frames in tables etc. Let's roll up our sleeves and fix them anyhow :) !! Here is our meta report for MSO formats: https://bugs.documentfoundation.org/showdependencytree.cgi?id=107585&hide_resolved=1 https://bugs.documentfoundation.org/showdependencytree.cgi?i... "depends on 1394 open bugs" - this includes all the further meta reports.
- kjhughes 9y ago
  The OOXML specification is over 5K pages long. Even Microsoft's own online products do not implement it entirely. One should not be surprised that outside parties may face challenges achieving 100% implementation and interoperability.
  - merb 9y ago
    and even word/excel has discrepancies between 2003, 2011, 2013 and 2016 especially between 2011 and 2013. Mostly overlapping layouting is differently.
- pbhjpbhj 9y ago
  Have Microsoft completely solved compatibility between different/same versions of MS Office now?
  - pletnes 9y ago
    Nope, I ran into word documents that word could not render just last month. And I hardly use word; I edit about 1-5 docs per month.
  - Spooky23 9y ago
    It took my enterprise almost a year to iron out the issues to move from 2010/2013 to ProPlus/2016.
- readams 9y ago
  Realistically LibreOffice is not really relevant competition for Microsoft. Google Docs is much more important, even though the capabilities are so widely disparate.
  - adrianN 9y ago
    I guess that depends on your filter bubble. I know many more people who use LibreOffice compared to Google Docs. Most people I know don't trust the cloud with their important documents.
    - ozim 9y ago
      I do not trust cloud even with unimportant documents. Especially if you can be locked out of your cloud account without reason. I only keep non critical offsite backups on cloud drives.
- mikekchar 9y ago
  A very long time ago, I used to work on document importing and exporting on Word Perfect so I'm relatively familiar with the problem in general. I haven't looked at this stuff in almost 20 years, so I have no idea about specifics at this point :-) What people generally don't understand is that the save file formats do not describe what the output will look like. Back in the old, old, old days you would write a word processor and to "save" the file, you would just write a binary image of your data structures. That was what the doc file format was originally. Although, people didn't tend to program this way back then, imagine a Rails server. Now imagine that you handed over your database (with tables that correspond to each model object). That's kind of analogous to a "save" file. In order to render the data, you need the business logic (which, in Rails, is usually housed in the controllers) and you need the rendering logic (which is usually housed in the views). Just having a description of the database tables doesn't really help me all that much in rendering the data as you can see. Now imagine that we take a different Rails app that happens to do the same kind of thing. Some people say, "Oh it would be swell if you could import the data from the first Rails app and render it in yours. You are basically doing the same app, so it should be easy". But it's obviously not -- I wrote my controllers and views completely differently than the other Rails app. My model objects contain completely different data and are structured completely differently. Even worse -- I don't even render things the same way. If I want to support this functionality, I basically need to completely rewrite my rails app to be exactly the same as the previous Rails app. And this is essentially the problem. If you want to render Word documents the way Word does, you basically have to rewrite Word in your word processor. If you also want to support your format then you have to maintain 2 word processors. If you want to supports some other file formats? It just gets worse and worse. So what you could do is completely rewrite the rendering engine so that it is more flexible. But the problem is that this is a massive undertaking and nobody will believe you if you tell them that it is necessary. Even worse, there is a surprising amount of variation in rendering. For example, how do you deal with page breaks in footnotes? Most people don't write footnotes that are longer than a page (if you look at my posting history, you might suspect that I'm one of those people, but I digress). However, it's very common in the legal field. There is a specific correct way to break pages in footnotes and Word historically has not done it correctly. They do it differently. This is actually what kept Corel/Word Perfect in business for a long time -- in order to print the document, you would have to use Word Perfect and since the file conversion was crap, you pretty much had to keep it as a Word Perfect file forever. I will say that while I worked on Word Perfect, Microsoft was very helpful in explaining how their formatter worked. They even regularly sent us bug reports when our import filters had regressions, etc. I've never believed that they intentionally obfuscated the process. It's just a difficult problem.
- kakwa_ 9y ago
  It's really difficult to achieve. The article mentions EMF/EMF+. this format is basically a list of calls to GDI.h and is a bitch to map to other graphic stacks when you are not on Windows. The specification of the format is public (kudos for Microsoft), it helps a lot. But there are a lot of corner cases and the spec can be quite hard to understand some times. Computing correctly the origin is tricky for example (VIEWPORTORGEX, WINDOWORGEX etc). For some stuff, it's impossible to get it right. One example that comes to my mind is a text encoding bug I got a year ago (I maintain an emf to svg conversion library). It took me one week to track it down. In most cases the strings are UTF-16. But in weird cases, when the ETO_GLYPH_INDEX flag is set, the "encoding" is directly the index of the glyph inside the selected font. It's not the most portable way to handle text... If you don't have the exact same font on your computer, there is a good chance the text will not be displayed correctly. And converting back to a well known encoding is tricky (using the cmap tables of the ttf file, you have to build a reverse cmap, pray there is a 1-1 mapping between glyph and unicode, and convert back to utf-8 or something). Even Microsoft itself, on the Mac OS, version got it wrong.
  - chris_wot 9y ago
    Can you link to that library?
    - buovjaga 9y ago
      I found it in his submission history: https://github.com/kakwa/libemf2svg https://github.com/kakwa/libemf2svg
      - kakwa_ 9y ago
        thanks ^^ For information, the bits of code that is handling "font index encoding": https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2svg_utils.c#L1012 https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2... To https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2svg_utils.c#L1278 https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2...
niftich 9y ago
He has a slide deck [1] that he links to in the interview, where he tries to highlight some challenges with grokking large codebases, and techniques one can use to compartmentalize understanding and build from there. I found it both a bit too colloquial yet also too opaque, but there's some useful nuggets in there. It also speaks to a significant problem: large codebases are inherently difficult to understand, which forms a barrier to entry. This is despite the fact that most software makes use of various abstractions to break problems into smaller parts. In the general sense, I believe it's worth thinking about how the particular abstraction mechanisms used in software impact a new contributor's productivity, vs. the productivity of someone more familiar with the codebase, especially for projects that are developed in the open, or with high developer turnover. [1] https://people.gnome.org/~michael/data/2016-04-29-solving-problems.pdf https://people.gnome.org/~michael/data/2016-04-29-solving-pr...
- jandrese 9y ago
  I've always found abstraction to be a double edged sword when trying to understand codebases. Sometimes it helps define the intent, but other times I'm going "WTF is this thing for real?" and tracking the abstractions through several files. My favorite is tracking down a member function and being completely confused until I realize that I missed one of the layers and it was overridden in that instantiation.
  - sbov 9y ago
    We tend to speak of abstractions in only positive terms. Some negative ones could include opaque, obtuse, disjointed, etc.
    - Gibbon1 9y ago
      When I think of abstractions I think abstraction == API and good API design is very hard.
  - flukus 9y ago
    I generally find that abstraction is good, but layering is bad. Let's say I have components A, B, and C that all do something useful and are good abstractions, the difference between good code and bade code is how they interact. In bad code A will call B and B will call C, from the top level it's completely opaque how everything fits together. I'd much rather A returns something and let's the caller pass it to B. There is very little difference in code required but it makes a world of difference to making it comprehensible.
ktpsns 9y ago
It's so amazing that the dinosaur LibreOffice/OpenOffice is still maintained. From the codebase, it is probably similarly old-fashioned like Thunderbird. Some folks may say it is hopeless to modernize. For me it does not even matter whether that's true or not -- it is a well-done and partially timeless and universal tool in my everyday work flow. While there are fancy and modern alternatives like the (closed source) Google Docs, LibreOffice always stays as a fall-back solution if nothing more works. Thank you for this work, developers!
- draw_down 9y ago
  Well, yeah, users typically don't care much about the code of something they use at all. Not that it should be otherwise but it's not exactly a new sentiment.
- dfox 9y ago
  The most crufty part in both mozilla's code base and libreoffice code base is essentially equivalent in that it tries to solve essentially same set of problems (cross-platform UI and object model), but the implementation is significantly different, because in mozilla's case this layer of cruft was introduced 20-ish years ago as a complete rewrite of the thing (and most of the really ugly hacks have to do with sub-par C++ compilers and support for classic MacOS), while there was no attempt at a major rewrite of the staroffice codebase until libreoffice (and it is quite obvious that the way libreoffice's rewrite proceeds is at least partially motivated by experience from how the netscape->mozilla rewrite suffered heavily from second system effect and in the end failed at least from the bussines standpoint).
  - ktpsns 9y ago
    Good point. Indeed it is interesting to follow the Firefox Quantum approach (which breaks with XUL) or KDE's open source office suite, aka Calligra (https://www.calligra.org/ https://www.calligra.org/) which is "plain C++/Qt" and thus not a sole effort to solve the cross-platform UI/object model problem such as Mozilla and StarOffice did in their time. Thanks to this, Calligra devs can probably concentrate much more on implementing features while getting a modern cross-plattform UI for free (literally: Qt is free).
ZenoArrow 9y ago
Thank you for sharing this article. Speaking personally, it comes at a great time. I've been toying with the idea of getting involved in open source, and was particularly interested in pushing forward open source office suites. Something that I'm particularly interested in with LibreOffice is helping the core being decoupled from the UI, to both help with portability and to speed up the rate of UI innovation. I had thought that I may have to start by myself, but based on this article it seems that the current LibreOffice developers are thinking along similar lines. If any LibreOffice developers are reading this, are there any sub teams setup that may be a good fit? To give some idea of my background, I would describe myself as a tester that knows how to code (not in C++ alas, but I'm familiar enough with debugging at a low level).
- buovjaga 9y ago
  Regarding the UI, there are certainly some... curiosities. To give you a random anecdote from the trenches: https://bugs.documentfoundation.org/show_bug.cgi?id=107087#c16 https://bugs.documentfoundation.org/show_bug.cgi?id=107087#c... "It's challenging to fix as one needs to tunnel certain data through 20 layers of abstraction, but hey, that's improved UX for you!" If you want to get into triaging bugs, I welcome you with open arms. I am desperate to get us more helping hands as after nearly 10000 analysed reports I am anxious to diversify. Here is the index of our QA tech articles: https://wiki.documentfoundation.org/QA https://wiki.documentfoundation.org/QA Basic debugging info for triagers: https://wiki.documentfoundation.org/QA/BugReport/Debug_Information https://wiki.documentfoundation.org/QA/BugReport/Debug_Infor... Debugging for devs: https://wiki.documentfoundation.org/Development/How_to_debug https://wiki.documentfoundation.org/Development/How_to_debug Binary bisecting: https://wiki.documentfoundation.org/QA/Bibisect https://wiki.documentfoundation.org/QA/Bibisect If you were thinking of working on the UI, please get in touch with the design team, which includes some devs (but could really use 10 more!): https://wiki.documentfoundation.org/Design https://wiki.documentfoundation.org/Design You will find me in the QA IRC channel (and most others): https://wiki.documentfoundation.org/QA/IRC https://wiki.documentfoundation.org/QA/IRC I'm heading to sleep now, though, but I hope to see you around!
  - ZenoArrow 9y ago
anotherevan 9y ago
Try doing that with your dynamically typed language.
- Retra 9y ago
  It's really not going to be easy either way.
dingdingdang 9y ago
Personally wish people would stop tinkering with the UI - polish but do not tinker/change needlessly. Case in point being that Linux likely remains fringe as a desktop platform simply because of it's continual tendency to morph on the UI side of things. Then again, Win10 is going down the same route so maybe it will all even out (...).
- aetherspawn 9y ago
  You mix UI and UX polish, and the UX on OpenOffice is honestly not very good ... I only had to use it the other day and struggled with finding half a dozen things that are just obvious with Word.
  - jmiserez 9y ago
    * obvious to Word users For complete novices, yes the UI is not very intuitive. But Word isn’t much better either. I recently had to start using Word again at work and even though the ribbons seem intuitive and user-friendly, actually finding the feature I needed has not always been easy. I’ve had to Google quite a bit (the help is excellent though). At least in LibreOffice everything is always in the menus and does not suddenly disappear if your window is not wide enough.
    - oblio 9y ago
      Office now has a command search box.
  - dingdingdang 9y ago
    I can recommend this read: https://careerfoundry.com/en/blog/ux-design/the-difference-between-ux-and-ui-design-a-laymans-guide/ https://careerfoundry.com/en/blog/ux-design/the-difference-b... UX doesn't enter into my criticism here.
- JetSpiegel 9y ago
  What does Linux (the kernel) have to do with UI? Unless you are creating kernel extensions, you don't directly interact with the Kernel? You mean Gnome 3? There are other desktops.
nshm 9y ago
Michael Meeks is a wonderful guy, love his "Up early..."
gnachman 9y ago
I recall seeing a Google tech talk over ten years ago where they talked about cleaning up OpenOffice (or was it StarOffice?) it was memorable because they mentioned it did #define private public. I’m impressed they kept at it and it sounds like it’s going well. Still glad it’s not my project, though :)
- the-tml 9y ago
  Sounded unfamiliar, so I checked. The Apache OpenOffice sources indeed have three such lines left (all in obscure fairly irrelevant parts of the code; two inside odd #ifdefs that are never true, one inside their OS/2 (yes!) backend). LibreOffice naturally hasn't any such crap any longer.
  - olavk 9y ago
    Do you know why both Apache OpenOffice and LibreOffice still exists?
    - chris_wot 9y ago
      History. Check the Wikipedia article.
gyrgtyn 9y ago
They should just redo it in electron
- thesmallestcat 9y ago
  Inside a container
SubiculumCode 9y ago
Generally I find their suite pretty full features and functional. There are occasional conversion snafus with MS formats, but honestly that is less on them and more on MS. In all honesty, my biggest issue is that Calc can slow to a crawl when a sheet gets even moderately large, especially if using filter/sorting.
- Pelam 9y ago
  Luckily he mentions ongoing multithreading work in the calc :)
- chris_wot 9y ago
  I’ve upvoted you - this sort of feedback is invaluable! Do you have an example spreadsheet? If you could log a bug and upload it, that would be helpful.
  - SubiculumCode 9y ago
    I'll see what I can do. Is there a suggested place to upload?
  - SubiculumCode 9y ago
    Open a LibreOffice Calc doc. Put a 1 in A1. Drag down to a thousand or so. Then drag to the right replicating that series out to say LL. On my decently beefy laptop (i7 quad core, 16 GB RAM, 8 Gb RAM free) it slows to an absolute crawl. On my server (16 dual threaded cores, 64G RAM), it can handle quite a bit more before slowing to a crawl.
    - SubiculumCode 9y ago
      laptop windows. Server Ubuntu
      - chris_wot 9y ago
        Try the bugzilla - http://bugs.documentfoundation.org/ http://bugs.documentfoundation.org/
e943uu9nt 9y ago
I love these sorts of projects and solving these sorts of problems. However, I don't have time to work on an Open Source project, and can't due to employer restrictions anyway. So I get out my inner refactoring by going to the Code Review Stack Exchange [0]. Check it out if you want to do some analysis of a small self-contained piece of code. [0] https://codereview.stackexchange.com https://codereview.stackexchange.com
chris_wot 9y ago
I'm trying to document the internals: https://chrissherlock1.gitbooks.io/inside-libreoffice/content/ https://chrissherlock1.gitbooks.io/inside-libreoffice/conten... I've also been, on the quiet, go through the commits and summarise it better. Before 2009, there was this insane way of grouping commits and it makes it hard to know who did what to the code!
deleted 9y ago
[deleted]
thinkloop 9y ago
Would be nice to restructure the article next to make it readable on mobile.
- fredsted 9y ago
  Reader Mode in Safari works well on this.
mscppcon24 9y ago
This reminds me of a presentation[0] from Microsoft at CPPCon about using the same giant ancient Office codebase on iOS and Android as on Windows. 0. https://cppcon.org/bonus-talk-cxx-in-ms-office-2014/ https://cppcon.org/bonus-talk-cxx-in-ms-office-2014/
ygh0309 9y ago
Error:Execution failed for task ':createStrippedConfigMain'. > A problem occurred starting process 'command 'D:/android/mobile-config.py''
- buovjaga 9y ago
  Are you trying to build the Android Viewer on Windows? I believe no one has ever done that, so I recommend you to stick with Linux: https://wiki.documentfoundation.org/Development/BuildingForAndroid https://wiki.documentfoundation.org/Development/BuildingForA...

489 points by buovjaga 9y ago

eloy 9y ago

I'm quite surprised that the markup incompatibility with Microsoft Office is still an issue. Are they actively sabotaging the Open Document Foundation, or is it just really difficult to achieve?

tadfisher 9y ago
I'm not surprised at all that someone on HN replies to a positive story about some open-source product with unrelated bug complaints, then implies the bugs are due to malice on the part of the developers.
- TomMarius 9y ago
  I think he meant Microsoft...
  - tadfisher 9y ago
    d'oh! I'll leave my comment up as a warning to others to reread carefully before getting annoyed.
- cabaalis 9y ago
  My worst downvotes have been when I'm referring directly to the motivations or knowledge of the commenter. My second-worst downvotes are when I'm opining on something I know nothing about. My suggestion (to myself, and anyone else interested) is to avoid these two pitfalls.
Lordarminius 9y ago
::Are they actively sabotaging the Open Document Foundation, or is it just really difficult to achieve? The former is a more likely scenario
beamatronic 9y ago
Is compatibility more of a problem with .doc or .docx? .doc shouldn't still be evolving, I'd imagine.
- _delirium 9y ago
  .doc is more stable, but it's also much more onerous to be fully compatible with. .docx was at least designed as a reasonably modern, rational document format with a specification. .doc by contrast evolved from generations of MS Word that for efficiency saved files by doing raw dumps of internal data structures as binary blobs, which got more and more complicated as things accreted over the years and versions. And there was no public documentation of the very complex binary format until 2008. Joel Spolsky has some on that here: https://www.joelonsoftware.com/2008/02/19/why-are-the-microsoft-office-file-formats-so-complicated-and-some-workarounds/ https://www.joelonsoftware.com/2008/02/19/why-are-the-micros...
  - nradov 9y ago
    The JPEG file formats are nearly as bad, especially once you get into camera vendor extensions. It was common practice back then. Just take a bunch of C structs and dump them from memory to disk.
buovjaga 9y ago
I don't think they need to actively sabotage anything and I would not want to dwell on those lines of thinking. I think a large part of the open issues are in corner case territory or due to the freedom of users to go wild with their document layouts, creating a mess of nested frames in tables etc. Let's roll up our sleeves and fix them anyhow :) !! Here is our meta report for MSO formats: https://bugs.documentfoundation.org/showdependencytree.cgi?id=107585&hide_resolved=1 https://bugs.documentfoundation.org/showdependencytree.cgi?i... "depends on 1394 open bugs" - this includes all the further meta reports.
kjhughes 9y ago
The OOXML specification is over 5K pages long. Even Microsoft's own online products do not implement it entirely. One should not be surprised that outside parties may face challenges achieving 100% implementation and interoperability.
- merb 9y ago
  and even word/excel has discrepancies between 2003, 2011, 2013 and 2016 especially between 2011 and 2013. Mostly overlapping layouting is differently.
pbhjpbhj 9y ago
Have Microsoft completely solved compatibility between different/same versions of MS Office now?
- pletnes 9y ago
  Nope, I ran into word documents that word could not render just last month. And I hardly use word; I edit about 1-5 docs per month.
- Spooky23 9y ago
  It took my enterprise almost a year to iron out the issues to move from 2010/2013 to ProPlus/2016.
readams 9y ago
Realistically LibreOffice is not really relevant competition for Microsoft. Google Docs is much more important, even though the capabilities are so widely disparate.
- adrianN 9y ago
  I guess that depends on your filter bubble. I know many more people who use LibreOffice compared to Google Docs. Most people I know don't trust the cloud with their important documents.
  - ozim 9y ago
    I do not trust cloud even with unimportant documents. Especially if you can be locked out of your cloud account without reason. I only keep non critical offsite backups on cloud drives.
mikekchar 9y ago
A very long time ago, I used to work on document importing and exporting on Word Perfect so I'm relatively familiar with the problem in general. I haven't looked at this stuff in almost 20 years, so I have no idea about specifics at this point :-) What people generally don't understand is that the save file formats do not describe what the output will look like. Back in the old, old, old days you would write a word processor and to "save" the file, you would just write a binary image of your data structures. That was what the doc file format was originally. Although, people didn't tend to program this way back then, imagine a Rails server. Now imagine that you handed over your database (with tables that correspond to each model object). That's kind of analogous to a "save" file. In order to render the data, you need the business logic (which, in Rails, is usually housed in the controllers) and you need the rendering logic (which is usually housed in the views). Just having a description of the database tables doesn't really help me all that much in rendering the data as you can see. Now imagine that we take a different Rails app that happens to do the same kind of thing. Some people say, "Oh it would be swell if you could import the data from the first Rails app and render it in yours. You are basically doing the same app, so it should be easy". But it's obviously not -- I wrote my controllers and views completely differently than the other Rails app. My model objects contain completely different data and are structured completely differently. Even worse -- I don't even render things the same way. If I want to support this functionality, I basically need to completely rewrite my rails app to be exactly the same as the previous Rails app. And this is essentially the problem. If you want to render Word documents the way Word does, you basically have to rewrite Word in your word processor. If you also want to support your format then you have to maintain 2 word processors. If you want to supports some other file formats? It just gets worse and worse. So what you could do is completely rewrite the rendering engine so that it is more flexible. But the problem is that this is a massive undertaking and nobody will believe you if you tell them that it is necessary. Even worse, there is a surprising amount of variation in rendering. For example, how do you deal with page breaks in footnotes? Most people don't write footnotes that are longer than a page (if you look at my posting history, you might suspect that I'm one of those people, but I digress). However, it's very common in the legal field. There is a specific correct way to break pages in footnotes and Word historically has not done it correctly. They do it differently. This is actually what kept Corel/Word Perfect in business for a long time -- in order to print the document, you would have to use Word Perfect and since the file conversion was crap, you pretty much had to keep it as a Word Perfect file forever. I will say that while I worked on Word Perfect, Microsoft was very helpful in explaining how their formatter worked. They even regularly sent us bug reports when our import filters had regressions, etc. I've never believed that they intentionally obfuscated the process. It's just a difficult problem.
kakwa_ 9y ago
It's really difficult to achieve. The article mentions EMF/EMF+. this format is basically a list of calls to GDI.h and is a bitch to map to other graphic stacks when you are not on Windows. The specification of the format is public (kudos for Microsoft), it helps a lot. But there are a lot of corner cases and the spec can be quite hard to understand some times. Computing correctly the origin is tricky for example (VIEWPORTORGEX, WINDOWORGEX etc). For some stuff, it's impossible to get it right. One example that comes to my mind is a text encoding bug I got a year ago (I maintain an emf to svg conversion library). It took me one week to track it down. In most cases the strings are UTF-16. But in weird cases, when the ETO_GLYPH_INDEX flag is set, the "encoding" is directly the index of the glyph inside the selected font. It's not the most portable way to handle text... If you don't have the exact same font on your computer, there is a good chance the text will not be displayed correctly. And converting back to a well known encoding is tricky (using the cmap tables of the ttf file, you have to build a reverse cmap, pray there is a 1-1 mapping between glyph and unicode, and convert back to utf-8 or something). Even Microsoft itself, on the Mac OS, version got it wrong.
- chris_wot 9y ago
  Can you link to that library?
  - buovjaga 9y ago
    I found it in his submission history: https://github.com/kakwa/libemf2svg https://github.com/kakwa/libemf2svg
    - kakwa_ 9y ago
      thanks ^^ For information, the bits of code that is handling "font index encoding": https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2svg_utils.c#L1012 https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2... To https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2svg_utils.c#L1278 https://github.com/kakwa/libemf2svg/blob/master/src/lib/emf2...

niftich 9y ago

He has a slide deck [1] that he links to in the interview, where he tries to highlight some challenges with grokking large codebases, and techniques one can use to compartmentalize understanding and build from there. I found it both a bit too colloquial yet also too opaque, but there's some useful nuggets in there. It also speaks to a significant problem: large codebases are inherently difficult to understand, which forms a barrier to entry. This is despite the fact that most software makes use of various abstractions to break problems into smaller parts. In the general sense, I believe it's worth thinking about how the particular abstraction mechanisms used in software impact a new contributor's productivity, vs. the productivity of someone more familiar with the codebase, especially for projects that are developed in the open, or with high developer turnover. [1] https://people.gnome.org/~michael/data/2016-04-29-solving-problems.pdf https://people.gnome.org/~michael/data/2016-04-29-solving-pr...

jandrese 9y ago
I've always found abstraction to be a double edged sword when trying to understand codebases. Sometimes it helps define the intent, but other times I'm going "WTF is this thing for real?" and tracking the abstractions through several files. My favorite is tracking down a member function and being completely confused until I realize that I missed one of the layers and it was overridden in that instantiation.
- sbov 9y ago
  We tend to speak of abstractions in only positive terms. Some negative ones could include opaque, obtuse, disjointed, etc.
  - Gibbon1 9y ago
    When I think of abstractions I think abstraction == API and good API design is very hard.
- flukus 9y ago
  I generally find that abstraction is good, but layering is bad. Let's say I have components A, B, and C that all do something useful and are good abstractions, the difference between good code and bade code is how they interact. In bad code A will call B and B will call C, from the top level it's completely opaque how everything fits together. I'd much rather A returns something and let's the caller pass it to B. There is very little difference in code required but it makes a world of difference to making it comprehensible.

ktpsns 9y ago

It's so amazing that the dinosaur LibreOffice/OpenOffice is still maintained. From the codebase, it is probably similarly old-fashioned like Thunderbird. Some folks may say it is hopeless to modernize. For me it does not even matter whether that's true or not -- it is a well-done and partially timeless and universal tool in my everyday work flow. While there are fancy and modern alternatives like the (closed source) Google Docs, LibreOffice always stays as a fall-back solution if nothing more works. Thank you for this work, developers!

draw_down 9y ago
Well, yeah, users typically don't care much about the code of something they use at all. Not that it should be otherwise but it's not exactly a new sentiment.
dfox 9y ago
The most crufty part in both mozilla's code base and libreoffice code base is essentially equivalent in that it tries to solve essentially same set of problems (cross-platform UI and object model), but the implementation is significantly different, because in mozilla's case this layer of cruft was introduced 20-ish years ago as a complete rewrite of the thing (and most of the really ugly hacks have to do with sub-par C++ compilers and support for classic MacOS), while there was no attempt at a major rewrite of the staroffice codebase until libreoffice (and it is quite obvious that the way libreoffice's rewrite proceeds is at least partially motivated by experience from how the netscape->mozilla rewrite suffered heavily from second system effect and in the end failed at least from the bussines standpoint).
- ktpsns 9y ago
  Good point. Indeed it is interesting to follow the Firefox Quantum approach (which breaks with XUL) or KDE's open source office suite, aka Calligra (https://www.calligra.org/ https://www.calligra.org/) which is "plain C++/Qt" and thus not a sole effort to solve the cross-platform UI/object model problem such as Mozilla and StarOffice did in their time. Thanks to this, Calligra devs can probably concentrate much more on implementing features while getting a modern cross-plattform UI for free (literally: Qt is free).

ZenoArrow 9y ago

Thank you for sharing this article. Speaking personally, it comes at a great time. I've been toying with the idea of getting involved in open source, and was particularly interested in pushing forward open source office suites. Something that I'm particularly interested in with LibreOffice is helping the core being decoupled from the UI, to both help with portability and to speed up the rate of UI innovation. I had thought that I may have to start by myself, but based on this article it seems that the current LibreOffice developers are thinking along similar lines. If any LibreOffice developers are reading this, are there any sub teams setup that may be a good fit? To give some idea of my background, I would describe myself as a tester that knows how to code (not in C++ alas, but I'm familiar enough with debugging at a low level).

buovjaga 9y ago
Regarding the UI, there are certainly some... curiosities. To give you a random anecdote from the trenches: https://bugs.documentfoundation.org/show_bug.cgi?id=107087#c16 https://bugs.documentfoundation.org/show_bug.cgi?id=107087#c... "It's challenging to fix as one needs to tunnel certain data through 20 layers of abstraction, but hey, that's improved UX for you!" If you want to get into triaging bugs, I welcome you with open arms. I am desperate to get us more helping hands as after nearly 10000 analysed reports I am anxious to diversify. Here is the index of our QA tech articles: https://wiki.documentfoundation.org/QA https://wiki.documentfoundation.org/QA Basic debugging info for triagers: https://wiki.documentfoundation.org/QA/BugReport/Debug_Information https://wiki.documentfoundation.org/QA/BugReport/Debug_Infor... Debugging for devs: https://wiki.documentfoundation.org/Development/How_to_debug https://wiki.documentfoundation.org/Development/How_to_debug Binary bisecting: https://wiki.documentfoundation.org/QA/Bibisect https://wiki.documentfoundation.org/QA/Bibisect If you were thinking of working on the UI, please get in touch with the design team, which includes some devs (but could really use 10 more!): https://wiki.documentfoundation.org/Design https://wiki.documentfoundation.org/Design You will find me in the QA IRC channel (and most others): https://wiki.documentfoundation.org/QA/IRC https://wiki.documentfoundation.org/QA/IRC I'm heading to sleep now, though, but I hope to see you around!
- ZenoArrow 9y ago

anotherevan 9y ago

Try doing that with your dynamically typed language.

Retra 9y ago
It's really not going to be easy either way.

dingdingdang 9y ago

Personally wish people would stop tinkering with the UI - polish but do not tinker/change needlessly. Case in point being that Linux likely remains fringe as a desktop platform simply because of it's continual tendency to morph on the UI side of things. Then again, Win10 is going down the same route so maybe it will all even out (...).

aetherspawn 9y ago
You mix UI and UX polish, and the UX on OpenOffice is honestly not very good ... I only had to use it the other day and struggled with finding half a dozen things that are just obvious with Word.
- jmiserez 9y ago
  * obvious to Word users For complete novices, yes the UI is not very intuitive. But Word isn’t much better either. I recently had to start using Word again at work and even though the ribbons seem intuitive and user-friendly, actually finding the feature I needed has not always been easy. I’ve had to Google quite a bit (the help is excellent though). At least in LibreOffice everything is always in the menus and does not suddenly disappear if your window is not wide enough.
  - oblio 9y ago
    Office now has a command search box.
- dingdingdang 9y ago
  I can recommend this read: https://careerfoundry.com/en/blog/ux-design/the-difference-between-ux-and-ui-design-a-laymans-guide/ https://careerfoundry.com/en/blog/ux-design/the-difference-b... UX doesn't enter into my criticism here.
JetSpiegel 9y ago
What does Linux (the kernel) have to do with UI? Unless you are creating kernel extensions, you don't directly interact with the Kernel? You mean Gnome 3? There are other desktops.

nshm 9y ago

Michael Meeks is a wonderful guy, love his "Up early..."

gnachman 9y ago

I recall seeing a Google tech talk over ten years ago where they talked about cleaning up OpenOffice (or was it StarOffice?) it was memorable because they mentioned it did #define private public. I’m impressed they kept at it and it sounds like it’s going well. Still glad it’s not my project, though :)

the-tml 9y ago
Sounded unfamiliar, so I checked. The Apache OpenOffice sources indeed have three such lines left (all in obscure fairly irrelevant parts of the code; two inside odd #ifdefs that are never true, one inside their OS/2 (yes!) backend). LibreOffice naturally hasn't any such crap any longer.
- olavk 9y ago
  Do you know why both Apache OpenOffice and LibreOffice still exists?
  - chris_wot 9y ago
    History. Check the Wikipedia article.

gyrgtyn 9y ago

They should just redo it in electron

thesmallestcat 9y ago
Inside a container

SubiculumCode 9y ago

Generally I find their suite pretty full features and functional. There are occasional conversion snafus with MS formats, but honestly that is less on them and more on MS. In all honesty, my biggest issue is that Calc can slow to a crawl when a sheet gets even moderately large, especially if using filter/sorting.

Pelam 9y ago
Luckily he mentions ongoing multithreading work in the calc :)
chris_wot 9y ago
I’ve upvoted you - this sort of feedback is invaluable! Do you have an example spreadsheet? If you could log a bug and upload it, that would be helpful.
- SubiculumCode 9y ago
  I'll see what I can do. Is there a suggested place to upload?
- SubiculumCode 9y ago
  Open a LibreOffice Calc doc. Put a 1 in A1. Drag down to a thousand or so. Then drag to the right replicating that series out to say LL. On my decently beefy laptop (i7 quad core, 16 GB RAM, 8 Gb RAM free) it slows to an absolute crawl. On my server (16 dual threaded cores, 64G RAM), it can handle quite a bit more before slowing to a crawl.
  - SubiculumCode 9y ago
    laptop windows. Server Ubuntu
    - chris_wot 9y ago
      Try the bugzilla - http://bugs.documentfoundation.org/ http://bugs.documentfoundation.org/

e943uu9nt 9y ago

I love these sorts of projects and solving these sorts of problems. However, I don't have time to work on an Open Source project, and can't due to employer restrictions anyway. So I get out my inner refactoring by going to the Code Review Stack Exchange [0]. Check it out if you want to do some analysis of a small self-contained piece of code. [0] https://codereview.stackexchange.com https://codereview.stackexchange.com

chris_wot 9y ago

I'm trying to document the internals: https://chrissherlock1.gitbooks.io/inside-libreoffice/content/ https://chrissherlock1.gitbooks.io/inside-libreoffice/conten... I've also been, on the quiet, go through the commits and summarise it better. Before 2009, there was this insane way of grouping commits and it makes it hard to know who did what to the code!

deleted 9y ago

[deleted]

thinkloop 9y ago

Would be nice to restructure the article next to make it readable on mobile.

fredsted 9y ago
Reader Mode in Safari works well on this.

mscppcon24 9y ago

This reminds me of a presentation[0] from Microsoft at CPPCon about using the same giant ancient Office codebase on iOS and Android as on Windows. 0. https://cppcon.org/bonus-talk-cxx-in-ms-office-2014/ https://cppcon.org/bonus-talk-cxx-in-ms-office-2014/

ygh0309 9y ago

Error:Execution failed for task ':createStrippedConfigMain'. > A problem occurred starting process 'command 'D:/android/mobile-config.py''

buovjaga 9y ago
Are you trying to build the Android Viewer on Windows? I believe no one has ever done that, so I recommend you to stick with Linux: https://wiki.documentfoundation.org/Development/BuildingForAndroid https://wiki.documentfoundation.org/Development/BuildingForA...