5 ms·

Thats a rather bizarre way to store config data I can't see any architectural reason why this would be desirable, does anyone have any insight into this?

vrotaru 15y ago
As far as I can tell the official reason for switching from gcong to gsetting is that the new backend is optimized for fast reading of config values. As for the old schema, there were two options separate xml files in a directory per-app and a big consolidated xml file. There were some measuring showing that the second variant which was opening just a file and not 50 was a lot faster and considerably reduced startup times. But now if you are going to put everything in a big pile of xml where it is hard to find and edit/remove/add options without a specialized tool you can as well use a binary format. At least, that's my reconstruction of events.
- rbanffy 15y ago
  So, the config system is slowly approaching a Windows registry... How about making opening the 50 or so files fast instead of putting everything into one big file and, then, into one smaller blob? Or we actually do like to make the same mistakes Microsoft does?
  - jerf 15y ago
    Relevent, though I don't know what side I come down on: http://blogs.msdn.com/b/oldnewthing/archive/2007/11/26/6523907.aspx http://blogs.msdn.com/b/oldnewthing/archive/2007/11/26/65239... Sometimes pathological problems result in pathological solutions and I'm not going to step up and try to identify which spawned which. Goodness knows the happy UNIX story of happy text files configuring your happy isolated services has all kinds of horrific exceptions that don't appear in the happy part of the story, like Sendmail or .emacs.
  - vrotaru 15y ago
    > So, the config system is slowly approaching a Windows registry... The config system may have been designed with Windows registry in mind, because Miguel de Icaza, the leader of Gnome project back then Miguel de Icaza (of Mono fame), was copying windows, and gconf-editor is a _registry editor_. As whether they'll make the same/worse/better mistake as Microsoft did, you guess is as good as mine.
- devs1010 15y ago
  Interesting, thanks for the info.. that said, you can still obviously read xml files into memory and hold them there if speed is the issue, I think its still easier to edit XML files rather than however its done with a BLOB (I've only used blobs myself when programming a Java app and persisting Java objects to a DB so maybe I don't have the full picture), anyways if they did that with XML they just need to expose mechanism to reload XML files after changes are made (to avoid needing to do a reboot, like Windows)
wlach 15y ago
Performance and footprint. If I remember correctly, the binary format is used so that dconf can mmap the configuration into memory. This means that a read on the configuration database doesn't involve any system calls and only one copy needs to be in memory (no matter how many processes are using it). http://live.gnome.org/dconf#Design http://live.gnome.org/dconf#Design I'm generally a fan of text-based configuration files but in this case I think the benefit exceeds the cost. Reading configuration used to take a non-trivial amount of time on startup.
- cpeterso 15y ago
  Is reading configuration files really such a slow yet common operation that it is a real performance bottleneck? If a GNOME login requires "thousands of dconf reads", perhaps they should modularize and lazy-load some of those settings. (Or not use IPC to query a config service.)
- dredmorbius 15y ago
  The proper way to do this is to maintain a text file for reference, check it for modifications, and keep the machine-preferred format elsewhere. Best of both worlds: rapid parsing / memory management (really, does it matter that much), while preserving human management. Oh yeah: and the textual version is definitive.
- rbanffy 15y ago
  > Performance and footprint One way out would be to compile the text-based config files into the config blob and rebuild the blob if the text-based config files change. I can't quite figure out an attack vector, but I always have a bad feeling when I trust a binary blob and load it into memory without any parsing.
- Zak 15y ago
  For some reason, gnome-settings-daemon reads ~/.thumbnails on login, and ~/.thumbnails was unbounded in size. Mine was 700M when I learned what caused all the disk IO. I think a lot of the perception of text config files being slow may have more to do with that than the actual performance of the configuration system. The middle ground would be to cache settings in a binary file.
mhw 15y ago
I don't know the precise reasons for the Gnome developers choosing to go with a binary format, but there are some fundamental (and well understood) constraints that you're working within. First of all, can we just take it as read that the majority of users will prefer to modify the configuration of their GUI using a GUI? This means that your application needs to be able to both read and write the configuration file format. Now, if your configuration file is textual, power users will want to edit it by hand. Look at the rest of this thread for evidence. This means that your application needs to be (more) robust in the face of errors in the configuration file syntax. Not insurmountable, but it's a bit more work. It would be good if your application recognises when the configuration file has been altered too, and reread the configuration so people don't have to restart the application. Power users will also want to version control the configuration file, and compare different revisions to see what has changed. This means that when you write out the configuration file you need to preserve the textual structure and ordering of the elements from when you read it in. The power user reordered the settings to group things in away they consider more logical? Your configuration file editor needs to preserve that. Most plain text configuration file formats also have some way of adding comments, and you'll need to preserve that. Oh, and all that whitespace that's not significant to your program but is significant to your typical power user. So the problem is pretty much comparable with building a syntax-aware text editor for programming languages. There was loads of research done on those (Google 'syntax directed editor') and their success can be measured by the fact that lots of programmers still choose vim, emacs, textmate and other plain text editors precisely because they don't try to do anything clever with your text. Eclipse and similar IDEs are probably the closest you get to a syntax directed editor these days, and they've still stuck to the plain text editor user interface. More sophisticated attempts to round-trip from program source code to editable UML diagrams or GUI layouts and back are routinely laughed at by the power users that you're hoping to win over with your clever plain text configuration file system, so don't think they'll be an easy audience to please either...

vrotaru 15y ago

As far as I can tell the official reason for switching from gcong to gsetting is that the new backend is optimized for fast reading of config values. As for the old schema, there were two options separate xml files in a directory per-app and a big consolidated xml file. There were some measuring showing that the second variant which was opening just a file and not 50 was a lot faster and considerably reduced startup times. But now if you are going to put everything in a big pile of xml where it is hard to find and edit/remove/add options without a specialized tool you can as well use a binary format. At least, that's my reconstruction of events.

rbanffy 15y ago
So, the config system is slowly approaching a Windows registry... How about making opening the 50 or so files fast instead of putting everything into one big file and, then, into one smaller blob? Or we actually do like to make the same mistakes Microsoft does?
- jerf 15y ago
  Relevent, though I don't know what side I come down on: http://blogs.msdn.com/b/oldnewthing/archive/2007/11/26/6523907.aspx http://blogs.msdn.com/b/oldnewthing/archive/2007/11/26/65239... Sometimes pathological problems result in pathological solutions and I'm not going to step up and try to identify which spawned which. Goodness knows the happy UNIX story of happy text files configuring your happy isolated services has all kinds of horrific exceptions that don't appear in the happy part of the story, like Sendmail or .emacs.
- vrotaru 15y ago
  > So, the config system is slowly approaching a Windows registry... The config system may have been designed with Windows registry in mind, because Miguel de Icaza, the leader of Gnome project back then Miguel de Icaza (of Mono fame), was copying windows, and gconf-editor is a _registry editor_. As whether they'll make the same/worse/better mistake as Microsoft did, you guess is as good as mine.
devs1010 15y ago
Interesting, thanks for the info.. that said, you can still obviously read xml files into memory and hold them there if speed is the issue, I think its still easier to edit XML files rather than however its done with a BLOB (I've only used blobs myself when programming a Java app and persisting Java objects to a DB so maybe I don't have the full picture), anyways if they did that with XML they just need to expose mechanism to reload XML files after changes are made (to avoid needing to do a reboot, like Windows)

wlach 15y ago

Performance and footprint. If I remember correctly, the binary format is used so that dconf can mmap the configuration into memory. This means that a read on the configuration database doesn't involve any system calls and only one copy needs to be in memory (no matter how many processes are using it). http://live.gnome.org/dconf#Design http://live.gnome.org/dconf#Design I'm generally a fan of text-based configuration files but in this case I think the benefit exceeds the cost. Reading configuration used to take a non-trivial amount of time on startup.

cpeterso 15y ago
Is reading configuration files really such a slow yet common operation that it is a real performance bottleneck? If a GNOME login requires "thousands of dconf reads", perhaps they should modularize and lazy-load some of those settings. (Or not use IPC to query a config service.)
dredmorbius 15y ago
The proper way to do this is to maintain a text file for reference, check it for modifications, and keep the machine-preferred format elsewhere. Best of both worlds: rapid parsing / memory management (really, does it matter that much), while preserving human management. Oh yeah: and the textual version is definitive.
rbanffy 15y ago
> Performance and footprint One way out would be to compile the text-based config files into the config blob and rebuild the blob if the text-based config files change. I can't quite figure out an attack vector, but I always have a bad feeling when I trust a binary blob and load it into memory without any parsing.
Zak 15y ago
For some reason, gnome-settings-daemon reads ~/.thumbnails on login, and ~/.thumbnails was unbounded in size. Mine was 700M when I learned what caused all the disk IO. I think a lot of the perception of text config files being slow may have more to do with that than the actual performance of the configuration system. The middle ground would be to cache settings in a binary file.

mhw 15y ago

I don't know the precise reasons for the Gnome developers choosing to go with a binary format, but there are some fundamental (and well understood) constraints that you're working within. First of all, can we just take it as read that the majority of users will prefer to modify the configuration of their GUI using a GUI? This means that your application needs to be able to both read and write the configuration file format. Now, if your configuration file is textual, power users will want to edit it by hand. Look at the rest of this thread for evidence. This means that your application needs to be (more) robust in the face of errors in the configuration file syntax. Not insurmountable, but it's a bit more work. It would be good if your application recognises when the configuration file has been altered too, and reread the configuration so people don't have to restart the application. Power users will also want to version control the configuration file, and compare different revisions to see what has changed. This means that when you write out the configuration file you need to preserve the textual structure and ordering of the elements from when you read it in. The power user reordered the settings to group things in away they consider more logical? Your configuration file editor needs to preserve that. Most plain text configuration file formats also have some way of adding comments, and you'll need to preserve that. Oh, and all that whitespace that's not significant to your program but is significant to your typical power user. So the problem is pretty much comparable with building a syntax-aware text editor for programming languages. There was loads of research done on those (Google 'syntax directed editor') and their success can be measured by the fact that lots of programmers still choose vim, emacs, textmate and other plain text editors precisely because they don't try to do anything clever with your text. Eclipse and similar IDEs are probably the closest you get to a syntax directed editor these days, and they've still stuck to the plain text editor user interface. More sophisticated attempts to round-trip from program source code to editable UML diagrams or GUI layouts and back are routinely laughed at by the power users that you're hoping to win over with your clever plain text configuration file system, so don't think they'll be an easy audience to please either...