5 ms·
What about an SQLite database for configuration? For most applications it would probably be overkill, but it seems useful for applications that need to change
by Tabular-Iceberg 6y ago
What about an SQLite database for configuration?
For most applications it would probably be overkill, but it seems useful for applications that need to change their own configuration (e.g. by the user through its GUI), or need its configuration changed by other tools.
- wirrbel 6y agoLet’s use a key value store and call it the configuration registry!
- AnonymousPlanet 6y agoYes! And then let's have anyone put their shit in there without order. So you have config and runtime data in the same area. Have multiple processes change the same value at will. Except for anything larger than a few Bytes. That we just disperse across like 10 possible locations on all hard drives!
- datavirtue 6y agoI use a code first approach. I build my config as plain old objects and serialize/deserialize them (JSON). 100% flexible and very easy to grok. New devs look at it and just know what is going on, immediately.
- jrochkind1 6y agoI want my configuration to be easily diffable by git.
- tzs 6y agoI'm fine with having a program that transforms the configuration file into a more diffable format, and then doing my diffs on the transformed files. I like for this a sorted name=value format, where non-scalars such as arrays and hashes are flattened into the names. E.g., if the config contains an array named "users" with 3 items, the name=value pairs would flatten to names users.0, users.1, and users.2. An array named "servers" whose entries are 3 hashes with the host name and port would flatten to names servers.0.host, servers.0.port, servers.1.host, servers.1.port, servers.2.host, servers.2.port. That gives you diffs that tell you what has actually changed in the configuration itself rather than in the formatting of the configuration file.
- brandmeyer 6y agoIntermediate solution: SQL command text that generates the configuration file. Command text goes into source control, and the database it generates gets deployed. cat my_configuration_text.sql | sqlite3 my_configuration_database.sqlite Closely related, sqldiff ships with sqlite. It generates the difference between sqlite database files as a bunch of {insert,update,delete} statements.
- jrochkind1 6y agoOK, but... why? Do you really have so much configuration that you can't just store it in memory on boot? Or is it for another reason?
- brandmeyer 6y agoIts not about the quantity. For my application the key driver was incremental remote update. If your deployment model is that 100% of the application configuration is linked into the container image, then I can see how flat files sound appealing. I completely agree with Dan Luu about the perils of files, though[1]. My application is a remote sensor that both requires incremental update, and is large enough to justify SQLite over something smaller. 1: https://danluu.com/deconstruct-files/ https://danluu.com/deconstruct-files/
- jrochkind1 6y agoThanks, makes sense! I don't think much of that Dan Luu article applies to read-only files deployed with the application, which is often what configuration files are. Definitely concurrency control is an issue with mutable files, that would all by itself make me hesitant to use them as a storage solution if that were in play. But it's not when the changes to the files are being done outside of the app being deployed, and they are deployed as read-only, any changes require app restart to pickup etc. That is how I have usually experienced configuration and I have personally never run into any problem with having it in the file system. Most of the other stuff in the article also seems to me not to apply to the standard configuration use case. (And if it did... it would apply to your source files too, right? Whether ruby or python or even JVM bytecode. Yet we obviously can and do put those in the filesystem. Ultimately read-only configuration are just another kind of source file they don't really have any special problems). But "incremental remote update" (without requiring app restart especially!) is definitely not the standard configuration use case. I agree that a 'real database' seems reasonable for that use case, whether sqlite3 or something else. Whether you try to control the configuration (that will wind up in a db) in your version control system, or just use standard db backup/clone techniques instead.
- mmcdermott 6y agoI have definitely seen some tables + SQL used for hierarchical configuration spread across artifacts/modules. My guess is that this has probably seen more deployment than is at first suspected.