6 ms·
Why Mongrel2 Doesn't Use INI Files (Zed Shaw)
- thefreshteapot 16y agoFor a simple site, this maybe overkill. Yet running a development setup on a VM and having live setups. ( A little bit like that the deviant guys wrote about http://news.ycombinator.com/item?id=1769637 http://news.ycombinator.com/item?id=1769637 ) I can see sqlite data being very handy. You could give each their own respective id and then just pull out all the records for them.
- zedshaw 16y agoI think you win a prize for seeing into the future.
- mfukar 16y agoI suppose the next question will be "But Zed, why sqlite3?". Good luck. :-)
- zedshaw 16y agoAlready get those. There's folks who want to use redis, couchdb, just about anything that can store configurations. I think the thing most existing sysadmins don't get is that this is written for people who have to manage mongrel2 servers in a modern way. It doesn't prevent you from doing your usual manual ssh work, but it's aimed at the future where people won't be doing that so much. It's kind of sad, because I want sysadmins to learn to code so bad, and I even wrote a book so they could learn it, and have helped tons of them get better jobs, yet they still resist awesome features like this. They're really just holding themselves back by not realizing automation and code are their future.
- danudey 16y agoI don't mean to nitpick, but the sysadmins you're talking about are the bad sysadmins (or maybe just mediocre sysadmins). Any sysadmin worth his salt knows that when you automate something, you only have to do it once, and you remove any mistakes down the road. Once you have the script written and tested, you don't have to worry about typos ever again. If it weren't for automation, my job would be unbearable, but writing scripts to automate systems is the most fun I get to have in my day job.
- tef_ 16y agoUsing the sqlite db as the canonical configuration is a nice idea, but reminds me of the fun of bind, editing one file to re-generate another. It is as if he is ignoring the advice in his older post about making software usable. Instead of having a simple text format that is the configuration, he encourages everyone to write their own. I can imagine the fun now on a mailing list - debugging n+1 languages to start a webserver. When you deploy, do you deploy the sqlite database file or the script that generates it? Does the daemon check if the db is out of date on startup? Does the script clear the existing db when it runs? Does it have merge logic? How do you keep them in sync? 'But it's powerful' I hear you cry, but my retort is simply 'it is effort to maintain, deploy and configure'. It smacks of: here is a bus port, go build yourself a keyboard. This sort of toolsmithery is what brings us delights like autoconf. A simple change would improve things a lot, whilst maintaining the flexibility he craves: instead of 'generate a config file' with your tool and hand it off to mongrel, the tool you write is part of mongrel. You write a script called /etc/mongrel/makeconfig, that takes one argument, the name of the mongrel settings database. When mongrel starts, it invokes makeconfig to build the database. You can provide a bunch of trivial makeconfig scripts as defaults, including a shell script that just runs a bunch of sqlite statements.
- cdavid 16y agoAgreed, the potential issue of having different people having different configuration sounds a bit dangerous. OTOH, most issues you mention are true whether you use a db or a plain configuration file - they are just more explicit with sqlite3 (which acts as the model in a MVC organization of your app settings). Incidentally, I am thinking quite hard about those issues (configuration representation, live change/querying, versioing) for internal apps, it is not an easy issue once your app has more than a few parameters.
- zedshaw 16y ago> Instead of having a simple text format that is the configuration, he encourages everyone to write their own. Uh no, you obviously didn't read the article where at the end I show you the default format we provide to you in m2sh. I think before you go waxing poetical about something you should learn more about it. And it's not powerful, it's simple. Take a look at the code that loads the configuration in mongrel2 from sqlite, and compare that to similar C code (also written by me) in the m2sh source to just parse a config file. Code doesn't lie and in this case, parsing a config is a hell of a lot harder than just loading it out of a sqlite file.
- pablohoffman 16y agoIs it me, or Zed is starting to get annoyed at the Python community now?. I wonder when he'll write "Python is a Ghetto".
- danudey 16y agoI haven't seen the Python community complaining about the config methodology. It usually just seems to be the people who can't understand new things that complain about change. As someone who would be managing this in production, I welcome it.
- telemachos 16y agoNo opinion one way or another about how Mongrel2 handles configuration, but are INI files really "the fad today"? Just strikes me as an odd insult for something so old. (I don't work with Python, so maybe that's the faddish part of all this?)
- maximilianburke 16y agoA technology doesn't have to be recent to still be applicable. INI files have been around a long time but they are a well understood mechanism for storing confuguration settings and have mature, well debugged, components for handling them. Not that I don't think that sqlite isn't mature but it seems like overkill for storing simple configuration settings.
- zedshaw 16y agoI usually find that when people request something weird, it's because they have a "special sauce" idea they want to implement and not tell me about. As for sqlite3 being overkill, go check out the code for loading the config with sqlite, and for just parsing the config in m2sh. Parsing is way harder and more overkill than just querying a db.
- telemachos 16y agoYou misunderstood my point (maybe my post was too short). I wasn't complaining that the INI format was too old. I just found it odd that Zed's post began by making broad jokes about fads and "the future" and then went on to talk about INI files. I didn't think INI-style configuration was a current fad - though now I wonder if it is among Python devs.
- tvon 16y agoYeah, I thought everyone was still all hot for yaml.
- tptacek 16y agoBad call. The canonical version of the server configuration doesn't play nice with version control, and so staff has to work with a facsimile and hope that nobody makes out-of-process changes. (INI files though? What year is it?)
- gaius 16y agoBecause programmers are expensive, but sysadmins are free. If you're a programmer working for a large organization, despite sitting in an open plan office surrounded by dozens of other programmers, and knowing that there are hundreds in the building, you probably nevertheless believe that yours is the only code running in production, and the entire operations staff exists in order to support it...
- cdavid 16y agoI don't follow: surely, a sql dump is possible with sqlite, and you could use just that in your VCS ? As for ini file, the format is supported in the python stdlib, which makes it a natural default if you don't need too much. If .ini is not enough, I think in general you need something with a bit some logic, at which step using the database itself makes sense in mongrel2 case.
- tptacek 16y agoEw. I used to write a lot of Python, but I didn't know there was a standard INI parser. Gross.
- zedshaw 16y agoWhat? You should totally be into .ini files. I mean, one incomprehensible difficult to automate bizarre ass user interface file is no better than any other.
- tptacek 16y agoIt's a false dichotomy to suggest that because INI files are incomprehensible, difficult to automate, and have bizarre asses, all text configuration files must be the same. But, like your comment, that's just message board geekery. I wouldn't want to suggest that it's worth arguing about.
- clyfe 16y agoI hate config files, having written a software tool GUI to configure lots of daemons that traditionally use config files. The steps, in general, were parse-modify-overwrite, a real hassle considering the diversity of grammars each daemon employs - following the make your own parser mantra. I welcome Zed's initiative of using sqlite, making a configuration GUI tool for it would be a breeze.
- nivertech 16y agoI did opposite in large Windows-based real-time algorithmic system with tens of thousands of parameters. The parameters were stored in MS-SQL database, under complex schema. For each branch and version, each developer were adding new parameters, so after merges or upgrades it quickly become parameters hell. I changed the parameters storage implementation to use CSV files, instead of MS-SQL, while external API did't changed. Now we were able to add .CSV files to the source control and parameters were versioned and automatically merged. Additional bonus - you can edit .CSV files in Excel ;)
- zedshaw 16y agoI don't think it's a valid comparison between your large data set and my configuration storage, but cool that you went simpler and got more out of it.
- nivertech 16y agoWhy limit yourself to sqllite? This is so old school! Why not use some NoSQL distributed key/value store? [1] With simple map/reduce you will be able to generate INI files or any other format you want ;) [1] Full disclosure: this is what I doing in my new project
- zedshaw 16y agoA couple people have mentioned this, and really I'll tell you a little secret: The config loading in Mongrel2 is all abstracted away and could support anything. I haven't done anything explicitly to let you write a "config load module" but it'd be possible. In theory, since the design kept this MVC idea, you could configure out of a config file, redis, couchdb, really anything you need to get your ops mojo working. I could have easily just went with a config file as the default setup instead of sqlite3, but then where would all the FUD slingers and bikeshedders go to waste their energy? :-)
- andrewvc 16y agoI like the SQLite config. SQLite seems like more of a PITA for some cases, but when using something like chef to configure an instance, SQLite makes more sense than using something like sed to mangle a config file. Now, yes, you can define a whole config file in a template, but sometimes you need to have a separate recipe that makes a change to an existing config (I'm looking at you engineyard), and for that, we must unfortunately resort to sed.
- zedshaw 16y agoExactly. With m2sh the regular small usage case is very easy, and you get tons of capabilities other servers don't have. It's not preventing you from working how you work now, just gives one more step. But, for everyone else, and the future of operations, this kind of configuration storage is insanely useful. The idea that we could point mongrel2 at a redis store in the future and have the 1000s of servers some companies put out automatically update their configs in realtime is just sexy as hell.
- wjlroe 16y agoThat's pretty much what we do at our company (albeit with Riak and it's not mongrel2) - we have x servers with shared config and they all need to get that config on startup. I shudder to remember the early days of running sed on multiple configuration files on multiple servers using parallel-ssh.
- yatsyk 16y agoI've not found answer to question why mongrel2 doesn't use ini files.. It's not so hard to generate sqlite file but it is even easier to generate ini, yaml, json or xml if you don't like particular ini format. And you can manage these files by git/svn and config loader could show you line with error if loading fails.