6 ms·
With the ubiquity and general awesomeness of sqlite I see no real reason usecase for these toolset over sqlite.
by jsilence 7y ago
With the ubiquity and general awesomeness of sqlite I see no real reason usecase for these toolset over sqlite.
- daenz 7y agoHuman readability. I don't need any other software but a text editor in order to easily view/edit the database.
- astrobe_ 7y agoMy ambivalence exactly. The user in likes plain text for its accessibility and flexibility, but the programmer in me dislike it for its gross inefficiency. Does some sort of general data compiler/decompiler exist? Please don't say Binary XML.
- _-___________-_ 7y agoThere are so many use-cases where the compute inefficiency of this just doesn't matter but the human efficiency of being able to read and edit the files is a huge win.
- taeric 7y agoConsider the gross inefficiency in the other direction. To simply edit done simple data, you have to have how much code and infrastructure? And how much of that is plagued by bitrot?
- astrobe_ 7y agoIt is a "scaling" problem. At small scales, yes, human time can be more valuable than computer time. At large scales, humans will sacrifice their own time to make computers work faster. The problem is with intermediate scale, where the computer takes a bit too long to do simple things.
- themodelplumber 7y agoCase: You really, really enjoy your text editor and have even built special tools that work within it, but it doesn't support SQLite and you generally enjoy using text tools anyway.
- foundart 7y agoOne key differentiator is that recutils are for use cases where you want your data files to be in a plain text format.
- rabidrat 7y agosqlite databases are binary blobs which don't have meaningful diffs if you check them into source control.
- typon 7y agoI wrote a SQLite <-> Json converter for this reason. Ended up coming up with a weird convention in the Json to represent relations. It might be cool to instead write a recfiles <-> SQLite converter. I want to use SQLite from within my app, but want to check in readable data to a source control system.
- brigandish 7y agoWhy would you check a database into source control? Not trying to be funny, genuinely wondering.
- lolc 7y agoSometimes you want data under version-control. You want to track changes and maybe you want to be able to reverse them. An example is human-supervised data-imports. Example: we operate a pipline that pulls datasets from multiple government agencies and merges them. The sources are updated periodically and updates frequently introduce new inconsistences. To reconciliate, we use auxiliary datasets that record tweaks to each source, such as entries to add, mutate or delete. In our implementation of the pipeline the source datasets, the tweak tables, and the resulting merged dataset are kept in Sqlite for ease of processing. The pipeline writes out dumps of each table during processing, even for intermediate stages. When running the pipeline, I can scan the diff and decide whether the changes are reasonable or tweaks must be introduced or removed. Once I'm satisifed with the result, the dumps are then committed to record the current known-good state. When somebody wants to know why one record in the result is the way it is, I can determine how it changed in the source data, how it was tweaked, and what the result was before and after. It's really easy to produce diffs between revisions. Code-reviews are conducted over changes in pipeline code, validation logic, source and result all in one step.
- ORioN63 7y agoWhile I agree with the general awesomeness of sqlite, I worry about the blanket statement that you've claimed. If you read the documentation 'Purpose' [1], it says, the issues with databases, like SQLite: - The stored data is not directly human readable. - The stored data is definitely not directly writable by humans. - They are program dependent. - They are not easily managed by version control systems. I just want to say that I also love sqlite, but I cannot avoid worrying that when the asteroid strikes and the ocean men invade, I will not be able to program a sqlite driver on a homemade 8 bit computer. [1] https://www.gnu.org/software/recutils/manual/Purpose.html#Purpose https://www.gnu.org/software/recutils/manual/Purpose.html#Pu...
- jsilence 7y agoSure, text files are awesome, I get it and generally I agree. But sqlite is in every nook and cranny of our computer and mobile systems, it is in Android, it is in the browsers. I find it hard to imagine a postapocalyptic world where one would be able to read a file somehow, yet not be able to get access to some sqlite processing software. I like the rectools for educational purposes, but beyond this for any however small purpose project I'd rather switch to sqlite pretty soon.
- bjoli 7y agoIn GNU Emacs SqlMode is not as nice to work in, whereas recfiles integrate out of the box with org mode. Doing the same with sqlite works, but is not nearly as nice.
- eeZah7Ux 7y agoSQLite is hugely more complex and therefore brittle and difficult to debug that this. Especially if you absolutely need 100% reliable compatibility between many different devices. Even more so if the format need to be part of a standard and/or a contract.
- anthk 7y ago- SQLite is damn easy. - It's almost a standard due to licensing. - Tiny, really tiny.
- lolc 7y agoAh the difference between easy and simple! It would be simple to implement an append-only writer in Recutils. I don't know where I'd start if I wanted that for Sqlite. I don't think the format allows it.
- eeZah7Ux 7y agoGo ahead and compare the SLOC count with a parser for the "rec" format. Also, why are you ignoring the other points?
- CarelessExpert 7y agoTell me when you can grep or sed an sqlite database, or cleanly check it into a git repo.
- bachmeier 7y agoI've never heard of Recutils before, but I've been using grep, sed, awk, cut, etc. to work with data for years. sqlite is just too much of a hassle for version control and sharing. If I use sqlite, I have to export to text, commit and push, pull on the new machine, then import into sqlite. I've never run into performance issues even with thousands of records.
- CarelessExpert 7y agoAnd where you need a reasonably performant relational data store, all those cumbersome gymnastics might be worth it. But if you're working with a modest dataset where volume and performance aren't a primary concern, working straight in text is infinitely more convenient. There's a reason people do so much bulk data processing in CSV, JSON, or other easily manipulated formats.