4 ms·
I'm already struggling with a diversity of versioning solutions. I'm obviously going to be adding ori to the roster somewhere -- it looks AWESOME -- but I'm al
by undoware 13y ago
I'm already struggling with a diversity of versioning solutions. I'm obviously going to be adding ori to the roster somewhere -- it looks AWESOME -- but I'm also increasingly frustrated with the fact that there is no meta-level tool to navigate all these sources of history:
1. Git for code, obviously, just not binaries larger than 25mb. Profound network effects (Github, hi) will keep this true indefinitely.
2. For versioning large binaries, I go back and forth between boar and git-annex, for the desultory reason that I can't really get down with either the symlink insanity of git-annex or the old-school server/client svnish model of boar.
2. Couchdb, which I use for (1) couch apps, (2) a backup medium, and (3) the main document store for my company. Couch is great, but is emphatically not a filesystem -- you get CRUD, but few of the niceties you'd get with a 'real' FS. That's okay -- the whole point of Couch is worse-is-better network-native data storage -- but I can only be pacified so many times by a recitation of the CAP theorem. At some point, you want a POSIX compliant filesystem, and you're willing to bite many bullets to get there, including those that Couch made a name for itself by dodging! In other words, you'll take a hit (probably on partition tolerance) in order to get, say, transactional atomicity, and so you'll toss Couch and go with a SQL, at least for some use cases. (Don't get me wrong -- you can pry couchdb from my cold dead paws -- but I also know when not* to use it.)
3. There are many ways of implementing file versioning in an RDBMS, and between Concrete5, Owncloud, and the innumerable sqlite databases scattered throughout my system, I'm sure all are in use, somewhere.
4. My main file server runs btrfs, and sometimes zfs, and the excellent 'snapper' tool that comes with OpenSUSE helps me maintain and access fs-layer diffs. My home directory is regularly snapshotted in this manner.
5. For backing up the linux machines on my network, I use rsnapshot, which uses non-symbolic links for creating differential backups. It's one of those old-school solutions that just happens to work great.
6. Google Drive and Dropbox both offer versioning, so at least when I have these installed (not at the moment) I arguably have a sixth source of version history.
7. My vms are all getting snapshotted on the daily too, and I use bedup to manage the btrfs partition where they are stored. The net effect is storage-equivalent to differential backups, although admittedly less efficient to create, at least on btrfs (deduplication occurs offline). But then again, having an image is the gold standard for backups for good reason.
8. And then there were eight. Welcome, ori. But you wouldn't happen to speak git, couch, btrfs, zfs, vmdk, and sql, would you? Because that would make my life a lot easier.
- dgroves 13y agoThe real answer of course is SmartOS: * ZFS for storage virtualization * Crossbow (dladm) for network virtualization * Zones for virtualization and containment * DTrace for introspection * SMF for service management * RBAC/BSM for auditing and role based security
- IgorPartola 13y agoYikes. It sounds like you are in a constant state of trying new file systems and backup solutions. Why not simplify and go with a good enough solution? Personally, I like zfs and rsnapshot. That is what I use on my NAS and it seems to work without me spending time one maintaining it after the initial setup. Why rsnapshot and not snapshots of the zfs volumes? Because I want to keep lots of backups and my understanding is that zfs loses performance after you cross into a few dozen snapshots on the type of hardware I have. Now, I never understood backing up your home for. Why? I keep any documents on the NAS (thanks VPN and ssh for making life easy here), dot files on GitHub, and source code in git with origin on either GitHub or the NAS depending on if it is private or not. Chat logs are going to be my one exception, but perhaps the servers will back them up for me instead a la GChat.
- undoware 13y agoYou're right, there's a lot going on, but -- and you are just going to have to believe me on this -- each method gives me something the others don't. As will ori. And honestly, I don't mind the extra moving parts: having recently suffered a catastrophic drive failure that set my business back a quarter, I am okay with a never-too-much-overkill approach to backups. My problems number two: On the first hand, there are often bizarre interactions between versioning systems when they are hosted atop of one another, breaking encapsulation in unexpected ways. You can't use .vmdk snapshotting on btrfs, for example, because the performance is whatever the opposite of 'breathtaking' is. (Sighgiving?) Meanwhile, a git repo on a Dropbox share is going to chew itself to pieces the first time there's an fs-level merge conflict in .git. I could go on. The second problem is the flip side of the first -- there is no way of importing and exporting history between versioning systems. I shouldn't worry about git and dropbox: dropbox should offer itself as UI for git.