13 ms·
My one-liner Linux Dropbox client
- dijit 8y agoI did something similar actually. I have my 1Password vault on Dropbox so I can get it on my Linux machines. Dropbox has some nice python modules/libraries to support me, so I’m able to download the directory as a zip and extract it in memory. It’s stupid and unidirectional, but I’m sure more talented people are able to build a bidirectional python Dropbox client in no time, with inotify since python has bindings for this too.
- stabbles 8y agoNot really the same, but this reminds me of the top comment of the Dropbox announcement: https://news.ycombinator.com/item?id=8863 https://news.ycombinator.com/item?id=8863
- dotancohen 8y agoMy thought exactly. My one-liner dropbox client looks a lot like `rsync foo bar`.
- lewiscollard 8y agoWhy does someone have to bring up this eleven-year-old comment any time file synchronisation or Dropbox is mentioned? Dropbox is now broken for certain Linux configurations and someone had a bit of fun writing a shell one-liner to replicate some of its functionality. Meanwhile, the original eleven year old comment was partly a throwaway remark that it didn't do anything he could not do already which means it wasn't interesting to him (plus two other points, but nobody links to it for those). That's not even "not really the same", it's "and since you mentioned Dropbox, let's make fun of BrandonM some more". It comes across as just mean-spirited and shitty to me.
- Dylan16807 8y ago> any time file synchronisation or Dropbox is mentioned This is one of the most relevant topics you could quote it on, really. The post is tackling one-way sync and calling it a rough equivalent to Dropbox.
- therealidiot 8y agoWhy does the client care which filesystem you use???
- jeremysmyth 8y agoDifferent supported features make it easier to maintain such a client. Example: Some filesystems notify observers on file change, some don't.
- jfyne 8y agoIt looks like there might be a bug. What happens when you add a new file? A quick scan of the entr help text suggests you should be using the -d flag.
- boulos 8y agoHa! I took too long to format my response :). The entr examples from [1] seem to do the same (ls *.rb for example). [1] http://eradman.com/entrproject/ http://eradman.com/entrproject/
- amelius 8y agoAlso, doesn't entr invoke its command in parallel when there are multiple changes in quick succession? And wouldn't multiple parallel instances of rclone potentially mess up the backup? (I didn't read any of the man pages)
- branweb 8y agoyep...specifically, use the -d flag in a loop since all it does it exits the process when a new file is added: while true; do ls -d src/*.c | entr -d <cmd>; done
- boulos 8y agoI hadn't seen entr before! My first glance at your one-liner: find $ORG_DIR | entr -r rclone sync -v $ORG_DIR $REMOTE:org is that find will enumerate all your existing files, but starting a new one won't get picked up. It seems like entr is prepared though, and you should just pass in -d. In fact why use find at all, if you want to rsync the directory?
- binaryapparatus 8y agoHuge additional plus, entr does not rely exclusively on inotify, it is also 'BSD kqueue wrapper. It is the only command line utility that I found that allows cross platform (linux/BSD) scripting with file system events. I have few scripts that rely on entr and I am always sure they are OS agnostic.
- lloeki 8y agoThen take a look at fswatch! (See sibling comment)
- lloeki 8y agoI didn't know about entr as well but I'm using fswatch which is largely similar and portable as well, supporting not only Linux and BSD but also non-WSL Windows and Solaris too, and leveraging the Filesystem Event API on macOS. https://github.com/emcrisostomo/fswatch https://github.com/emcrisostomo/fswatch
- petepete 8y agoI love entr for running tests while I'm working, it's nice to have a tool that works reliably in any ecosystem.
- kgbme 8y agoIt's mentioned on reddit, that passing the -r flag will pick up new files, right? Link @https://www.reddit.com/r/linux/comments/a92m1u/comment/ecgqftv https://www.reddit.com/r/linux/comments/a92m1u/comment/ecgqf... EDIT Oh I see what you're saying; that's confusing, though, how's it supposed to work in the first place, then?? :))
- conanthe 8y agoDropbox is a no no for one simple reason - there is no end to end encryption. Unless I don't know something? I wouldn't want a disgruntled employee to fiddle with my files.
- dewey 8y agoJust sync an encrypted image if your threat model includes disgruntled employees motivated enough to mess with your account.
- zapzupnz 8y agoCould be problematic if the image is huge.
- ubercow13 8y agoYou could use something like gocryptfs that encrypts at the file level
- zuck9 8y agoCryptomator is another cross-platform alternative for file-level encryption: https://cryptomator.org https://cryptomator.org Also includes first-class support for cloud syncing apps.
- treypitt 8y agoAnyone have experience with Boxcryptor? Haven't tried it since a few years ago (https://www.boxcryptor.com/en/ https://www.boxcryptor.com/en/)
- whitepoplar 8y agoIs there any Dropbox-like service that lets you control your own private keys, without resorting to the ugliness of uploading an encrypted image to Dropbox?
- 8y ago
- lloeki 8y ago> Recently, the proprietary Dropbox Linux client dropped support for all Linux file systems except unencrypted ext4. What the heck. Anyone has more information about that? Any announcement? And why would it fail to work on an encrypted (LUKS?) ext4 FS when encryption seemed to me to sit below the filesystem (since AFAIK ext4 doesn't support encryption itself)?
- cpach 8y agoI too was a bit surprised by that statement. But according to Dropbox, full disk encryption should work fine. ”If you received a notification on Linux and you are running ext4, it may be because you are also running ecryptfs. ecrypfts is not supported. However, we support full disk encryption systems, such as LUKS for Linux users.” Source: https://www.dropboxforum.com/t5/Error-messages/Dropbox-client-warns-me-that-it-ll-stop-syncing-in-Nov-why/m-p/290058 https://www.dropboxforum.com/t5/Error-messages/Dropbox-clien...
- feb 8y agoThat news hit HN top stories a few months ago: https://news.ycombinator.com/item?id=17732912 https://news.ycombinator.com/item?id=17732912
- discordianfish 8y agoIt's FUD. ext4 with LUKS, which is what pretty much everyone is using on a desktop, is still working and the client supported.
- mooreds 8y agoPrevious discussion of entr: https://news.ycombinator.com/item?id=13856623 https://news.ycombinator.com/item?id=13856623
- ekianjo 8y agoIt's kind of funny to replace a proprietary client for a proprietary service with FOSS tools when you could completely replace Dropbox with FOSS alternatives instead and enjoy Freedom: Syncthing works very well, and while syncing is not its main feature you could also work with nextcloud.
- rsync 8y agoIf only there was a platform agnostic, standards-based, cloud platform that you could point any old SSH tool at. Imagine if such a provider existed and was running on top of ZFS and maintained the current, stable version of 'borg[1]' on the server side ... Probably too much to ask. [1] https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/
- ViViDboarder 8y agoGiven the username, I’m assuming this is tongue-in-cheek because rsync.net is pretty much that.
- ndnxhs 8y agoSeedbox server hosting might be usable here. They are bascically shared Linux hosts with loads of disk space and network.
- tssva 8y agoSince it is being used with systemd a systemd path unit could be used to monitor the file system removing the need to use entr.
- amenod 8y agoThanks, didn't know systemd can do that [0]! I must grudgingly admit that it seems to be quite useful, though I still don't like having a single app doing all that stuff. But that's off topic... :) [0] https://blog.andrewkeech.com/posts/170809_path.html https://blog.andrewkeech.com/posts/170809_path.html
- tyingq 8y agoBoth path units and entr use the inotify kernel API on Linux. If your needs get more complex, there are python, perl, etc, APIs for the inotify libraries. Also, if you have an older linux distro that predates inotify(), there's similar functionality within auditd that will log changes to a file with a tag of your choosing, then you can tail the file.
- mixmastamyk 8y agodnotify is another older one.
- panzerklein 8y agoI setup Syncthing [1] to do the same for my vimwiki folder. It was surprisingly easy, and doesn't require any external storage services. And it can even automatically sync files without internet, over the local network. [1] https://syncthing.net/ https://syncthing.net/
- hellcow 8y agoI've been using Syncthing for more than a year across ~6 machines to keep them in sync. It works great for more complex setups than the article describes.
- jogundas 8y agoThough very useful, syncthing unfortunately uses direct communication which leads to problems with port forwarding.
- yjftsjthsd-h 8y agoThey have relay servers.
- jogundas 8y agoThanks, did know about that! https://relays.syncthing.net https://relays.syncthing.net and https://docs.syncthing.net/users/relaying.html https://docs.syncthing.net/users/relaying.html .
- jeena 8y agoI use Syncthing for my KeepassX on 6 computers (Linux) and my phone. My music folder too. My photos folder between 3 computers. And then we use Syncthing to sharo Call of Duty maps between 5 people (Mac, Windows). All in all I really like it. I also start it with systemd on Linux.
- gsich 8y agoSadly, the Android-App still can't save stuff on the SD card. My main show-stopper.
- beagle3 8y ago.... Except, a file added to your dropbox will only appear in your local box the next time you create a local file -- there is nothing to wake the local entr when a file is remotely added to dropbox.
- deleted 8y ago[deleted]
- elliotec 8y agoCould write a little script to add then delete a temp file on some chron job or at login or something.
- beagle3 8y agoIf I have a shared dropbox folder with someone, as soon as they put the file in, I get a copy. There is no way to get a notification that would provide a similar experience without the real dropbox client, afaik.
- deleted 8y ago[deleted]
- greatquux 8y agoDropbox will work if you just create a local loopback file for an ext4 filesystem and mount the Dropbox folder on it.
- algo646464 8y agoThe above is a great answer. To expand on this (from some googling). All of this requires root. 1) Create a sparse file (actual size depends on usage) truncate -s 100G /home/zzz/Dropbox.image 2) Create a ext4 filesystem on it mkfs.ext4 -m0 /home/zzz/Dropbox.image 3) Mount it as a loopback file system sudo mount /home/zzz/Dropbox.image /home/zzz/Dropbox-fs 4) Create / Copy your dropbox folder to Dropbox-fs sudo mkdir /home/zzz/Dropbox-fs/Dropbox sudo chown -R zzz.zzz /home/zzz/Dropbox-fs/Dropbox cp -r /home/zzz/Dropbox-original /home/zzz/Dropbox-fs/Dropbox 5) Start Dropbox and point it to the new location 6) If we want to mount this automatically at login. Adding it to /etc/fstab may not work because of encryption. But perhaps some script at login to mount it will work. For example, we can create a script at /usr/local/bin/mountDB.sh and then add a line to /etc/sudoers zzz ALL=(root) NOPASSWD: /usr/local/bin/mountDB.sh Then somehow call this script at login of user zzz.
- GordonS 8y agoI've never heard of a 'loopback filesystem' before. Can you expand a little on what this is?
- prewett 8y agoIt lets you mount a file as a filesystem, instead of an actual hardware device. I think it's called "loopback" because instead of reading from /dev/whatever it loops back to /dev/original and reads the specified file. See the man page for `mount`: https://linux.die.net/man/8/mount https://linux.die.net/man/8/mount (search for "loop")
- Spivak 8y agoThe right term is a 'loopback device' which allows you to use a file on an existing filesystem as a block device. Here's what the flow looks like: open(/path/to/myfile) -> ext4 driver -> block operations on /dev/loop0 -> open(/path/to/loop-file) -> block operations on the data region of the file -> btrfs driver -> block operations on /dev/sda
- vanous 8y agorclone is great, i have been using it to make backup of google photos. I had tried so many different solutions before achieving this. Great to see it being here on HN.
- hv42 8y agorclone is such a great tool indeed. This is almost the only viable approach for transferring your data from one cloud to another. Huge thanks to the creator and contributors!
- shittyadmin 8y agoIf you did want to run the original Dropbox client, you could likely make a blob image on your encrypted partition, loopmount it and then sync to that.
- amerine 8y agoThis is cool. Thanks for sharing.
- kheiron1729 8y agoTLDR: https://rclone.org/ https://rclone.org/
- fouc 8y agoWow, this is better than getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem.
- ilogik 8y agoreference acknowledged https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
- didip 8y agoThe true workhorse behind this one-liner is inotify(Linux) or FSEvents(Mac) API. Not sure what is the Windows equivalent. If your favorite language has access to those, then you can basically build your own file syncing client.
- smilliken 8y agoConsider using git or another SCM instead. You don't need a centralized server, you can push/pull between any two machines with ssh (or a handful of other protocols). Encrypted end to end. Supports branches, useful for stuff like dotfiles on different machines. You don't need commit messages, just make an alias to add all changes and commit with and empty message. You have a full history, can revert back to any state, inspect any old version, diff versions, etc. There's git clients for mobile. Free as in beer and freedom. Check-summed. Puts you in control of when to sync, supports offline. Future-proof, acquisition-proof, VC-proof. Familiar.
- cheez 8y agoI do this, but 90% of the time I forget to pull on the other machine and have to deal with conflicts.
- acranox 8y agoCheck out Sparkleshare.
- cheez 8y agoOoh this looks good!
- zipperhead 8y agoYes, you'd have to automate the pull for this to work, and then you'd have a way to notify on conflicts so that they are resolved ASAP. Come to think of it, Dropbox isn't really so great at the notification aspect of this either. It creates a 'conflicted' file but there is no notification that I'm aware of. I have to remind myself to periodically check for conflicts.
- geofft 8y agoIs there a standard-ish git merge driver that resolves all merges with a Dropbox-style "conflicted copy", so you can unconditionally git pull in a cronjob and it never leaves the repo in an unmerged state?
- techntoke 8y agorclone is great, but I find syncing manually in intervals to be ideal since you get to see what is changing from your destination/remote. I use CrashPlan Pro to back up everything automatically, and then a few times a week I'll run rclone on my data drive so that I can see what has changed.
- saltybytes 8y agoThis is _VERY_ cool but I stopped using dropbox and keep everything on my self-hosted Nextcloud.
- spatular 8y agoIn that setup entr only watches files that existed when systemd service was first started. So new files won't trigger backup. Something like this should work: while true; do find $ORG_DIR | entr -d -r rclone sync -v $ORG_DIR $REMOTE:org; done
- ISO-morphism 8y agoThat's constantly polling the filesystem, isn't it? I've never used it myself, but I know systemd has file watching built in. I think using `PathModified=/path/to/org` would also eliminate the need for entr [1]. [1] https://www.freedesktop.org/software/systemd/man/systemd.path.html https://www.freedesktop.org/software/systemd/man/systemd.pat...
- spatular 8y agoIt should re-poll only when a new file is added. In op's usage scenario it won't be very ofthen. The systemd solution may be even better, though it seems there is no way to ignore changes in temporary files that emacs likes to create.
- datafatmunger 8y agoI got one too, but syncs to ipfs/ipld and uses gpg for encryption. https://github.com/peerparty/fs2ipld/blob/master/fs2ipld.sh https://github.com/peerparty/fs2ipld/blob/master/fs2ipld.sh
- not_a_terrorist 8y agoYou will be interested by this free and open source software: duplicati.com