10 ms·
Clone Dropbox with a Raspberry Pi and BTSync
- ximonn77 13y agoHi, I'm doing something similar, but instead of BTsync, I use ssh & unison/rsync. The remote pi calls back the main pi by ssh. The main pi uses this connection to sync files. So the remote pi can be behind a firewall and no issue. Im interesting in sharing the setup - and improving it, if anyone interested let me know. Im at padrone.nl
- reddit_clone 13y ago>unison The forgotten gem. It was a poster child for OCaml. Didn't get the attention it deserved.
- iuguy 13y agoIt doesn't need to be a USB 3.0 drive. The RPi is USB 2.0 only.
- rubenfonseca 13y agoI've stopped reading when he really recommended that you get a USB 3.0 drive to connect to a RPi :P
- tankbot 13y agoI kept reading expecting an explanation for this, but there isn't one. Does he understand the difference? Who knows. The explanation of "every step of the build" is pretty limited too.
- reustle 13y agoI do understand this but what is wrong with recommending picking up a USB 3 external drive over USB 2? I doubt the Pi is the only device it will be used with.
- MrSlo 13y ago"I am still working on the best way to access the data without disconnecting the drive from the Raspberry Pi every time I want to add/remove something." It sounds like he's putting the data on the USB drives from other computers, not from the RPi.
- davidarkemp2 13y agowith a 150Mbps wifi adapter
- Zoomla 13y agoplus, the RPi can't even achieve full USB 2.0 speed, it is not a good choice for a file server.
- bigiain 13y agoFWIW, in this use-case the limit is the wifi or more likey your internet connection speed. You don't really need blazing disk read/write speeds for your "Dropbox clone" - if you can read and write data as fast as your adsl connection goes - anything more is "wasted". As a general purpose file server though - yeah, a 'Pi isn't going to stream multiple HD videos to all 4 bedrooms on your home network.
- gnur 13y agoBoth my wifi and internet connection speed are well over the 2megabytes/s I'm getting on my pi, which really is the only downside I have found with my pi.
- Zoomla 13y agoWhen I setup a file-server for myself, I set it up on the local network I use most, so the Internet speed is relevant, but not that much.
- Daviey 13y agoEverytime I read BTSync mentioned... i hope it's been open sourced.. then get disappointed, and move on.
- krupan 13y agobtsync is really nice, just install the software, no account creation needed, and if you just want to sync computers on a LAN, no internet connection needed. I was worried about btsync being closed source but when I looked at alternatives like dropbox and spideroak I noticed they weren't open either and you have to sign up for an account and give them personal information to get serious amounts of storage.
- scrrr 13y agoEven if it's nice, I won't recommend it to anyone. If it becomes big, I don't want the next Edward Snowden to tell me it's been used by .gov in the future.
- tuananh 13y agoNo but the way Dropbox and btsync work are way way different. > BitTorrent Sync synchronizes your files using a peer-to-peer (P2P) protocol.
- jamwt 13y agoThey are different, but Dropbox does have some "P2P" behavior with LAN sync: https://www.dropbox.com/help/137/en https://www.dropbox.com/help/137/en Note: Dropbox employee, but this is public information.
- cstuder 13y agoThen there are git-annex assistant (No Windows version) and SparkleShare, which are both Open Source and should be able to do the same. Missing from both though is a mobile version. I quite like my Elements (A text editor for Dropbox files).
- toggle 13y agoEveryone who's looking to run their own private Dropbox should check out ownCloud[1]. It's free, open source, you install it on your own server (or Raspberry Pi), and it's not P2P like BTsync is. It works more like Dropbox, with a centralized server. You just run that server yourself. They have apps for Windows, Mac, Linux, iOS and Android. I've been using it for about two months now with no problems. [1] http://owncloud.org http://owncloud.org
- swombat 13y agoHow well does BTSync work in terms of a large team collaborating together? Dropbox has a habit of just working, with surprisingly few hiccups (none so far except during massive folder migrations). Does BTSync work that well with a team of 10? 20? 30? that work on the same set of documents (though not on the same document simultaneously, so no need for e.g. git).
- IceyEC 13y agoGiven that it's using the Bittorrent protocol (Basically) to handle the actual transfers, yes it will work; the issue you may run into is that there can be a lot of files and data available in the share and every client will try to download all files.
- swombat 13y agoThat happens with Dropbox too though.
- sillysaurus2 13y agoActually not. Dropbox has a feature called Selective Sync which lets you choose which files are synced to which computers.
- swombat 13y agoWe're not using that. Dropbox is a shared drive for the company. My question, still unanswered, remains whether anyone has made btsync work well in a setup where it's used as a shared drive for 10-20 people who interact with many files, though usually not the same file simultaneously (though that happens occasionally, and Dropbox seems to support it...)
- korg250 13y agoBased on neumann comment above, btsyng still has a lot of issues. Keep using Dropbox.
- pierot 13y agoI've been wanting to cancel Dropbox as well. But I choose a different path which includes Bitcasa. I wrote a blogpost about it earlier: http://blog.noort.be/2013/08/02/backup-in-2013.html http://blog.noort.be/2013/08/02/backup-in-2013.html
- aidos 13y agoI installed and played with BTSync yesterday. It was really easy to get it all up and running syncing between two local machines and a remote server. The config was really simple compared to other stuff I've used. Sometimes it could take a while before the syncing would start which is a bit of a pain. We run a system where we have a lot of data on a server that we want to see locally. Edits are made both locally and on the server. At the moment we're using Aerofs to handle the syncing. As a general rule it works really well. The issue we've run into is with pre-seeding new machines (ie, when our spot instance based server get killed). I need to play around a bit more but it's time consuming to get a machine back online and totally synced (there may be a solution to this). Mostly it's a pain being at the mercy of someone else's magic. Aero is great, except when it isn't - and we don't know the rules. That's a real pain if it's business critical. BTSync looked great - until I realised that it's not opensource either. There are a few syncing options on there but they don't all tick all the boxes. Unison doesn't trigger on fs events so you have to run it manually. BTSync isn't opensource. Dropbox syncs to their servers and back. With AeroFS you don't always know how it's doing what it's doing. So far AeroFS has been the winner for us. BTSync definitely looks like it fills the same space though.
- CrankyPants 13y agoWe tested AeroFS and BTSync on a variety of data in shared folders, and consistently found sync issues with BTSync. We eventually abandoned BTSync for AeroFS, and while it hasn't been perfect, it's been considerably better at handling syncing issues. Anecdotal, and just one case, but hope someone finds it helpful.
- aidos 13y agoUseful to know. I only tested btsync briefly yesterday and it wasn't picking up changes as quick as Aero. Again, single data point but I'd recommend Aero from what I've seen (and we use it all day every day).
- neumann 13y agoWe have had the same experience. I loved the idea of BTsync, but we encountered too many issues. We tested sparkleshare and lipsync too. Unfortunately, none of them dealt with conflicts and merges as they were meant to. BTsync would often not pick up changes (using osx and linux clients), and/or overwrite new files with an older version. It also had a slow latency when a file did change. As to the open source nature of it, you can supposedly limit its access to the shared servers through configuration, but this caused more sync problems. These tests made us appreciate the Dropbox implementation even more. Unfortunately, we need non-cloud syncing for our data, so I wrote a complex cron/rsync/inotify hack with its own issues - but at least we know the limitations and have custom workarounds. Personally, I hope BTsync will mature and be a feasible alternative.
- hipaulshi 13y agoI used my dropbox for really important files. But then all the rest is archives and don't mind it getting it next day. All my external hard drives never last more than 2 years. While it is an interesting approach I am concerned about reliability if it is an archive. I tried out Bitcasa for Mac and it is terrible. My Finder basically stop responding after bitcasa started. I have been working on a simplified version of AWS boto library for glacier, in the goal of making an open source tool like a command line sftp/ftp style access to glacier.
- bargl 13y agoI really liked the implementation listed out on this site. I went through it and it was a great learning experience. You also get a sweet web interface with owncloud. http://blog.bittorrent.com/2013/05/23/how-i-created-my-own-personal-cloud-using-bittorrent-sync-owncloud-and-raspberry-pi/ http://blog.bittorrent.com/2013/05/23/how-i-created-my-own-p...
- taway2012 13y agoI'm uncomfortable about the opacity of BTsync's privacy architecture. AFAIK even the protocol isn't documented. The code obviously isn't open source. I wouldn't use it sync anything except Truecrypt volumes. :)
- deleted 13y ago[deleted]
- AsymetricCom 13y agoOn top of that (literally and figuratively) is Pi's ARM's TrustZone hypervisor..
- jewel 13y agoThis is the same issue I have with btsync, so about a month ago I started working on an open-source clone, with similar features but an open, documented protocol. https://github.com/jewel/clearskies https://github.com/jewel/clearskies
- schabernakk 13y agobest of luck. i really hope you succeed with this. it seems that right now all opensource dropbox alternatives, while great, have a shortcoming somewhere (git annex, sparkleshare, btsync...)
- reustle 13y agoI'm looking forward to clearskies and will gladly switch over to it once it's ready!
- mahyarm 13y agoOr encfs, a per file encrypted filesystem.
- bigiain 13y agoI'm using BTSync with EncFS, but if you don't trust the BTSync binary – it's probably got access to the non-encrypted data as well unless you've jumped through a few more hoops than a "typical BTSync installation". I'd guess 99.9% of BTSync users have the binary installed with at least enough permissions to read anything the logged in user can read (which includes the cleartext-mounted stuff in your EncFS or TrueCrypt volumes).
- hrnnnnnn 13y agoMy friend ran BTSync on a raspi, and it was dreadfully slow to index for large amounts of data. Like, multiple days of 100% CPU slow.
- vjoel 13y agoAlso, with encrypted volumes, just reading files on the raspi is very slow. Consider replacing raspi with odroid, utilite, or other fast, fanless, cheap arm sbc.
- bigiain 13y agoIf the 'Pi is just a "cloud storage appliance", there's no need for it to care that the data is encrypted. I use EncFS to encrypt on my laptops/desktops/servers/phones/tablets, and have BTSync sync the encrypted partitions to a machine at home that doesn't have EncFS installed. I'm using an old Mac Mini rather than a 'Pi, but the same principle works - the Mac Mini serves my archive/backup needs for my synced data (it's using software raid1 over two drives for reliability, and does TimeMachine backups to a rotating set of external drives for archiving), but none of the data on the Mini or it's drives is of any use if hacked/stolen, since if doesn't have EncFS or the crypto keys.
- limpangel 13y agoI have done something similar, but used a Raspberry Pi and Seafile (http://seafile.com http://seafile.com) instead of BTSync. I did not get the chance to test it with large media files, but works great for documents and photos. Also has mobile and cross platform clients. And the best part is it's open-source: https://github.com/haiwen/seafile https://github.com/haiwen/seafile
- kcorbitt 13y agoI'm using Seafile with a VPS. I had the VPS anyway for other projects and it's great to be able to access my files from anywhere. I'm really impressed with Seafile's polish, too. The only feature I miss is not being able to host files publicly so others can download them like Dropbox lets you. I ended up with a workable solution though by installing a seafile client on the seafile server as well. I then used it to sync a particular directory with my PC and made that directory available through nginx.
- lins05 13y agoYou can generate shared link to a file or folder on your seafile website and send it to others.
- nsomaru 13y agoCould you share your experiences, for example how many files do you have on the server? I have read[0] that Seafile struggles with larger amounts of files and I was hoping to use it as a drop-in NAS replacement for businesses. [0] http://blog.patshead.com/2013/06/self-hosted-cloud-storage-changing-the-plan-and-scaling-back.html http://blog.patshead.com/2013/06/self-hosted-cloud-storage-c...
- bostonbot 13y agoI tried this recently... it works well except with the mobile integration. I then purchased a TonidoPlug and things seemed to go much more smoothly.
- mynegation 13y agoIt is the umpteenth time I see a "Dropbox clone" on HN, but none of them is. I badly want an open-source sync online storage client and server, cross-platform, with encryption at least om a server side, that I can install on my own VPS or my own host at home. Online storage became an important element of a digital life, pretty much like Web earlier. I know that Dropbox is an YC company, but I wonder where are Apaches and Nginxes of the online storage?
- nodata 13y agoSee jewel's comment below https://news.ycombinator.com/item?id=6648109 https://news.ycombinator.com/item?id=6648109
- kcorbitt 13y agoI think Seafile checks all your boxes. I run it on a VPS and use the Windows, Linux and Android clients daily without any issues. I haven't tried the encryption but it allows you to password-protect folders. http://seafile.com/en/home/ http://seafile.com/en/home/