6 ms·

BitTorrent takes on Dropbox with personal file sharing

72 points by FluidDjango 15y ago

sukuriant 15y ago
Under the assumption that it uses something like BitTorrent as its underlying protocol, how does it take into account different versions of the same file, and which of the versions will win? While multiple clients are updating their version of the same file? I can think of some ways for this to work, but I wonder about how it's officially done. Does anyone have insight?
- jwhitlark 15y ago
  It's solving for this problem: http://xkcd.com/949/ http://xkcd.com/949/ It operates at the torrent level, not the file level, so while you certainly can create a new torrent with updated content, it'll show up as a new entry instead of an update. As a quick technical sketch, it streamlines torrent creation, transferring torrent references to others, torrent downloading (you either get a link your client understands, or a pre-built, quick-start client with the torrent already installed), and provides temporary cloud backed seeding. Disclaimer: I work on the project as an engineer. These comments are my own, and have nothing to do with Bittorrent the company.
  - icebraining 15y ago
    So despite the post title, it's not really a direct Dropbox competitor, since it won't support realtime syncing, right? Btw, since one "donates" disk space in exchange for free service, I assume that will contain data from other people; is that protected (encrypted) or could I just peruse through your photos if I happened to be a peer chosen to seed them?
    - chimeracoder 15y ago
      I'm not sure how this is implemented, but I can imagine a case in which this would work out. First, let's assume the data is encrypted with some strong encryption algorithm and similarly strong encryption key. This provides some security, but a lot less than you might think. Because data is being stored locally for a potential attacker, it is very easy for them (hypothetically) to make a large number of copies of that data and then distribute it for brute-forcing. This dramatically decreases the amount of time it would take to crack the encryption, even by brute force. [1] However, this assumes that they have the entire data at their disposal. In a torrent, you oftentimes only have chunks (say 1 MB each), and everyone doesn't download the entire torrent. So someone may only have a few megabytes worth of data. Still that's potentially enough data to contain sensitive information, even a few pictures. However, now let's assume that the data is scrambled. This means that the data is divided into 1 MB chunks of data that are not necessarily contiguous. So a 1 GB torrent would have 1000 chunks of 1 MB each, but each chunk of size 1 MB would have data from various parts of the torrent. The scrambling mechanism could be randomized and sent (securely) to the second party so that they are the only ones able to fit the pieces of data together correctly. The protocol could enforce that individuals cannot host more than _____ chunks/MB of the same file, or even that a file has to be distributed among IP addresses from different IP blocks (allocated geographically), thus reducing the chance that a single attacker has access to large parts of the same file. It does not rule out the possibility of a coordinated attack over a large geographic region, but it reduces the risk, and it would probably suffice for most use cases. Heck, people send public Dropbox links over plaintext, so it's not like we're talking about people who place a high value on security anyway (or, at least, people who think that their data is valuable enough to be worth the effort to crack). [1] There's a nice website that illustrates the advantage of having local copies of the encrypted data, but I can't find it now. It was basically a password strength-checking website that tells you how long it would take to crack your password by various methods.
      - icebraining 15y ago
        Oh, I'm sure it can be protected, no doubt. And you probably don't even need scrambling, just generate a random AES key for each torrent and distribute it only to the people who you shared the torrent with - passwords aren't needed. I was just looking if it was actually protected or assumed public.
mikemoka 15y ago
does it work this way? user shares file > the system stores the file on S3 > the system distributes the shared file through torrent > "enough" torrent peers are reached > the system REMOVES the file from S3 > a week goes by and the file loses momentum > available peers drop > the file is lost.
- jwhitlark 15y ago
  The file isn't lost, it's just not seeded from S3 anymore. It's not a backup service, it's a way to transfer large items to family/friends. If it had existed at the time, I'd have used it to send my wedding photos and videos, for instance, instead of burning them to a dvd and mailing them.
  - mikemoka 15y ago
    ok but you can't present it as a true dropbox alternative then
    - jwhitlark 15y ago
      Interestingly enough, we don't present it as a true dropbox alternative. It's a different use case: share, not sync. The dropbox comparison seems to be pure editorializing. It's been enlightening to see the difference between what I work on, and what the articles say. Like a game of telephone.
scrod 15y ago
>The system isn’t up and running yet, but the idea is that users will receive free storage for their files by sharing some hard drive space and bandwidth with other users. This is how Wuala works, if I understand correctly. And let's not forget AeroFS. While these products are somewhat similar, I absolutely love seeing apps of this nature (syncing distributed P2P) proliferate.
- someperson 15y ago
  Is AeroFS even alive? Waiting a few weeks or months for an early product to be perfected is fine, but it's been more than a year. This is ridiculous. AeroFS want users to invite their friends to put themselves higher on an invite list... For a product that will use MY hard drive space and bandwidth.
  - scrod 15y ago
    I've been using it on my computers as well as with a friend for at least a few months now — it works pretty well. And it uses disk space only to store your own files plus those that others share with you, the same as Dropbox.
  - LogicX 15y ago
    Agreed - I loved the concept, but couldn't get it to work the way I needed it to, even with the main dev's help. (It took weeks to sync a 50GB folder over a gigabit lan, it wouldn't properly recognize the folder if I rsync'd it first - it duplicated everything) So I abandoned it for now, until such time that it got more mature and worked for me -- but I just visited the site a few days ago, and saw its still private beta. Though looking again just now, I found the changelogs: http://support.aerofs.com/customer/portal/articles/240198-aerofs-ea29-1-update http://support.aerofs.com/customer/portal/articles/240198-ae... and it looks very relevant to my issue, so I may have to give it a try again!
    - someperson 15y ago
      I'm complaining that I never got an invite, even after an entire year! I emailed the devs a few weeks back and never got a response. I want to use their product! If their product is good, I'll tell my friends. Not before I've even tried it.
abegrill 15y ago
When it comes to p2p sync TonidoSync is probably the best one out there. It has clients for all the leading mobile platforms (iOS, Android and Windows 7). The software is also cross platform (Windows, Linux and Mac)
kristen12 15y ago
This is not sync. If u are looking for p2p sync tonidosync is probably the best one out there. Tonidosync is cross platform and it has mobile apps for all the leading mobile platforms.
codemechanic 15y ago
How it is same as dropbox?
evgen 15y ago
Once upon a time, before "p2p" was even a term and when Napster ruled the rose, there was a system called Mojo Nation. It was a distributed storage system that broke a file into distinct pieces identified by a hash and then spread those pieces out to peer nodes that would return them when asked and the map of what pieces were needed to rebuild the file was stored in a service called a tracker, which existed as public services and which you could run yourself for private files. To incentivise people to keep their agent running and serving disk space you would get credits for serving data chunks or acting as a relay node (NAT traversal was still in its infancy and so other peers would act as supernodes to forward traffic for you.) This system used what was called "Paris Metro Pricing" which basically means you could spend your credits to move to the front of the line when it came time to pull data. It was complicated. Strong crypto end-to-end, stacking relay nodes enabled you to do Tor-like onion routing, centralized servers that would provide a persistent backing store for the peer storage when there was not a sufficient critical mass of peers for a part of the storage grid, etc. A real hairball of complexity that wheezed and groaned but somehow managed to deliver bits. "But people just want to share music and movies." said one engineer. "All of this complexity is silly. Let's cut out the persistence and make it easy for someone to just share a file." And he was right. "No." said the people running the company, "that would end up getting us sued into oblivion." After seeing what happened to others in the space at the time it appears they were right too. So our diligent engineer took an axe to the open source code. He stripped out the crypto, streamlined the payments to eliminate a central broker and just operate between two peers (e.g. tit for tat) and eliminated the persistence. Each "file" became its own distinct peer network. He created a true MVP for the system before anyone knew what that term meant. He called the system BitTorrent. And now we just need to add back the crypto and the circle will be complete... [The unfortunate reality is that the math just does not work out. It works well for ephemeral, popular data where you can maintain the peer grid, but not for a lot of files that have only a small set of peers that are interested in any particulare file. We are not talking about long-tail torrents with a few people who might be downloading at any particular time; you and your family might have a short-term interest in video's of juniors little league game but after a few weeks no one will continue to participate in that particular peer group nor are they likely to rejoin, but the service must still maintain the backing store. Perhaps this makes sense as a pivot to try to re-capture a large set of peers for some other, more valuable, distribution service, but I honestly don't see this particular product going very far.]
tikhonj 15y ago
Ignoring the technical aspect, I really like the aesthetics of the UI. It's a nice break from the usual 3D mac-style apps that try to emulate shelving or books.
foz 15y ago
Did anyone actually bother to try this? it seems to be uTorrent.

72 points by FluidDjango 15y ago

Under the assumption that it uses something like BitTorrent as its underlying protocol, how does it take into account different versions of the same file, and which of the versions will win? While multiple clients are updating their version of the same file? I can think of some ways for this to work, but I wonder about how it's officially done. Does anyone have insight?

jwhitlark 15y ago
It's solving for this problem: http://xkcd.com/949/ http://xkcd.com/949/ It operates at the torrent level, not the file level, so while you certainly can create a new torrent with updated content, it'll show up as a new entry instead of an update. As a quick technical sketch, it streamlines torrent creation, transferring torrent references to others, torrent downloading (you either get a link your client understands, or a pre-built, quick-start client with the torrent already installed), and provides temporary cloud backed seeding. Disclaimer: I work on the project as an engineer. These comments are my own, and have nothing to do with Bittorrent the company.
- icebraining 15y ago
  So despite the post title, it's not really a direct Dropbox competitor, since it won't support realtime syncing, right? Btw, since one "donates" disk space in exchange for free service, I assume that will contain data from other people; is that protected (encrypted) or could I just peruse through your photos if I happened to be a peer chosen to seed them?
  - chimeracoder 15y ago
    I'm not sure how this is implemented, but I can imagine a case in which this would work out. First, let's assume the data is encrypted with some strong encryption algorithm and similarly strong encryption key. This provides some security, but a lot less than you might think. Because data is being stored locally for a potential attacker, it is very easy for them (hypothetically) to make a large number of copies of that data and then distribute it for brute-forcing. This dramatically decreases the amount of time it would take to crack the encryption, even by brute force. [1] However, this assumes that they have the entire data at their disposal. In a torrent, you oftentimes only have chunks (say 1 MB each), and everyone doesn't download the entire torrent. So someone may only have a few megabytes worth of data. Still that's potentially enough data to contain sensitive information, even a few pictures. However, now let's assume that the data is scrambled. This means that the data is divided into 1 MB chunks of data that are not necessarily contiguous. So a 1 GB torrent would have 1000 chunks of 1 MB each, but each chunk of size 1 MB would have data from various parts of the torrent. The scrambling mechanism could be randomized and sent (securely) to the second party so that they are the only ones able to fit the pieces of data together correctly. The protocol could enforce that individuals cannot host more than _____ chunks/MB of the same file, or even that a file has to be distributed among IP addresses from different IP blocks (allocated geographically), thus reducing the chance that a single attacker has access to large parts of the same file. It does not rule out the possibility of a coordinated attack over a large geographic region, but it reduces the risk, and it would probably suffice for most use cases. Heck, people send public Dropbox links over plaintext, so it's not like we're talking about people who place a high value on security anyway (or, at least, people who think that their data is valuable enough to be worth the effort to crack). [1] There's a nice website that illustrates the advantage of having local copies of the encrypted data, but I can't find it now. It was basically a password strength-checking website that tells you how long it would take to crack your password by various methods.
    - icebraining 15y ago
      Oh, I'm sure it can be protected, no doubt. And you probably don't even need scrambling, just generate a random AES key for each torrent and distribute it only to the people who you shared the torrent with - passwords aren't needed. I was just looking if it was actually protected or assumed public.

mikemoka 15y ago

does it work this way? user shares file > the system stores the file on S3 > the system distributes the shared file through torrent > "enough" torrent peers are reached > the system REMOVES the file from S3 > a week goes by and the file loses momentum > available peers drop > the file is lost.

jwhitlark 15y ago
The file isn't lost, it's just not seeded from S3 anymore. It's not a backup service, it's a way to transfer large items to family/friends. If it had existed at the time, I'd have used it to send my wedding photos and videos, for instance, instead of burning them to a dvd and mailing them.
- mikemoka 15y ago
  ok but you can't present it as a true dropbox alternative then
  - jwhitlark 15y ago
    Interestingly enough, we don't present it as a true dropbox alternative. It's a different use case: share, not sync. The dropbox comparison seems to be pure editorializing. It's been enlightening to see the difference between what I work on, and what the articles say. Like a game of telephone.

scrod 15y ago

>The system isn’t up and running yet, but the idea is that users will receive free storage for their files by sharing some hard drive space and bandwidth with other users. This is how Wuala works, if I understand correctly. And let's not forget AeroFS. While these products are somewhat similar, I absolutely love seeing apps of this nature (syncing distributed P2P) proliferate.

someperson 15y ago
Is AeroFS even alive? Waiting a few weeks or months for an early product to be perfected is fine, but it's been more than a year. This is ridiculous. AeroFS want users to invite their friends to put themselves higher on an invite list... For a product that will use MY hard drive space and bandwidth.
- scrod 15y ago
  I've been using it on my computers as well as with a friend for at least a few months now — it works pretty well. And it uses disk space only to store your own files plus those that others share with you, the same as Dropbox.
- LogicX 15y ago
  Agreed - I loved the concept, but couldn't get it to work the way I needed it to, even with the main dev's help. (It took weeks to sync a 50GB folder over a gigabit lan, it wouldn't properly recognize the folder if I rsync'd it first - it duplicated everything) So I abandoned it for now, until such time that it got more mature and worked for me -- but I just visited the site a few days ago, and saw its still private beta. Though looking again just now, I found the changelogs: http://support.aerofs.com/customer/portal/articles/240198-aerofs-ea29-1-update http://support.aerofs.com/customer/portal/articles/240198-ae... and it looks very relevant to my issue, so I may have to give it a try again!
  - someperson 15y ago
    I'm complaining that I never got an invite, even after an entire year! I emailed the devs a few weeks back and never got a response. I want to use their product! If their product is good, I'll tell my friends. Not before I've even tried it.

abegrill 15y ago

When it comes to p2p sync TonidoSync is probably the best one out there. It has clients for all the leading mobile platforms (iOS, Android and Windows 7). The software is also cross platform (Windows, Linux and Mac)

kristen12 15y ago

This is not sync. If u are looking for p2p sync tonidosync is probably the best one out there. Tonidosync is cross platform and it has mobile apps for all the leading mobile platforms.

codemechanic 15y ago

How it is same as dropbox?

evgen 15y ago

Once upon a time, before "p2p" was even a term and when Napster ruled the rose, there was a system called Mojo Nation. It was a distributed storage system that broke a file into distinct pieces identified by a hash and then spread those pieces out to peer nodes that would return them when asked and the map of what pieces were needed to rebuild the file was stored in a service called a tracker, which existed as public services and which you could run yourself for private files. To incentivise people to keep their agent running and serving disk space you would get credits for serving data chunks or acting as a relay node (NAT traversal was still in its infancy and so other peers would act as supernodes to forward traffic for you.) This system used what was called "Paris Metro Pricing" which basically means you could spend your credits to move to the front of the line when it came time to pull data. It was complicated. Strong crypto end-to-end, stacking relay nodes enabled you to do Tor-like onion routing, centralized servers that would provide a persistent backing store for the peer storage when there was not a sufficient critical mass of peers for a part of the storage grid, etc. A real hairball of complexity that wheezed and groaned but somehow managed to deliver bits. "But people just want to share music and movies." said one engineer. "All of this complexity is silly. Let's cut out the persistence and make it easy for someone to just share a file." And he was right. "No." said the people running the company, "that would end up getting us sued into oblivion." After seeing what happened to others in the space at the time it appears they were right too. So our diligent engineer took an axe to the open source code. He stripped out the crypto, streamlined the payments to eliminate a central broker and just operate between two peers (e.g. tit for tat) and eliminated the persistence. Each "file" became its own distinct peer network. He created a true MVP for the system before anyone knew what that term meant. He called the system BitTorrent. And now we just need to add back the crypto and the circle will be complete... [The unfortunate reality is that the math just does not work out. It works well for ephemeral, popular data where you can maintain the peer grid, but not for a lot of files that have only a small set of peers that are interested in any particulare file. We are not talking about long-tail torrents with a few people who might be downloading at any particular time; you and your family might have a short-term interest in video's of juniors little league game but after a few weeks no one will continue to participate in that particular peer group nor are they likely to rejoin, but the service must still maintain the backing store. Perhaps this makes sense as a pivot to try to re-capture a large set of peers for some other, more valuable, distribution service, but I honestly don't see this particular product going very far.]

tikhonj 15y ago

Ignoring the technical aspect, I really like the aesthetics of the UI. It's a nice break from the usual 3D mac-style apps that try to emulate shelving or books.

foz 15y ago

Did anyone actually bother to try this? it seems to be uTorrent.