7 ms·

Dropbox rival Space Monkey puts 'cloud' in your house

52 points by ivoflipse 15y ago

sek 15y ago
This is a great idea for the long term. There will always be a need for something like this, as an alternative to the cloud. Dropbox should have come up with this, to differentiate themselves from their competitors. Sounds like a good acquisition candidate.
kijin 15y ago
So, the company gives you a 2TB external hard drive for free, on the condition that you use 1TB to store your own data and 1TB to store copies of other people's encrypted data? Sounds interesting, until you get a huge bill for bandwidth overage. I have barely 100GB of bandwidth to spare every month (out of a total of 300GB/mo), so if that hard drive gets filled in less than 10 months, we've got a problem. They're effectively pushing their costs onto ISPs, who in turn will push their costs onto customers. Restoring backups is also going to be pretty slow if your data is stored on somebody's 10M/1M DSL line. They'd better market it in countries where everyone has 100M and no bandwidth caps.
- eli 15y ago
  It seems to be working ok for Spotify
- andrewgodwin 15y ago
  Not to mention the obvious legal ramifications, even of encrypted data - in the UK, it's a crime not to give up the key to encrypted data, even if you claim to not have it (though I would hope in this case it's provable you really never did, and that would somehow help.)
  - the_paul 15y ago
    It's not dissimilar to the legal ramifications of browsing HN, then. Look, encrypted data! xQq84/+wpysSnUveVUX6592sHzGBptxysvtginJr2OFkDifXYtNCFg==
- divtxt 15y ago
  Yes, bandwidth is a concern, but it should be easy to let users choose bandwidth limits. The question then is whether the product is still useful if it takes 10/20/30 months to fill the hard drive. I suspect that there's a segment of the population for which it will.
- jtolds 15y ago
  ISPs pay for bandwidth leaving and entering their network. inter-network bandwidth is actually free for them, so it's very likely spacemonkey could prefer peering with other users on the same ISP.
  - kijin 15y ago
    That would certainly help, but there's a limit to same-ISP peering. For example, I use a relatively small ISP, so not many people are going to be using the same ISP that I do. Even if you use a larger ISP, you're screwed if even a single person dumps 100GB/mo of data on your SpaceMonkey drive.
  - wmf 15y ago
    Upstream bandwidth on DOCSIS causes congestion that requires capex (node splits) to fix, so it's definitely not free. See http://www.ietf.org/proceedings/74/slides/alto-1.pdf http://www.ietf.org/proceedings/74/slides/alto-1.pdf
- kokey 15y ago
  It's funny, it's just as the world started to become more sane again by storing data centrally now we have a company that will bring us back to storing things locally and share it out on links that at the last mile are not implemented to be full duplex. The economics of this is just wrong, and hopefully it won't take off.
hndl 15y ago
I think this is a great idea but I think Dropbox has a pretty well established brand name. My guess is that most people aren't going to find the need to switch. The ones who might make the switch are the paranoid folks who would think this is some sort of more secure way to sync. The idea is nice but the challenge, IMO, will be in marketing this.
- eli 15y ago
  Dropbox is also kinda expensive. $125/year per user starts to add up if you've got a lot of users.
  - hndl 15y ago
    How did you arrive at $125/user? A link for the unaware?
    - tomku 15y ago
      $125/user is the amount it costs to add another user to a Dropbox for Teams account after your first five users. https://www2.dropbox.com/teams/pricing https://www2.dropbox.com/teams/pricing
  - typicalrunt 15y ago
    It depends on your financial situation in life. Instead of thinking "expensive" versus "cheap" -- because they're just words with no clear definition -- you need to think about the cost for what you are getting. $125/year is just over $10/month. Nowadays in the US/UK/Canada, that's about 2 cups of coffee at Starbucks. For that price, you are receiving 1TB of backup space for yourself. Each person is $10/month, which is useful if you need to collaborate and not worry about setup. Insurance on your home always varies, but usually runs about $50-100 per month. And that's just the price you pay for when things are going well. For me, there's a cost to peace of mind and the ability for me to save time in my life to do things other than maintenance. FWIW, I look at Dropbox as a monthly insurance premium to ensure that my files will always be around and I don't need to worry that my in-home RAID server must blow up at any time.
dpark 15y ago
They give me half of a 2TB drive? Doesn't that mean that I'm getting only one copy of my data stored outside my house? So if a power surge fries my equipment, there's one copy of my data spread across a couple dozen other machines. I guess that's fine as long as no one but me experiences any failures... It's possible that they're just overselling, and they expect most people to use only a couple hundred megs, which would allow lots of redundancy. If so, that seems like a really bad idea, since heavy usage of an on-site disk is likely (let's back up all these pirated movies!), and their buildout requires either signing up new customers or swapping devices, neither of which is quick or cheap.
- divtxt 15y ago
  I wish they had gone with At least 2 copies offsite - that was one of my first thoughts too. Then I remember that I used to do backups at most once a year until I moved to the Mac and Time Machine. (edit: clarify sentence)
  - dpark 15y ago
    How do they guarantee at least two offsite copies? That would require that they give everyone 1TB out of 3TB. Unless they're also storing on their own servers, in which case the consumer-housed device seems pointless. And two copies is still pretty low for flaky infrastructure. They're basically setting up your data as a torrent cloud, except there are no seeds if you lose your equipment, so you're trying to download fragments from a bunch of other machines, and each fragment is only on a couple of machines.
    - divtxt 15y ago
      What I meant was: I wish they had gone with at least 2 copies offsite. Sorry for the ambiguity. And you're right - 2 copies would mean 1TB out of 3TB.
      - dpark 15y ago
        Ah, thanks for clarifying.
alenahemkova 15y ago
Didn't Wuala use to do something like this? Except more for P2P file backup rather than sync?
- jbellis 15y ago
  P2P backup / sync has been done before (allmydata is another example from longer ago). It's that plus the appliance that makes Space Monkey unique, IMO.
  - alenahemkova 15y ago
    IIRC what wuala did was give people more storage if they give up a part of their hard drive for other to sync with, so it's exactly the same model except without external hardware and more flexibility in choosing how much storage you want to trade off. Wuala also keep a centralize server in addition to this P2P cloud so I think the purpose of the cloud was more for bandwidth.
    - jtolds 15y ago
      it's one thing to pay money and gain space, but it's another thing to pay money and lose disk space you have. i feel like people are more likely to pick the former.
cbs 15y ago
This sounds awesome! I've been waiting to see a step this direction for a long time. For a lot of people dropbox is the best tool for files, but its ridiculous for the trip between two computers in the same room to have a layover in A3.
- jpschorr 15y ago
  That's why dropbox has lan sync: https://www.dropbox.com/help/137 https://www.dropbox.com/help/137
  - cbs 15y ago
    That has a few big catches, including maintaining an internet connection and being in the same broadcast domain. And looks like it doesn't have lan-only drives, I maintain a box serving up samba shares specifically because the work files in a specific project work files would blow my bandwidth cap for the month in 2 round trips.
    - dpark 15y ago
      > That has a few big catches, including maintaining an internet connection and being in the same broadcast domain. Are those actual big catches? Those seem like pretty reasonable constraints. What machine do you have attached to your LAN that can't access the internet? And the network restriction is probably just fine for >99% of their customers.
      - cbs 15y ago
        I know its not going to impact the vast majority of their users, so they don't have to care. For my uses, they're all pretty big. That's why I'm so happy to see a new product that may fit into those situations better. Not every computer is on a SOHO network, or a network with many properties of the same. The network I'm on most of the time is huge which means its heavily subnetted and no one user gets great throughput. My apartment has the aforementioned cap. My old man's place has no internet connection at all, just wifi for getting files to the media computer connected to the TV. By building a product that can serve these users, they'll have a product that can better serve users on ideal connections too!
dpark 15y ago
Actually, you know who could probably sell this? Comcast. Make a single device that serves as a cable modem and also has this storage angle built in as either a "free" service with the bundle or as a small upsell.
- wmf 15y ago
  I think Comcast wants to reduce upstream congestion, not increase it. Edit: I'm talking about upstream on the last mile.
  - dpark 15y ago
    Why? If anything, I'd expect them to have plenty of upstream bandwidth, given that typical traffic patterns are primarily downstream.
    - sek 15y ago
      They sell the upstream to hosting providers. http://en.wikipedia.org/wiki/Peering http://en.wikipedia.org/wiki/Peering
      - dpark 15y ago
        I don't see how this is relevant. If Comcast were to sell this (or an equivalent device), they'd keep all the traffic within their own network.
        sek 15y ago
        When you have a peering contract, you offer your own network. Every bit not used is so valuable for cable providers, we consumers forget that because we all have flatrates.
        dpark 15y ago
        I don't see how this is valuable. By definition a peering contract is not staying inside Comcast's network. The bandwidth for this (if Comcast were selling it) would stay entirely within the network, where it's not significantly competing with peered traffic. Maybe I'm misunderstanding what you're suggesting.
typicalrunt 15y ago
Interesting idea, but this worries me: When you sign up for Space Monkey, you get a physical networked hard disk with 1TB of storage free for you to use. It's actually a 2TB drive; the rest gets used by other people on the system. So 1TB is filled with other people's data? What about security? Dropbox had a simple security issue that potentially leaked all users' data. But now, the data is on your drive. You could mirror the drive ('dd') and try to break into other user's information without anyone suspecting anything. Or what happens if someone stores child pr0n on SpaceMonkey, and it gets mirrored to your drive? Legally, are you now on the hook for possessing the information since it's on your device?
- tomstokes 15y ago
  Good question. The Space Monkey software mitigates these issues in two ways: First, all data is carefully encrypted before it ever leaves the user's Space Monkey device for replication across the network. This is obviously a key concern for all of our potential customers, so a lot of time has been spent getting this part right. Second, the system breaks each file into many small blocks with built-in parity and other such redundancy measures. These blocks are then spread across many devices to both increase redundancy of the network and distribute the bandwidth usage fairly across nodes. As a result, you wouldn't really ever store any identifiable data from any one other customer. Therefore, if the government wanted to seize evidence, they would gain nothing useful by seizing any Space Monkey device other than the suspect's, which contains the local cache of the user's own data.
- the_paul 15y ago
  This is one of the most straightforward possible applications for cryptography in data storage- no worries about key negotiation or cipher negotiation or PKI infrastructures. And yes, they're encrypting your data before sending it anywhere- that's pretty explicitly mentioned everywhere the service is explained. So, if you're asking whether you're legally liable for the content of data on your hard drive which you have no possibility of decrypting, um, the answer is "no".
  - kijin 15y ago
wmf 15y ago
This has been shown not to work: http://people.cs.umass.edu/~emery/pubs/TFS.pdf http://people.cs.umass.edu/~emery/pubs/TFS.pdf (section 5.1) I wonder how they think they're getting around the upstream replication bottleneck.
- jbellis 15y ago
  Upvoted, but I skimmed 5.1 and missed the "shown not to work" part. Can you be more specific?
  - wmf 15y ago
    Whenever a device fails or goes offline, the data that was stored there needs to be re-replicated from the source. But figure 6 shows that even with a 2 Mbps "turbo" uplink you can only replicate ~25 GB; if you have more data than that, replicas will fail faster than you can recreate them.
    - jbellis 15y ago
      Sounds like that's addressed by storing pieces of data from many sources in the non-local part of space monkey storage: even if each source only has a 256K uplink, multiply that by 100 or 1000 and you're in pretty good shape.
- deleted 15y ago
  [deleted]
- toomuchtodo 15y ago
  Probably by preferring storage nodes on faster connections.
calanya 15y ago
I'm glad to see Dropbox as defining this space.
takinola 15y ago
I would love to see a Time machine equivalent for Windows. I already have a 2TB storage device at home but pretty much every local backup/sync service sucks cojones. I'd gladly pay a flat fee for a product that does this
JVIDEL 15y ago
And here I thought it was an actually solid rival to dropbox you could install in your own servers and enjoy the same level of functionaly. Yes I know there are some hacks around using FTP apps to get a similar experience, but is not the same, is still clunky, sometimes too buggy, and let alone the average dropbox user wouldn't be able to even read the instructions, let alone put it together.

52 points by ivoflipse 15y ago

sek 15y ago

This is a great idea for the long term. There will always be a need for something like this, as an alternative to the cloud. Dropbox should have come up with this, to differentiate themselves from their competitors. Sounds like a good acquisition candidate.

kijin 15y ago

So, the company gives you a 2TB external hard drive for free, on the condition that you use 1TB to store your own data and 1TB to store copies of other people's encrypted data? Sounds interesting, until you get a huge bill for bandwidth overage. I have barely 100GB of bandwidth to spare every month (out of a total of 300GB/mo), so if that hard drive gets filled in less than 10 months, we've got a problem. They're effectively pushing their costs onto ISPs, who in turn will push their costs onto customers. Restoring backups is also going to be pretty slow if your data is stored on somebody's 10M/1M DSL line. They'd better market it in countries where everyone has 100M and no bandwidth caps.

eli 15y ago
It seems to be working ok for Spotify
andrewgodwin 15y ago
Not to mention the obvious legal ramifications, even of encrypted data - in the UK, it's a crime not to give up the key to encrypted data, even if you claim to not have it (though I would hope in this case it's provable you really never did, and that would somehow help.)
- the_paul 15y ago
  It's not dissimilar to the legal ramifications of browsing HN, then. Look, encrypted data! xQq84/+wpysSnUveVUX6592sHzGBptxysvtginJr2OFkDifXYtNCFg==
divtxt 15y ago
Yes, bandwidth is a concern, but it should be easy to let users choose bandwidth limits. The question then is whether the product is still useful if it takes 10/20/30 months to fill the hard drive. I suspect that there's a segment of the population for which it will.
jtolds 15y ago
ISPs pay for bandwidth leaving and entering their network. inter-network bandwidth is actually free for them, so it's very likely spacemonkey could prefer peering with other users on the same ISP.
- kijin 15y ago
  That would certainly help, but there's a limit to same-ISP peering. For example, I use a relatively small ISP, so not many people are going to be using the same ISP that I do. Even if you use a larger ISP, you're screwed if even a single person dumps 100GB/mo of data on your SpaceMonkey drive.
- wmf 15y ago
  Upstream bandwidth on DOCSIS causes congestion that requires capex (node splits) to fix, so it's definitely not free. See http://www.ietf.org/proceedings/74/slides/alto-1.pdf http://www.ietf.org/proceedings/74/slides/alto-1.pdf
kokey 15y ago
It's funny, it's just as the world started to become more sane again by storing data centrally now we have a company that will bring us back to storing things locally and share it out on links that at the last mile are not implemented to be full duplex. The economics of this is just wrong, and hopefully it won't take off.

hndl 15y ago

I think this is a great idea but I think Dropbox has a pretty well established brand name. My guess is that most people aren't going to find the need to switch. The ones who might make the switch are the paranoid folks who would think this is some sort of more secure way to sync. The idea is nice but the challenge, IMO, will be in marketing this.

eli 15y ago
Dropbox is also kinda expensive. $125/year per user starts to add up if you've got a lot of users.
- hndl 15y ago
  How did you arrive at $125/user? A link for the unaware?
  - tomku 15y ago
    $125/user is the amount it costs to add another user to a Dropbox for Teams account after your first five users. https://www2.dropbox.com/teams/pricing https://www2.dropbox.com/teams/pricing
- typicalrunt 15y ago
  It depends on your financial situation in life. Instead of thinking "expensive" versus "cheap" -- because they're just words with no clear definition -- you need to think about the cost for what you are getting. $125/year is just over $10/month. Nowadays in the US/UK/Canada, that's about 2 cups of coffee at Starbucks. For that price, you are receiving 1TB of backup space for yourself. Each person is $10/month, which is useful if you need to collaborate and not worry about setup. Insurance on your home always varies, but usually runs about $50-100 per month. And that's just the price you pay for when things are going well. For me, there's a cost to peace of mind and the ability for me to save time in my life to do things other than maintenance. FWIW, I look at Dropbox as a monthly insurance premium to ensure that my files will always be around and I don't need to worry that my in-home RAID server must blow up at any time.

dpark 15y ago

They give me half of a 2TB drive? Doesn't that mean that I'm getting only one copy of my data stored outside my house? So if a power surge fries my equipment, there's one copy of my data spread across a couple dozen other machines. I guess that's fine as long as no one but me experiences any failures... It's possible that they're just overselling, and they expect most people to use only a couple hundred megs, which would allow lots of redundancy. If so, that seems like a really bad idea, since heavy usage of an on-site disk is likely (let's back up all these pirated movies!), and their buildout requires either signing up new customers or swapping devices, neither of which is quick or cheap.

divtxt 15y ago
I wish they had gone with At least 2 copies offsite - that was one of my first thoughts too. Then I remember that I used to do backups at most once a year until I moved to the Mac and Time Machine. (edit: clarify sentence)
- dpark 15y ago
  How do they guarantee at least two offsite copies? That would require that they give everyone 1TB out of 3TB. Unless they're also storing on their own servers, in which case the consumer-housed device seems pointless. And two copies is still pretty low for flaky infrastructure. They're basically setting up your data as a torrent cloud, except there are no seeds if you lose your equipment, so you're trying to download fragments from a bunch of other machines, and each fragment is only on a couple of machines.
  - divtxt 15y ago
    What I meant was: I wish they had gone with at least 2 copies offsite. Sorry for the ambiguity. And you're right - 2 copies would mean 1TB out of 3TB.
    - dpark 15y ago
      Ah, thanks for clarifying.

alenahemkova 15y ago

Didn't Wuala use to do something like this? Except more for P2P file backup rather than sync?

jbellis 15y ago
P2P backup / sync has been done before (allmydata is another example from longer ago). It's that plus the appliance that makes Space Monkey unique, IMO.
- alenahemkova 15y ago
  IIRC what wuala did was give people more storage if they give up a part of their hard drive for other to sync with, so it's exactly the same model except without external hardware and more flexibility in choosing how much storage you want to trade off. Wuala also keep a centralize server in addition to this P2P cloud so I think the purpose of the cloud was more for bandwidth.
  - jtolds 15y ago
    it's one thing to pay money and gain space, but it's another thing to pay money and lose disk space you have. i feel like people are more likely to pick the former.

cbs 15y ago

This sounds awesome! I've been waiting to see a step this direction for a long time. For a lot of people dropbox is the best tool for files, but its ridiculous for the trip between two computers in the same room to have a layover in A3.

jpschorr 15y ago
That's why dropbox has lan sync: https://www.dropbox.com/help/137 https://www.dropbox.com/help/137
- cbs 15y ago
  That has a few big catches, including maintaining an internet connection and being in the same broadcast domain. And looks like it doesn't have lan-only drives, I maintain a box serving up samba shares specifically because the work files in a specific project work files would blow my bandwidth cap for the month in 2 round trips.
  - dpark 15y ago
    > That has a few big catches, including maintaining an internet connection and being in the same broadcast domain. Are those actual big catches? Those seem like pretty reasonable constraints. What machine do you have attached to your LAN that can't access the internet? And the network restriction is probably just fine for >99% of their customers.
    - cbs 15y ago
      I know its not going to impact the vast majority of their users, so they don't have to care. For my uses, they're all pretty big. That's why I'm so happy to see a new product that may fit into those situations better. Not every computer is on a SOHO network, or a network with many properties of the same. The network I'm on most of the time is huge which means its heavily subnetted and no one user gets great throughput. My apartment has the aforementioned cap. My old man's place has no internet connection at all, just wifi for getting files to the media computer connected to the TV. By building a product that can serve these users, they'll have a product that can better serve users on ideal connections too!

dpark 15y ago

Actually, you know who could probably sell this? Comcast. Make a single device that serves as a cable modem and also has this storage angle built in as either a "free" service with the bundle or as a small upsell.

wmf 15y ago
I think Comcast wants to reduce upstream congestion, not increase it. Edit: I'm talking about upstream on the last mile.
- dpark 15y ago
  Why? If anything, I'd expect them to have plenty of upstream bandwidth, given that typical traffic patterns are primarily downstream.
  - sek 15y ago
    They sell the upstream to hosting providers. http://en.wikipedia.org/wiki/Peering http://en.wikipedia.org/wiki/Peering
    - dpark 15y ago
      I don't see how this is relevant. If Comcast were to sell this (or an equivalent device), they'd keep all the traffic within their own network.
      - sek 15y ago
        When you have a peering contract, you offer your own network. Every bit not used is so valuable for cable providers, we consumers forget that because we all have flatrates.
        dpark 15y ago
        I don't see how this is valuable. By definition a peering contract is not staying inside Comcast's network. The bandwidth for this (if Comcast were selling it) would stay entirely within the network, where it's not significantly competing with peered traffic. Maybe I'm misunderstanding what you're suggesting.

typicalrunt 15y ago

Interesting idea, but this worries me: When you sign up for Space Monkey, you get a physical networked hard disk with 1TB of storage free for you to use. It's actually a 2TB drive; the rest gets used by other people on the system. So 1TB is filled with other people's data? What about security? Dropbox had a simple security issue that potentially leaked all users' data. But now, the data is on your drive. You could mirror the drive ('dd') and try to break into other user's information without anyone suspecting anything. Or what happens if someone stores child pr0n on SpaceMonkey, and it gets mirrored to your drive? Legally, are you now on the hook for possessing the information since it's on your device?

tomstokes 15y ago
Good question. The Space Monkey software mitigates these issues in two ways: First, all data is carefully encrypted before it ever leaves the user's Space Monkey device for replication across the network. This is obviously a key concern for all of our potential customers, so a lot of time has been spent getting this part right. Second, the system breaks each file into many small blocks with built-in parity and other such redundancy measures. These blocks are then spread across many devices to both increase redundancy of the network and distribute the bandwidth usage fairly across nodes. As a result, you wouldn't really ever store any identifiable data from any one other customer. Therefore, if the government wanted to seize evidence, they would gain nothing useful by seizing any Space Monkey device other than the suspect's, which contains the local cache of the user's own data.
the_paul 15y ago
This is one of the most straightforward possible applications for cryptography in data storage- no worries about key negotiation or cipher negotiation or PKI infrastructures. And yes, they're encrypting your data before sending it anywhere- that's pretty explicitly mentioned everywhere the service is explained. So, if you're asking whether you're legally liable for the content of data on your hard drive which you have no possibility of decrypting, um, the answer is "no".
- kijin 15y ago

wmf 15y ago

This has been shown not to work: http://people.cs.umass.edu/~emery/pubs/TFS.pdf http://people.cs.umass.edu/~emery/pubs/TFS.pdf (section 5.1) I wonder how they think they're getting around the upstream replication bottleneck.

jbellis 15y ago
Upvoted, but I skimmed 5.1 and missed the "shown not to work" part. Can you be more specific?
- wmf 15y ago
  Whenever a device fails or goes offline, the data that was stored there needs to be re-replicated from the source. But figure 6 shows that even with a 2 Mbps "turbo" uplink you can only replicate ~25 GB; if you have more data than that, replicas will fail faster than you can recreate them.
  - jbellis 15y ago
    Sounds like that's addressed by storing pieces of data from many sources in the non-local part of space monkey storage: even if each source only has a 256K uplink, multiply that by 100 or 1000 and you're in pretty good shape.
deleted 15y ago
[deleted]
toomuchtodo 15y ago
Probably by preferring storage nodes on faster connections.

calanya 15y ago

I'm glad to see Dropbox as defining this space.

takinola 15y ago

I would love to see a Time machine equivalent for Windows. I already have a 2TB storage device at home but pretty much every local backup/sync service sucks cojones. I'd gladly pay a flat fee for a product that does this

JVIDEL 15y ago

And here I thought it was an actually solid rival to dropbox you could install in your own servers and enjoy the same level of functionaly. Yes I know there are some hacks around using FTP apps to get a similar experience, but is not the same, is still clunky, sometimes too buggy, and let alone the average dropbox user wouldn't be able to even read the instructions, let alone put it together.