4 ms·

For massive take-off, Sandstorm is almost-there with making everyone's online world de-centralized. It's solved half of the problem beautifully -- it's very ea

For massive take-off, Sandstorm is almost-there with making everyone's online world de-centralized. It's solved half of the problem beautifully -- it's very easy, once setup, to launch any new app on your server. The other half of the problem is moving of server/data. Right now, either you use their self hosted instance of sandstorm, or setup your own server. There's the hassle of keeping track of data/server. There's pseudo-centralization happening here, given the difficulty of changing compute or storage by a layperson. However, if sandstorm had built in federated storage (either based on bittorrent filesystem or any other p2p storage), then I no longer worry about losing my data. I simply move my domain to a new "server-provider" or use the main project's own service, and then at a later date can spin up my own compute, or use another provider, and point them to my data on the p2p/federated storage.

kentonv 10y ago
FWIW, it's straightforward to see how this could be accomplished with Sandstorm's model, since user data is very clearly divided into grains (app instances / documents) with clear owners, and these grains can easily be transferred between servers. Today, you can click "download backup" to get a zip, then upload it to some other server, and end up with exactly the same grain reproduced there. We plan to streamline this further, to allow mass transfers and synchronizing grains between servers. It's a major goal for us that it should be trivial to move between hosts, so that you're never stuck with a host that you don't like.
- anilgulecha 10y ago
  Good to know. Just to re-iterate one point, the ideal is that the server does not have a hold over the data (by say withholding download of backup data), or that the data is held tied to a certain server. Server and data (encrypted) should just be commodities, and the domain DNS is what should describe ownership. That's why I would like the focus to be on federation than easy import/export.
  - kentonv 10y ago
    Realistically, if you want to run code on the server, the server needs some control over the data. There are some distributed systems lately that really only distribute storage while all computation happens client-side. That doesn't work well if you want real-time collaboration (the latency is too high) or multi-user permission constraints. Not to mention, it requires rewriting the world. So, I'm skeptical of those approaches. I'd rather make it easy to get your data out when you need/want to. But it's certainly an opinion that is open to debate.
    - anilgulecha 10y ago
      Copying from a sister comment on this thread: > To ensure low latency, I'm OK with the protocol allowing a server to cache and render recent data, while a background service syncs to the de-centralized store. This would be a help for the UI from the protocol, but the ideal IMO should be something that normal SaaS companies/products hold you on for -- they have your data hostage. I understand that for sandstorm above many not be possible as you'd have to re-write a lot, and this feature does not hold sufficient priority for the relevant code re-write to happen. That's too bad though -- I was expecting this need of mine to be provided by the project. Still I wish you the best of luck as you're head and shoulders the better option above any of the current SaaS apps for a personal clouds.
  - michaelchisari 10y ago
    > Server and data (encrypted) should just be commodities They should be. But the difficulties and complexities of getting that right are what's kept that from becoming reality yet.
    - anilgulecha 10y ago
      Understood. The exciting thing about sandstorm to me was (and is) that it promised a lay-person easy server and cloud data management. To make this viable, the user would only need to pay for compute and storage. However, compute and storage providers must not be able to hold user data hostage. Yes, the ideal is hard to acheive, but, for example, nntp had shown decades ago that data storage federation is possible. It's about coming up with the right protocol. To ensure low latencies, I'm OK with the protocol allowing a server to cache and render recent data, while a background service syncs to the de-centralized store.
- deleted 10y ago
  [deleted]
qznc 10y ago
I don't believe built in federation would be a good idea. There should be a reasonable business in hosting sandstorm. I would like a german or european Sandstorm hoster for example. This would mean my data stays within my jurisdiction and my privacy rights can be enforced. Any data which escapes to the US is open since Privacy Shield is a political joke.
- anilgulecha 10y ago
  The idea of the data staing once place is fine, but that can be where the key resides? The domain information acts as identity and perhaps also the location of the above key. Businesses can be built on commodity storage/compute. But these cannot be given the keys to the kingdom, which you currently do with any saas service.
Taek 10y ago
Spent the past 2.5 years building a decentralized cloud storage platform (Sia), it's still in the early days of usable, but maybe we can help somehow here? Today, you need to save all of your metadata or you can't retrieve the files. But it's a lot easier to do a safe/broad backup of 10MB than XXXXGB. At some point down the road, I think we can have it as simple as 'private key in, your data out' Ina decentralizated netowrk, as your key is used to contact a bunch of hosts in a 1-of-N failure mode (one honest host is all you need to get the right data) to fetch your metadata.

kentonv 10y ago

FWIW, it's straightforward to see how this could be accomplished with Sandstorm's model, since user data is very clearly divided into grains (app instances / documents) with clear owners, and these grains can easily be transferred between servers. Today, you can click "download backup" to get a zip, then upload it to some other server, and end up with exactly the same grain reproduced there. We plan to streamline this further, to allow mass transfers and synchronizing grains between servers. It's a major goal for us that it should be trivial to move between hosts, so that you're never stuck with a host that you don't like.

anilgulecha 10y ago
Good to know. Just to re-iterate one point, the ideal is that the server does not have a hold over the data (by say withholding download of backup data), or that the data is held tied to a certain server. Server and data (encrypted) should just be commodities, and the domain DNS is what should describe ownership. That's why I would like the focus to be on federation than easy import/export.
- kentonv 10y ago
  Realistically, if you want to run code on the server, the server needs some control over the data. There are some distributed systems lately that really only distribute storage while all computation happens client-side. That doesn't work well if you want real-time collaboration (the latency is too high) or multi-user permission constraints. Not to mention, it requires rewriting the world. So, I'm skeptical of those approaches. I'd rather make it easy to get your data out when you need/want to. But it's certainly an opinion that is open to debate.
  - anilgulecha 10y ago
    Copying from a sister comment on this thread: > To ensure low latency, I'm OK with the protocol allowing a server to cache and render recent data, while a background service syncs to the de-centralized store. This would be a help for the UI from the protocol, but the ideal IMO should be something that normal SaaS companies/products hold you on for -- they have your data hostage. I understand that for sandstorm above many not be possible as you'd have to re-write a lot, and this feature does not hold sufficient priority for the relevant code re-write to happen. That's too bad though -- I was expecting this need of mine to be provided by the project. Still I wish you the best of luck as you're head and shoulders the better option above any of the current SaaS apps for a personal clouds.
- michaelchisari 10y ago
  > Server and data (encrypted) should just be commodities They should be. But the difficulties and complexities of getting that right are what's kept that from becoming reality yet.
  - anilgulecha 10y ago
    Understood. The exciting thing about sandstorm to me was (and is) that it promised a lay-person easy server and cloud data management. To make this viable, the user would only need to pay for compute and storage. However, compute and storage providers must not be able to hold user data hostage. Yes, the ideal is hard to acheive, but, for example, nntp had shown decades ago that data storage federation is possible. It's about coming up with the right protocol. To ensure low latencies, I'm OK with the protocol allowing a server to cache and render recent data, while a background service syncs to the de-centralized store.
deleted 10y ago
[deleted]

qznc 10y ago

I don't believe built in federation would be a good idea. There should be a reasonable business in hosting sandstorm. I would like a german or european Sandstorm hoster for example. This would mean my data stays within my jurisdiction and my privacy rights can be enforced. Any data which escapes to the US is open since Privacy Shield is a political joke.

anilgulecha 10y ago
The idea of the data staing once place is fine, but that can be where the key resides? The domain information acts as identity and perhaps also the location of the above key. Businesses can be built on commodity storage/compute. But these cannot be given the keys to the kingdom, which you currently do with any saas service.

Taek 10y ago

Spent the past 2.5 years building a decentralized cloud storage platform (Sia), it's still in the early days of usable, but maybe we can help somehow here? Today, you need to save all of your metadata or you can't retrieve the files. But it's a lot easier to do a safe/broad backup of 10MB than XXXXGB. At some point down the road, I think we can have it as simple as 'private key in, your data out' Ina decentralizated netowrk, as your key is used to contact a bunch of hosts in a 1-of-N failure mode (one honest host is all you need to get the right data) to fetch your metadata.