5 ms·

I really would love to see decentralized apps, but I don't see how to get around the "back end storage" problem. If you have a ton of data (e.g., video) how and

I really would love to see decentralized apps, but I don't see how to get around the "back end storage" problem. If you have a ton of data (e.g., video) how and where does this get stored in a decentralized system? It seems impossible to compete with the fast and reliable infrastructure of the big folks (e.g. Amazon, Google, MS, ...).

sanity31415 5y ago
Locutus is more like a cache than long-term storage, so data can fall out of the network if nobody is requesting it, but information publishers can serve as a backup for data stored in the network, replacing it if it falls out. I think making the publisher responsible for long-term retention and the decentralized network responsible for distribution is the best option here. Other tools like FileCoin take the alternate approach of trying to do long-term storage in a decentralized way.
wmf 5y ago
A bunch of projects are working on storage: Filecoin, Sia, Storj, etc.
- trappist 5y ago
  Not just working on it. We use Storj in production and it's dramatically cheaper than S3.
rklaehn 5y ago
There are plenty of options for long term storage of large assets in a decentralised way. Also, decentralised does not have to mean that you don't use traditional cloud providers. What is missing is a really polished way to do decentralized apps, so that you don't have to think about the decentralized aspect all the time, but just use it because it is the best option. But it is coming. Full disclosure: hacking on something to make writing decentralized apps easier: https://github.com/cloudpeers/tlfs https://github.com/cloudpeers/tlfs
- indigochill 5y ago
  > so that you don't have to think about the decentralized aspect all the time Depending how strict you want to be about this, torrents and Secure Scuttlebutt are pretty good in this respect. I especially love Secure Scuttlebutt because it simply makes the client the store of which messages it's seen which also means there is no authoritative view of the whole network but that's by design. But probably the even better example here is email. It's the OG federated protocol and nobody thinks of it that way. Probably because casual users have consolidated around a handful of convenient clients (gmail, hotmail, whatever) and not thought too hard about how emails get exchanged from one corporate mail server to another.
  - rklaehn 5y ago
    Yes, there is a lot of awesome stuff that just works. SSB, the entire DAT/hypercore community, tools like syncthing that just do one thing well. I like the SSB approach of just embracing subjectivity. I was quite amazed when discovering all this at dtn.is (Before I only knew the IPFS community). But I feel there is still something missing for very wide spread adoption. What we are trying to do with TLFS is to combine - a CRDT with strict schema ( Adapted from https://arxiv.org/abs/1603.01529 https://arxiv.org/abs/1603.01529 ) - a built in way to do schema evolution ( Adapted from https://www.inkandswitch.com/cambria/ https://www.inkandswitch.com/cambria/ ) - local first / p2p deployment and operation even for native apps, based on https://github.com/libp2p/rust-libp2p https://github.com/libp2p/rust-libp2p , faciliated by https://github.com/cloudpeers/x https://github.com/cloudpeers/x We're not nearly there yet, but ideally it should be so easy to write small collaborative apps that it is a good option even if you don't care about the idealistic motivations for decentralized apps.
BlueTemplar 5y ago
I've been quite surprised by the reliability of Popcorn Time (relatively bad, but torrents have not been designed with this in mind, have they?), and especially PeerTube (how is that even possible ?!).
- jjeaff 5y ago
  Isn't decentralized sharing with resiliency exactly what torrents we're created for?
  - BlueTemplar 5y ago
    I mean for video streaming. (Unlike, say, eMule, which had "get first packets first" option implemented out of the box ?)
Ambolia 5y ago
This is a problem, but this is a problem with centralized websites too. How many links to broken blogs, broken youtube videos or to broken tweets have you ever encountered? Heck, most people don't even link to tweets anymore, they just post screenshots. The current internet is terrible at archiving information.
cryptome12345 5y ago
Why not starting paying them for their services on a decentralized network like BTC pays for computer power? I can see a decentralized design for paid hosting like that.

sanity31415 5y ago

Locutus is more like a cache than long-term storage, so data can fall out of the network if nobody is requesting it, but information publishers can serve as a backup for data stored in the network, replacing it if it falls out. I think making the publisher responsible for long-term retention and the decentralized network responsible for distribution is the best option here. Other tools like FileCoin take the alternate approach of trying to do long-term storage in a decentralized way.

wmf 5y ago

A bunch of projects are working on storage: Filecoin, Sia, Storj, etc.

trappist 5y ago
Not just working on it. We use Storj in production and it's dramatically cheaper than S3.

rklaehn 5y ago

There are plenty of options for long term storage of large assets in a decentralised way. Also, decentralised does not have to mean that you don't use traditional cloud providers. What is missing is a really polished way to do decentralized apps, so that you don't have to think about the decentralized aspect all the time, but just use it because it is the best option. But it is coming. Full disclosure: hacking on something to make writing decentralized apps easier: https://github.com/cloudpeers/tlfs https://github.com/cloudpeers/tlfs

indigochill 5y ago
> so that you don't have to think about the decentralized aspect all the time Depending how strict you want to be about this, torrents and Secure Scuttlebutt are pretty good in this respect. I especially love Secure Scuttlebutt because it simply makes the client the store of which messages it's seen which also means there is no authoritative view of the whole network but that's by design. But probably the even better example here is email. It's the OG federated protocol and nobody thinks of it that way. Probably because casual users have consolidated around a handful of convenient clients (gmail, hotmail, whatever) and not thought too hard about how emails get exchanged from one corporate mail server to another.
- rklaehn 5y ago
  Yes, there is a lot of awesome stuff that just works. SSB, the entire DAT/hypercore community, tools like syncthing that just do one thing well. I like the SSB approach of just embracing subjectivity. I was quite amazed when discovering all this at dtn.is (Before I only knew the IPFS community). But I feel there is still something missing for very wide spread adoption. What we are trying to do with TLFS is to combine - a CRDT with strict schema ( Adapted from https://arxiv.org/abs/1603.01529 https://arxiv.org/abs/1603.01529 ) - a built in way to do schema evolution ( Adapted from https://www.inkandswitch.com/cambria/ https://www.inkandswitch.com/cambria/ ) - local first / p2p deployment and operation even for native apps, based on https://github.com/libp2p/rust-libp2p https://github.com/libp2p/rust-libp2p , faciliated by https://github.com/cloudpeers/x https://github.com/cloudpeers/x We're not nearly there yet, but ideally it should be so easy to write small collaborative apps that it is a good option even if you don't care about the idealistic motivations for decentralized apps.

BlueTemplar 5y ago

I've been quite surprised by the reliability of Popcorn Time (relatively bad, but torrents have not been designed with this in mind, have they?), and especially PeerTube (how is that even possible ?!).

jjeaff 5y ago
Isn't decentralized sharing with resiliency exactly what torrents we're created for?
- BlueTemplar 5y ago
  I mean for video streaming. (Unlike, say, eMule, which had "get first packets first" option implemented out of the box ?)

Ambolia 5y ago

This is a problem, but this is a problem with centralized websites too. How many links to broken blogs, broken youtube videos or to broken tweets have you ever encountered? Heck, most people don't even link to tweets anymore, they just post screenshots. The current internet is terrible at archiving information.

cryptome12345 5y ago

Why not starting paying them for their services on a decentralized network like BTC pays for computer power? I can see a decentralized design for paid hosting like that.