4 ms·

There are a lot of nuanced ways a trustless distributed system can be more efficient. The article is pretty light on details but consider the content addressin

There are a lot of nuanced ways a trustless distributed system can be more efficient. The article is pretty light on details but consider the content addressing mechanism. The data can be securely served by any device and independently verified by any recipient. The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. It’s like edge computing with any device that can connect to one another. Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client but the there are a number of somewhat nuanced theoretical efficiencies that can be achieved by making content addressing the core of a distributed system (caching, [as mentioned] fault-tolerance, parallel computing, etc).

acdha 6y ago
> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be true for that to be a net-positive I'm skeptical that it'd be easy to hit anywhere close to the best-case theoretical scenario.
- rudolph9 6y ago
  > Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice. Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there? The key to the efficiency of file coin is not some magic cryptographic token, it's not even "blockchain", the key is the underlying protocols being unified for synchronizing devices across multiple tiers of networking. Filecoin mostly just makes sure the data doesn't disappear and gives you an optional substitute for cloud storage. Content addressing, p2p communication by default, flexible/future-proof standards for negotiated what data is and how it is formatted, and in modular plug-and-play utilization of many different communication/synchronization techniques is the key to applications being built in more effective and efficient way than conventional client/server architecture. The word that comes to mind is "flexibility". Want you store to be S3 buckets? Great! Want to host a private network, only your devices can access? Great! Want to joint the public network? Great! Want to ensure persistence of data via a token? Great! Want to just use the address mechanism? Great! Want to run the software in the cloud, on you local machine, in a sandboxed browser tab and have them all seamlessly forming a p2p network over a multitude of network standards (tcp, udp, web-sockets, etc)? Great! I worked extensively last year with various ipfs techs utilized in a private p2p network consisting of ARM nodes capture time-series measurement devices and cloud based persistent nodes managing the pinset of data and persisting long-term storage. There are definitely some rough edges but again it's open-source built is such a modular way it's almost difficult to develop your self into a corner. The software coming out of protocol labs is overwhelmingly not new technology. Protocol lab large focuses on utilization of battle-tested existing standards and tech (many of which are relatively ancient like TCP, ssh, git, json, etc) unified, without breaking existing stands, via a common addressing mechanism. tl;dr Filecoin/crypto-tokens just a side note! The vast majority of software coming out of protocol labs is independently useful and largely focuses on bridging together widely-used battle-tested tech.
  - acdha 6y ago
    > Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there? Yes, what Dropbox, Crashplan, etc. offered a decade ago — it's a neat sounding idea but there are two reasons why this isn't a huge win in practice: people are mobile so the number of times where you need an uncached file and happen to be on the same network is relatively low and increasingly few people have an always-on device with a ton of local storage free (phones are really good at generating high volumes of data so this is a non-trivial problem). Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.
    - rudolph9 6y ago
      > Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings. The idea that ipfs/related-project are trying to beat conventional services is big misconception. The overwhelming majority of the tech does not preclude usage in conventional services. For example, IPFS can 100% be configured as server/client offering roughly the same costs/reliability many are used to. The advantage is interoperability with many different "services", conventional and alternative (e.g. filecoin) alike.
      - acdha 6y ago
        Can you expand that interoperability thought? It’s unclear to me how this could change the main obstruction of vendors choosing not to encourage interoperability - anyone who keeps their API locked down is unlikely to adopt standard IPFS.
zucker42 6y ago
Is this really an advantage of the fact that the system is trustless and the ownership of storage media is distributed? You can an imagine a system where Amazon owns all the hard drive but still distributes/duplicates files across many different hard drives, matching the theoretical performance characteristics of Filecoin. In fact storing data on S3 from my cursory understanding already distributes data across multiple drives based on their durability claims.
- rudolph9 6y ago
  The advantage is the flexibility. I kind of went on a rant about this [here][1] but to answer your question more directly; the advantage is how the flexibility the addressing mechanism provides. Blockchain-tokens are frankly kind of boring, they solve the double spend problem. What's interesting is the trustless distributed system tech is being developed around them! [1]: https://news.ycombinator.com/item?id=24218074 https://news.ycombinator.com/item?id=24218074

acdha 6y ago

> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be true for that to be a net-positive I'm skeptical that it'd be easy to hit anywhere close to the best-case theoretical scenario.

rudolph9 6y ago
> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice. Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there? The key to the efficiency of file coin is not some magic cryptographic token, it's not even "blockchain", the key is the underlying protocols being unified for synchronizing devices across multiple tiers of networking. Filecoin mostly just makes sure the data doesn't disappear and gives you an optional substitute for cloud storage. Content addressing, p2p communication by default, flexible/future-proof standards for negotiated what data is and how it is formatted, and in modular plug-and-play utilization of many different communication/synchronization techniques is the key to applications being built in more effective and efficient way than conventional client/server architecture. The word that comes to mind is "flexibility". Want you store to be S3 buckets? Great! Want to host a private network, only your devices can access? Great! Want to joint the public network? Great! Want to ensure persistence of data via a token? Great! Want to just use the address mechanism? Great! Want to run the software in the cloud, on you local machine, in a sandboxed browser tab and have them all seamlessly forming a p2p network over a multitude of network standards (tcp, udp, web-sockets, etc)? Great! I worked extensively last year with various ipfs techs utilized in a private p2p network consisting of ARM nodes capture time-series measurement devices and cloud based persistent nodes managing the pinset of data and persisting long-term storage. There are definitely some rough edges but again it's open-source built is such a modular way it's almost difficult to develop your self into a corner. The software coming out of protocol labs is overwhelmingly not new technology. Protocol lab large focuses on utilization of battle-tested existing standards and tech (many of which are relatively ancient like TCP, ssh, git, json, etc) unified, without breaking existing stands, via a common addressing mechanism. tl;dr Filecoin/crypto-tokens just a side note! The vast majority of software coming out of protocol labs is independently useful and largely focuses on bridging together widely-used battle-tested tech.
- acdha 6y ago
  > Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there? Yes, what Dropbox, Crashplan, etc. offered a decade ago — it's a neat sounding idea but there are two reasons why this isn't a huge win in practice: people are mobile so the number of times where you need an uncached file and happen to be on the same network is relatively low and increasingly few people have an always-on device with a ton of local storage free (phones are really good at generating high volumes of data so this is a non-trivial problem). Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.
  - rudolph9 6y ago
    > Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings. The idea that ipfs/related-project are trying to beat conventional services is big misconception. The overwhelming majority of the tech does not preclude usage in conventional services. For example, IPFS can 100% be configured as server/client offering roughly the same costs/reliability many are used to. The advantage is interoperability with many different "services", conventional and alternative (e.g. filecoin) alike.
    - acdha 6y ago
      Can you expand that interoperability thought? It’s unclear to me how this could change the main obstruction of vendors choosing not to encourage interoperability - anyone who keeps their API locked down is unlikely to adopt standard IPFS.

zucker42 6y ago

Is this really an advantage of the fact that the system is trustless and the ownership of storage media is distributed? You can an imagine a system where Amazon owns all the hard drive but still distributes/duplicates files across many different hard drives, matching the theoretical performance characteristics of Filecoin. In fact storing data on S3 from my cursory understanding already distributes data across multiple drives based on their durability claims.

rudolph9 6y ago
The advantage is the flexibility. I kind of went on a rant about this [here][1] but to answer your question more directly; the advantage is how the flexibility the addressing mechanism provides. Blockchain-tokens are frankly kind of boring, they solve the double spend problem. What's interesting is the trustless distributed system tech is being developed around them! [1]: https://news.ycombinator.com/item?id=24218074 https://news.ycombinator.com/item?id=24218074