6 ms·

So, the same question and point of criticism that I've brought up in past discussions about IPFS, and that so far has not yet been sufficiently answered by anyb

So, the same question and point of criticism that I've brought up in past discussions about IPFS, and that so far has not yet been sufficiently answered by anybody: The claim is that IPFS could replace HTTP, the web, and so on. The only thing I see, however, is a distributed filesystem, which is only one part of the puzzle. Real-world applications require backend systems with access control, mutable data, certain information being kept secret, and so on - something that seems fundamentally at odds with the design of IPFS. How would IPFS cover all of these cases? As it stands, it essentially looks to me like a Tahoe-LAFS variant that's more tailored towards large networks - but for it to "replace HTTP", it will have to not only cover every existing HTTP usecase, but also do so without introducing significant complexity for developers. Seriously, I'd like to see an answer to this, regardless of whether it's a practical solution or a remark along the lines of "this isn't possible". I'm getting fairly tired of the hype around IPFS, with seemingly none of its users understanding the limitations or how it fits (or doesn't fit) into existing workflows. I can't really take it seriously until I see technical arguments.

dsp1234 11y ago
https://github.com/ipfs/faq/issues/28 https://github.com/ipfs/faq/issues/28 may help out in understanding where they are going as far as dynamic content, authorization, etc. From just a quick look (I'm not IPFS expert): IPNS is used as the mutable state driving 'dynamic' content. The dynamic content is still published as a normal immutable reference, but is referenced by an IPNS name/path/hash. To me this appears to be like a CNAME in dns. IPNS provides the pointer to the immutible hash, and for dynamic content you would only ever use the IPNS. For authorization, this could be done via PKI. Take some data, encrypt it with one or more user's public keys, then post it to IPFS. Then the recipient, and only the recipient, would be able to access the content. I can't really take it seriously until I see technical arguments. Well, to be fair, the technical arguments appear to exist (after all there is a IPFS draft document which covers the stuff I mention above), and while something like replacing HTTP is obviously extreme, those arguments appear (on their face) to not be completely crazy. So maybe if there are specific technical issues that aren't covered by the draft documents and other documentation on the IPFS site, github faq page, etc, then maybe bring them up and people can help you answer those questions.
ossreality 11y ago
>Real-world applications require backend systems with access control, mutable data, certain information being kept secret, and so on - something that seems fundamentally at odds with the design of IPFS. And not a single one of those things you mentioned has anything to do with HTTP. Those are all application level semantics. So why would IPFS be any different. I fail to see why IPFS needs to speak to these things.
natrius 11y ago
Let's separate the use cases: we want a permanent, public web, and we want applications that don't die when servers get shut down. IPFS clearly solves the first use case, so let's talk about the second. In this context, IPFS is part of the toolkit for building decentralized applications. The architecture of these applications is fundamentally different from today's apps. Users control the data, not the application developer. Private data can live on the user's machine or some cloud service they trust to hold the data for them. IPFS probably isn't a good fit for private data, but public apps like a decentralized Twitter will probably use it for all of their data. Encrypting private data and storing it with IPFS will sometimes make sense. To deal with mutable data, it depends what your goals are. If users don't need to agree with each other about the value of the data, they can store it however they want and maintain their own view of the world. If they need to agree on data that only one person can change, then you can use IPNS to manage mutable references controlled by one keypair. If users need to agree on data about their interactions with each other, then they need to use a decentralized cloud computer that can enforce rules on a historical record. We call these enforcement clouds "blockchains," and Ethereum is a blockchain that defines a process for defining new rules so application developers don't have to build their own blockchains. Building decentralized apps with IPFS and Ethereum is the most exciting work I've ever done. It doesn't just improve the apps themselves, it fundamentally changes the economics of our industry. Most businesses are built on network effects, but decentralized apps don't bottle up the value of their networks: they give them to their users. That's more like the world I want to live in. If you want to be a part of this change, you should join us at ConsenSys, or join the decentralization movement in general.
- droffel 11y ago
  I've recently been considering building an application that runs on a backend of Ethereum+Storj. The ability to store (and retrieve) data in Storj chain feels like a necessary component in getting a full DApp set up, right now I'm grappling with a concept that would (somehow) require the ability to provide users with a single-use key to download data from Storj. The future of cryptocurrencies is bright, there's now a large set of tools in the toolbox to use to build resilient, permanent applications for people to use. I'm really excited to see what people build in 2016, the cryptocurrency ecosystem has been growing rapidly in the shadows, and they're finally breaking into the mainstream.
smokeyj 11y ago
It seems like ultimately there will be an external protocol for distributed databases. For example bigchaindb.
- jokoon 11y ago
  This is much more interesting. Accessing data instead of accessing files or addresses makes more sense if you're making an application. With such a database system, it would be trivial to build a decentralized forum, for example.
  - noffle 11y ago
    The "FS" part of IPFS can be misleading: you can store arbitrary data in IPFS. The tooling merely provides a means of viewing Merkle DAGs (the base data structure) as a file hierarchy because it's a pretty clean 1:1 mapping that's familiar to many people.
dr_win 11y ago
Counter example: Git is immutable object store successfully replacing mutable CVS databases. The trick is to build mutable functionality on top immutable store. In case of git it is branches/tags that are light-weight mutable pointers to immutable objects. Additionally I believe access control and privacy can be built on top of immutable store using cryptography.
- ianopolous 11y ago
  You've nailed it. At Peergos [1], we are doing exactly that, with privacy as our primary focus. We layer encryption on top of IPNS + IPFS to get a distributed private, access controlled filesystem + social network that hides your friendship graph. IPFS is a wonderful model to work with. [1] https://github.com/ianopolous/Peergos https://github.com/ianopolous/Peergos
- joepie91_ 11y ago
  "Using cryptography" is (like some of the other things mentioned) nice in theory, but has significant drawbacks as well. If you've ever worked with stateless servers using something like JWT, you will have run into the issue of 'stale data', essentially the cache invalidation problem. Trying to solve ACLs with cryptography is prone to the exact same issues, as well as overhead problems. The crypto itself is immutable. EDIT: In fact, Git is a great example of how you can't really do access control very easily in such a setup. Have you ever tried restricting access to a specific branch in a repository?
bachback 11y ago
the name in itself shows this is not a viable concept at all: a global filesystem? the web was all about getting rid of file abstractions and introducing hypertext. the world (and the net) isn't about just data. its about code, security mechanisms, etc. etc. filesystems are not part of the puzzle, they are just old abstractions the computing world hasn't get rid of yet.
- effie 11y ago
  IPFS is not replacement of web by a filesystem, it is replacement of the current platform for the web - centralized hosting - by a new platform - decentralized global filesystem. The usual web technologies - hyperlinks, dynamic web pages run by scripts - are on upper layer and can remain in use.
- ossreality 11y ago
  This is so wildly wrong it's hard to understand how this thought even made sense in your head. I'm not even talking in the context of IPFS. Your entire notion of the web and what hyperlinks mean is just... wrong. It doesn't even map to the web as it exists today.
kelvin0 11y ago
Breaking news: IPFS partners with Wolfram, and ditches JS for Wolfram Language. Your new browser is Mathematica.
- _prometheus 11y ago
  haha, wouldn't this be so interesting? wish wolfram was open source
weavejester 11y ago
Real-world applications require backend systems with access control, mutable data, certain information being kept secret, and so on - something that seems fundamentally at odds with the design of IPFS. I don't think that's at odds with the design. If you want to keep something secret, you encrypt it. If you want to manage access control, you sign it. A client application can discard information that cannot be decrypted, or isn't signed by an authorised key. You don't get atomicity guarantees of course, but you can often get away without them. I don't see a system like IPFS as a replacement for all of the web, just a sizeable percentage of it.
- joepie91_ 11y ago
  Access control is not authorization. And re-encrypting with different keys every time the ACL changes is generally not practical.
kordless 11y ago
> How would IPFS cover all of these cases? Seriously, I'd like to see an answer to this... This is illogical, as evidenced by the illogical replies you are getting below. You are asking in a "serious" way for the IPFS community, developers or Juan to explain how all current use cases would work on IPFS. That's a pretty tall order for an Open Source project that (seriously) owes you nothing. I would suggest you attempt to run an IPFS node yourself and write a few applications and tell us how it goes.
- joepie91_ 11y ago
  It's a question I'm asking because that is the claim that is being made. You should address this reply to those making the claim, not to the person questioning it. EDIT: Also, please get out with the "owes you nothing" argument, I'm really growing sick of it. If I see something generating undeserved hype with potentially bad consequences for 'society at large', then I will damn well call it out. Don't like it, don't reply to it. It has precisely nothing to do with anybody owing me anything.
_prometheus 11y ago
Thanks for the feedback! We have answers to all these questions over at the many repos that we use to work on IPFS. Look around in: - https://github.com/ipfs/faq/issues https://github.com/ipfs/faq/issues - https://github.com/ipfs/notes/issues https://github.com/ipfs/notes/issues - https://github.com/ipfs/apps/issues https://github.com/ipfs/apps/issues The technical answers exist. Please look for them! I'd love to write personalized answers for every person that asks, but i better spend my time developing. Also you should look at the rest of the discussions too, not just my compressed answers. be part of the discussion, raise your concerns, and have them answered. For sake of saving you time, some short / compressed answers: - mutable data: look into ipns (in the ipfs paper or repos linked above) and look into CRDTs. CRDTs can layer cleanly on top of ipfs for distributed mutable state. if you haven't seen CRDTs: http://hal.upmc.fr/inria-00555588/document http://hal.upmc.fr/inria-00555588/document - access controls / certain info remaining secret: data encryption and capabilities. you mention Tahoe-LAFS, yep!, we can (and will) build a cap system similar to it on top of raw ipfs objects (ideally actually collaborating with the excellent Tahoe-LAFS team directly). --- oh and many users want easy selective disclosure for encrypted multiparty records. we've not built this in yet because we have other priorities right now. if you'd like to contribute to this we'd love the help! - "it will have to not only cover every existing HTTP usecase" -- we don't aim to cover _every_ HTTP facility, instead we can power a nicer model for distributed data, where you replicate datastructures directly. This looks a lot closer to the data models people use in apps today (think single page app models, {backbone, react, meteor, ...}. these are layered on top of APIs like REST, but can trivially layer over IPFS too) - pub/sub: you didnt mention it, but it is necessary for fast (subsecond) updates to mutable data in the large (>millions of nodes). we're working on these designs now, and impls coming in a few weeks/months. (some simple non-scalable versions already proposed and implemented to get started, but the long term solutions coming later). I ask you please take a deeper look at the comm channels of our community, and that you voice your concerns + questions in our FAQs and other relevant repos. That way we can improve our models and systems based on your input. And if you have time to help us implement some ideas, join us! :) more links at this answer https://news.ycombinator.com/item?id=11137313 https://news.ycombinator.com/item?id=11137313
- collinmanderson 11y ago
jude- 11y ago
I don't think IPFS is trying to replace HTTP; it's trying to replace URLs. IPFS's goal is to offer universal content-centric resource location as an alternative to the Web's host-centric resource location. This is a nice thing because it decouples content from hosting--you can get the content from any reachable peer who has it. A good chunk of Team IPFS's engineering efforts have been put towards offering a wide variety of protocols to efficiently resolve a content hash to the content in a wide variety of contexts, covering things like peer discovery, content hash announcements, hash naming (IPNS), super-peers (ipfs-cluster), pub/sub, and so on. IPFS is definitely not a filesystem in the classical (POSIX-y) sense. I think the naming is unfortunate, because it implies that IPFS has things a global root directory, permission bits, users, ACLs, immutable human-meaningful names that resolve to mutable content, a POSIX-y consistency model, etc. IPFS does none of these things--it defers them to layers above it. All IPFS is concerned with is providing a Merkle DAG abstraction, and the necessary network protocols to walk it and fetch the underlying content.
- joepie91_ 11y ago
  Yes, and that's the problem - because it cannot address many of the features required for dynamic content. I'm sure it works fine for static content, but that's the easy problem to solve (and it has been solved before).
  - jude- 11y ago
    You're right, IPFS does not try to address dynamic content. I don't speak for the authors, but I think their opinion is that doing so is out of scope. I think they expect the application to represent each piece of mutable content as a Merkle DAG, and have the application acquire each user's written version of the content from IPFS, merge them, and announce the "latest" version under a new hash.

dsp1234 11y ago

https://github.com/ipfs/faq/issues/28 https://github.com/ipfs/faq/issues/28 may help out in understanding where they are going as far as dynamic content, authorization, etc. From just a quick look (I'm not IPFS expert): IPNS is used as the mutable state driving 'dynamic' content. The dynamic content is still published as a normal immutable reference, but is referenced by an IPNS name/path/hash. To me this appears to be like a CNAME in dns. IPNS provides the pointer to the immutible hash, and for dynamic content you would only ever use the IPNS. For authorization, this could be done via PKI. Take some data, encrypt it with one or more user's public keys, then post it to IPFS. Then the recipient, and only the recipient, would be able to access the content. I can't really take it seriously until I see technical arguments. Well, to be fair, the technical arguments appear to exist (after all there is a IPFS draft document which covers the stuff I mention above), and while something like replacing HTTP is obviously extreme, those arguments appear (on their face) to not be completely crazy. So maybe if there are specific technical issues that aren't covered by the draft documents and other documentation on the IPFS site, github faq page, etc, then maybe bring them up and people can help you answer those questions.

ossreality 11y ago

>Real-world applications require backend systems with access control, mutable data, certain information being kept secret, and so on - something that seems fundamentally at odds with the design of IPFS. And not a single one of those things you mentioned has anything to do with HTTP. Those are all application level semantics. So why would IPFS be any different. I fail to see why IPFS needs to speak to these things.

natrius 11y ago

Let's separate the use cases: we want a permanent, public web, and we want applications that don't die when servers get shut down. IPFS clearly solves the first use case, so let's talk about the second. In this context, IPFS is part of the toolkit for building decentralized applications. The architecture of these applications is fundamentally different from today's apps. Users control the data, not the application developer. Private data can live on the user's machine or some cloud service they trust to hold the data for them. IPFS probably isn't a good fit for private data, but public apps like a decentralized Twitter will probably use it for all of their data. Encrypting private data and storing it with IPFS will sometimes make sense. To deal with mutable data, it depends what your goals are. If users don't need to agree with each other about the value of the data, they can store it however they want and maintain their own view of the world. If they need to agree on data that only one person can change, then you can use IPNS to manage mutable references controlled by one keypair. If users need to agree on data about their interactions with each other, then they need to use a decentralized cloud computer that can enforce rules on a historical record. We call these enforcement clouds "blockchains," and Ethereum is a blockchain that defines a process for defining new rules so application developers don't have to build their own blockchains. Building decentralized apps with IPFS and Ethereum is the most exciting work I've ever done. It doesn't just improve the apps themselves, it fundamentally changes the economics of our industry. Most businesses are built on network effects, but decentralized apps don't bottle up the value of their networks: they give them to their users. That's more like the world I want to live in. If you want to be a part of this change, you should join us at ConsenSys, or join the decentralization movement in general.

droffel 11y ago
I've recently been considering building an application that runs on a backend of Ethereum+Storj. The ability to store (and retrieve) data in Storj chain feels like a necessary component in getting a full DApp set up, right now I'm grappling with a concept that would (somehow) require the ability to provide users with a single-use key to download data from Storj. The future of cryptocurrencies is bright, there's now a large set of tools in the toolbox to use to build resilient, permanent applications for people to use. I'm really excited to see what people build in 2016, the cryptocurrency ecosystem has been growing rapidly in the shadows, and they're finally breaking into the mainstream.

smokeyj 11y ago

It seems like ultimately there will be an external protocol for distributed databases. For example bigchaindb.

jokoon 11y ago
This is much more interesting. Accessing data instead of accessing files or addresses makes more sense if you're making an application. With such a database system, it would be trivial to build a decentralized forum, for example.
- noffle 11y ago
  The "FS" part of IPFS can be misleading: you can store arbitrary data in IPFS. The tooling merely provides a means of viewing Merkle DAGs (the base data structure) as a file hierarchy because it's a pretty clean 1:1 mapping that's familiar to many people.

dr_win 11y ago

Counter example: Git is immutable object store successfully replacing mutable CVS databases. The trick is to build mutable functionality on top immutable store. In case of git it is branches/tags that are light-weight mutable pointers to immutable objects. Additionally I believe access control and privacy can be built on top of immutable store using cryptography.

ianopolous 11y ago
You've nailed it. At Peergos [1], we are doing exactly that, with privacy as our primary focus. We layer encryption on top of IPNS + IPFS to get a distributed private, access controlled filesystem + social network that hides your friendship graph. IPFS is a wonderful model to work with. [1] https://github.com/ianopolous/Peergos https://github.com/ianopolous/Peergos
joepie91_ 11y ago
"Using cryptography" is (like some of the other things mentioned) nice in theory, but has significant drawbacks as well. If you've ever worked with stateless servers using something like JWT, you will have run into the issue of 'stale data', essentially the cache invalidation problem. Trying to solve ACLs with cryptography is prone to the exact same issues, as well as overhead problems. The crypto itself is immutable. EDIT: In fact, Git is a great example of how you can't really do access control very easily in such a setup. Have you ever tried restricting access to a specific branch in a repository?

bachback 11y ago

the name in itself shows this is not a viable concept at all: a global filesystem? the web was all about getting rid of file abstractions and introducing hypertext. the world (and the net) isn't about just data. its about code, security mechanisms, etc. etc. filesystems are not part of the puzzle, they are just old abstractions the computing world hasn't get rid of yet.

effie 11y ago
IPFS is not replacement of web by a filesystem, it is replacement of the current platform for the web - centralized hosting - by a new platform - decentralized global filesystem. The usual web technologies - hyperlinks, dynamic web pages run by scripts - are on upper layer and can remain in use.
ossreality 11y ago
This is so wildly wrong it's hard to understand how this thought even made sense in your head. I'm not even talking in the context of IPFS. Your entire notion of the web and what hyperlinks mean is just... wrong. It doesn't even map to the web as it exists today.

kelvin0 11y ago

Breaking news: IPFS partners with Wolfram, and ditches JS for Wolfram Language. Your new browser is Mathematica.

_prometheus 11y ago
haha, wouldn't this be so interesting? wish wolfram was open source

weavejester 11y ago

Real-world applications require backend systems with access control, mutable data, certain information being kept secret, and so on - something that seems fundamentally at odds with the design of IPFS. I don't think that's at odds with the design. If you want to keep something secret, you encrypt it. If you want to manage access control, you sign it. A client application can discard information that cannot be decrypted, or isn't signed by an authorised key. You don't get atomicity guarantees of course, but you can often get away without them. I don't see a system like IPFS as a replacement for all of the web, just a sizeable percentage of it.

joepie91_ 11y ago
Access control is not authorization. And re-encrypting with different keys every time the ACL changes is generally not practical.

kordless 11y ago

> How would IPFS cover all of these cases? Seriously, I'd like to see an answer to this... This is illogical, as evidenced by the illogical replies you are getting below. You are asking in a "serious" way for the IPFS community, developers or Juan to explain how all current use cases would work on IPFS. That's a pretty tall order for an Open Source project that (seriously) owes you nothing. I would suggest you attempt to run an IPFS node yourself and write a few applications and tell us how it goes.

joepie91_ 11y ago
It's a question I'm asking because that is the claim that is being made. You should address this reply to those making the claim, not to the person questioning it. EDIT: Also, please get out with the "owes you nothing" argument, I'm really growing sick of it. If I see something generating undeserved hype with potentially bad consequences for 'society at large', then I will damn well call it out. Don't like it, don't reply to it. It has precisely nothing to do with anybody owing me anything.

_prometheus 11y ago

Thanks for the feedback! We have answers to all these questions over at the many repos that we use to work on IPFS. Look around in: - https://github.com/ipfs/faq/issues https://github.com/ipfs/faq/issues - https://github.com/ipfs/notes/issues https://github.com/ipfs/notes/issues - https://github.com/ipfs/apps/issues https://github.com/ipfs/apps/issues The technical answers exist. Please look for them! I'd love to write personalized answers for every person that asks, but i better spend my time developing. Also you should look at the rest of the discussions too, not just my compressed answers. be part of the discussion, raise your concerns, and have them answered. For sake of saving you time, some short / compressed answers: - mutable data: look into ipns (in the ipfs paper or repos linked above) and look into CRDTs. CRDTs can layer cleanly on top of ipfs for distributed mutable state. if you haven't seen CRDTs: http://hal.upmc.fr/inria-00555588/document http://hal.upmc.fr/inria-00555588/document - access controls / certain info remaining secret: data encryption and capabilities. you mention Tahoe-LAFS, yep!, we can (and will) build a cap system similar to it on top of raw ipfs objects (ideally actually collaborating with the excellent Tahoe-LAFS team directly). --- oh and many users want easy selective disclosure for encrypted multiparty records. we've not built this in yet because we have other priorities right now. if you'd like to contribute to this we'd love the help! - "it will have to not only cover every existing HTTP usecase" -- we don't aim to cover _every_ HTTP facility, instead we can power a nicer model for distributed data, where you replicate datastructures directly. This looks a lot closer to the data models people use in apps today (think single page app models, {backbone, react, meteor, ...}. these are layered on top of APIs like REST, but can trivially layer over IPFS too) - pub/sub: you didnt mention it, but it is necessary for fast (subsecond) updates to mutable data in the large (>millions of nodes). we're working on these designs now, and impls coming in a few weeks/months. (some simple non-scalable versions already proposed and implemented to get started, but the long term solutions coming later). I ask you please take a deeper look at the comm channels of our community, and that you voice your concerns + questions in our FAQs and other relevant repos. That way we can improve our models and systems based on your input. And if you have time to help us implement some ideas, join us! :) more links at this answer https://news.ycombinator.com/item?id=11137313 https://news.ycombinator.com/item?id=11137313

collinmanderson 11y ago

jude- 11y ago

I don't think IPFS is trying to replace HTTP; it's trying to replace URLs. IPFS's goal is to offer universal content-centric resource location as an alternative to the Web's host-centric resource location. This is a nice thing because it decouples content from hosting--you can get the content from any reachable peer who has it. A good chunk of Team IPFS's engineering efforts have been put towards offering a wide variety of protocols to efficiently resolve a content hash to the content in a wide variety of contexts, covering things like peer discovery, content hash announcements, hash naming (IPNS), super-peers (ipfs-cluster), pub/sub, and so on. IPFS is definitely not a filesystem in the classical (POSIX-y) sense. I think the naming is unfortunate, because it implies that IPFS has things a global root directory, permission bits, users, ACLs, immutable human-meaningful names that resolve to mutable content, a POSIX-y consistency model, etc. IPFS does none of these things--it defers them to layers above it. All IPFS is concerned with is providing a Merkle DAG abstraction, and the necessary network protocols to walk it and fetch the underlying content.

joepie91_ 11y ago
Yes, and that's the problem - because it cannot address many of the features required for dynamic content. I'm sure it works fine for static content, but that's the easy problem to solve (and it has been solved before).
- jude- 11y ago
  You're right, IPFS does not try to address dynamic content. I don't speak for the authors, but I think their opinion is that doing so is out of scope. I think they expect the application to represent each piece of mutable content as a Merkle DAG, and have the application acquire each user's written version of the content from IPFS, merge them, and announce the "latest" version under a new hash.