3 ms·

This is an intriguing addition to Bittorrent; this kind of feature exists in IPFS [1], but of course Bittorrent has a much, much larger install base, so this pr

by niftich 10y ago

This is an intriguing addition to Bittorrent; this kind of feature exists in IPFS [1], but of course Bittorrent has a much, much larger install base, so this progressive enhancement could bring similar or near-equivalent features to a wider audience. This implementation relies on BEP 44 (Storing arbitrary data in the DHT) [2], which, together with the design of the DHT, does have some security implications [3]. [1] https://ipfs.io/ https://ipfs.io/ [2] http://www.bittorrent.org/beps/bep_0044.html http://www.bittorrent.org/beps/bep_0044.html [3] https://gist.github.com/substack/eadd13302d785dc13aac#file-results-markdown https://gist.github.com/substack/eadd13302d785dc13aac#file-r...

Mizza 10y ago
IPFS still doesn't have a content discovery mechanism, which I found very disappointing. It's not a killer app without that.
- niftich 10y ago
  Well, to be fair, neither does HTTP, short of either name-services (DNS), or crawling-and-indexing. IFPS has IPNS [1], the name services (the feature most-directly relevant to the Bittorrent enhancement in question). To date, I'm unaware of an IFPS crawler-and-indexer search engine. [1] https://github.com/ipfs/faq/issues/16 https://github.com/ipfs/faq/issues/16
  - Mizza 10y ago
    Why is that fair? They're designed for different purposes. IPFS falls far short of being useful in a transformative way because it lacks this important feature. Otherwise, it'll be a kind of gimmicky alternative to HTTP, "auto-archiving HTTP", that never got used for anything real.
    - niftich 10y ago
      I don't disagree that having content discovery would be nice, but IFPS makes a pretty good case [1] about what it brings to the table over HTTP. My point was that HTTP also has "zero out-of-the-box support" for content discovery, yet once search engines came out, we were fine. [1] https://ipfs.io/#why https://ipfs.io/#why
      - brownhats 10y ago
        If by fine you mean a centralized search engine is a good solution for a decentralized network, then yeah.
    - nine_k 10y ago
      Did you ever use the web in 1995? Content discovery is neither a part of HTTP nor the architecture of the web. It's a feature if it's current landscape, built on top of existing non-ideal ALI and brute force (scanning). Good web search (represented by Google) appeared around 1998, years after the general availability of the web, when the corpus of web pages was already large. Up to this moment, search is powered by ad revenue. I don't see how IPFS is significantly different in this regard.
      - Mizza 10y ago
        It isn't 1995, this is exactly my point. If we're trying to design new systems, we shouldn't design them with the exact same problems. What's the point of "decentralizing" if it just means another Google?
        nine_k 10y ago
        The approach of yacy.net may be partly applicable to IPFS. Donating resources to a "traditional" scanning search engine is also probably doable. But unlike Web, IPFS lacks intense linking and thus "citation ranking" (PageRank-like). Measuring relevance is harder.
- viraptor 10y ago
  I don't really believe the same network can carry its metadata. (attached directly to the content) I mean, in an ideal world it would be great, but in practice how would that work? You can copy anyone's metadata to the file you published. How can anyone tell a difference? (unless you put some kind of signed ranking system on top of that, but that's just another layer waiting to be abused - like kazaa's star ratings) It's basically the reason search engines stopped trusting meta tags on the websites.
- bobajeff 10y ago
  What sort of mechanism are you referring to? How are such things implemented in other technologies?
lkjhgfdsa57 10y ago
Seems to fit the same usecase as Freenet USK keys: https://wiki.freenetproject.org/Updateable_Subspace_Key https://wiki.freenetproject.org/Updateable_Subspace_Key They allow the holder of the private key to add new data and clients who know the public USK can get notified of updates. In Freenet you can also get the old versions of the data for as long as they stay around on the network.
- niftich 10y ago
  Freenet is remarkably well thought-out and solves a lot of these difficult issues like censorship resistance, deniable encryption, distributed data storage, and mutability; but it doesn't get much press these days.
aninhumer 10y ago
Indeed, I'm surprised it's taken this long for people to take advantage of bittorrent DHT for other p2p stuff. You don't even need to change the protocol to start using it. It's a ready made hash-to-peerlist mapping, so you could just use it for peer-discovery/bootstrapping and do other stuff out-of-band.
- rakoo 10y ago
  There already was a BEP to exchange arbitrary stuff, BEP44, for mutable or immutable items. It's taken this long to use this BEP specifically for mutable torrents, though, because it turned out that not many people had a need for mutable torrents.
  - aninhumer 10y ago
    Is this BEP44 actually implemented in most clients? I was experimenting with p2p some years ago, and I considered extending the protocol, but I figured it would be more practical to work out-of-band, rather than relying on widespread implementation of an extension (which didn't seem that likely to me at the time).

by niftich 10y ago

Mizza 10y ago

IPFS still doesn't have a content discovery mechanism, which I found very disappointing. It's not a killer app without that.

niftich 10y ago
Well, to be fair, neither does HTTP, short of either name-services (DNS), or crawling-and-indexing. IFPS has IPNS [1], the name services (the feature most-directly relevant to the Bittorrent enhancement in question). To date, I'm unaware of an IFPS crawler-and-indexer search engine. [1] https://github.com/ipfs/faq/issues/16 https://github.com/ipfs/faq/issues/16
- Mizza 10y ago
  Why is that fair? They're designed for different purposes. IPFS falls far short of being useful in a transformative way because it lacks this important feature. Otherwise, it'll be a kind of gimmicky alternative to HTTP, "auto-archiving HTTP", that never got used for anything real.
  - niftich 10y ago
    I don't disagree that having content discovery would be nice, but IFPS makes a pretty good case [1] about what it brings to the table over HTTP. My point was that HTTP also has "zero out-of-the-box support" for content discovery, yet once search engines came out, we were fine. [1] https://ipfs.io/#why https://ipfs.io/#why
    - brownhats 10y ago
      If by fine you mean a centralized search engine is a good solution for a decentralized network, then yeah.
  - nine_k 10y ago
    Did you ever use the web in 1995? Content discovery is neither a part of HTTP nor the architecture of the web. It's a feature if it's current landscape, built on top of existing non-ideal ALI and brute force (scanning). Good web search (represented by Google) appeared around 1998, years after the general availability of the web, when the corpus of web pages was already large. Up to this moment, search is powered by ad revenue. I don't see how IPFS is significantly different in this regard.
    - Mizza 10y ago
      It isn't 1995, this is exactly my point. If we're trying to design new systems, we shouldn't design them with the exact same problems. What's the point of "decentralizing" if it just means another Google?
      - nine_k 10y ago
        The approach of yacy.net may be partly applicable to IPFS. Donating resources to a "traditional" scanning search engine is also probably doable. But unlike Web, IPFS lacks intense linking and thus "citation ranking" (PageRank-like). Measuring relevance is harder.
viraptor 10y ago
I don't really believe the same network can carry its metadata. (attached directly to the content) I mean, in an ideal world it would be great, but in practice how would that work? You can copy anyone's metadata to the file you published. How can anyone tell a difference? (unless you put some kind of signed ranking system on top of that, but that's just another layer waiting to be abused - like kazaa's star ratings) It's basically the reason search engines stopped trusting meta tags on the websites.
bobajeff 10y ago
What sort of mechanism are you referring to? How are such things implemented in other technologies?

lkjhgfdsa57 10y ago

Seems to fit the same usecase as Freenet USK keys: https://wiki.freenetproject.org/Updateable_Subspace_Key https://wiki.freenetproject.org/Updateable_Subspace_Key They allow the holder of the private key to add new data and clients who know the public USK can get notified of updates. In Freenet you can also get the old versions of the data for as long as they stay around on the network.

niftich 10y ago
Freenet is remarkably well thought-out and solves a lot of these difficult issues like censorship resistance, deniable encryption, distributed data storage, and mutability; but it doesn't get much press these days.

aninhumer 10y ago

Indeed, I'm surprised it's taken this long for people to take advantage of bittorrent DHT for other p2p stuff. You don't even need to change the protocol to start using it. It's a ready made hash-to-peerlist mapping, so you could just use it for peer-discovery/bootstrapping and do other stuff out-of-band.

rakoo 10y ago
There already was a BEP to exchange arbitrary stuff, BEP44, for mutable or immutable items. It's taken this long to use this BEP specifically for mutable torrents, though, because it turned out that not many people had a need for mutable torrents.
- aninhumer 10y ago
  Is this BEP44 actually implemented in most clients? I was experimenting with p2p some years ago, and I considered extending the protocol, but I figured it would be more practical to work out-of-band, rather than relying on widespread implementation of an extension (which didn't seem that likely to me at the time).