9 ms·

Has it? I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy,

Has it? I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it. Worse yet, IPFS still provides no real way to deal with private or local data, which drastically limits its use. I love the idea about data being addressable by hash, but I don't feel IPFS has actually delivered anything meaningful in that area yet. And with ipfs-search.com shutdown, there is not even any way left to explore what is actually on the network now. And technical issues aside, there is also the legal problem that IPFS conflicts with copyright. Redistributing anything is illegal by default, unless somebody gives you permission, IPFS provides no means to track that permission. You can't attach a GPL or an author to a hash.

sandbx 3y ago
libgen uses it
- timtom39 3y ago
  In practice libgen uses torrents. The available seed nodes for IPFS are... few. This is largely due to the software/protocol being pretty bad compared to battle tested torrents.
  - arsome 3y ago
    Perhaps if you're downloading full dumps, but distribution of individual books is often done via IPFS.
diggan 3y ago
> there is also the legal problem that IPFS conflicts with copyright. Redistributing anything is illegal by default I think you might misunderstand how IPFS works, or possibly confuse it with Freenet. When you use IPFS, nothing is automatically distributed, unless they happen to know the hash of the content you've added locally. It's not until someone starts to explicitly request data from you, that you start sharing anything.
- grumbel 3y ago
  Everything in your local cache is redistributed, meaning every single website you view through IPFS ends up getting potentially redistributed. Can you guarantee that each and every item you view on the Web allows redistribution?
  - jerbear4328 3y ago
    I don't know how it works legally, but if you publish your own content to a distributed protocol like IPFS you should expect it to be distributed. If content that shouldn't be redistributed is put on the public IPFS, it should be the person who added it's fault, imo.
    - grumbel 3y ago
      The person who added it, is not tracked. A file can be added by anybody at anytime and anywhere and it will get the same content address. It's up to the person doing the redistribution to ensure it's legal, but IPFS gives them no tools to do so.
  - amelius 3y ago
    Perhaps it should use the fact that any file A can be written as the XOR of two files B and C, where B and C look like complete noise. If you only host B and another person only hosts C, then there can be no copyright infringements, while you can still reconstruct A by simply XORing the other files together. Of course at the expense of having to store and transfer twice the amount of data.
    - fulladder 3y ago
      This is engineer logic. This kind of thing would never fly in court. If you want an example, go back and look at what happened to Aereo back in 2014. They came up with a clever hack to work around broadcast TV copyright. It went to the U.S. Supreme Court and got shot down.
      - 3y ago
jrwr 3y ago
IPFS also has issues with large datasets, anything larger then a few TBs cases nodes to explode and indexing takes forever.
- realPubkey 3y ago
  Also the lookup time for files is too slow to use it on anything user facing.
rglullis 3y ago
I was working on the same building as the team from Protocol Labs. I talked with a couple of devs and it seems like they never even considered that most people who would be interested in running a node would like to have control over who gets access to the pinned files. I think I opened a ticket asking for an ACL system, but it got closed. I really had high hopes for it, but I realized that all I really want is an object storage with content addressable urls.
- deleted 3y ago
  [deleted]
- pclmulqdq 3y ago
  ACLs are fundamentally incompatible with blockchains. Your only solution is to encrypt those files, and have off-chain key management if you ever want to revoke access.
  - rglullis 3y ago
    IPFS and distributed filesystems have nothing to do with blockchains.
    - mhluongo 3y ago
      Filecoin, the thing funding IPFS, certainly does :)
      - dartos 3y ago
        Sure, but IPFS, the technology, doesn’t.
    - pclmulqdq 3y ago
      IPFS is entwined with (and pretty much 100% backed by) Filecoin, which is a blockchain. Introducing a point of incompatibility would be a nonstarter.
      - 3y ago
tutfbhuf 3y ago
> Has it? I have never seen anybody using IPFS in the wild Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time, often via an IPFS gateway such as Cloudflare's IPFS gateway, so you don't even notice it's using IPFS. These shadow libraries have millions of users per day, especially academics.
- shuckles 3y ago
  I’ve recently observed that both IPFS and the Cloudflare gateways are extremely unreliable on Libgen.
  - poopsmithe 3y ago
    That's besides the point. The point being made was that IPFS is being used in production. You bring up a valid point though. From my experience, IPFS suffers from a tragedy of the commons issue. It's performance is generally terrible because the free providers are overloaded.
    - shuckles 3y ago
      Not really: > Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time I would like to see evidence this is actually true. My theory is that IPFS's "success" in hosting Libgen is roughly turning ICO proceeds into free hosting. Paying for dedicated servers might have been more cost efficient and, given the performance issues, more robust!
  - beaugunderson 3y ago
    100%, and same for Standard Template Construct. There are basically no reliable IPFS gateways with book or paper content. They all get struck from the more reliable gateways immediately.
    - shuckles 3y ago
      Unfortunate behavior in an apocalypse-proof distributed filesystem.
geokon 3y ago
Wouldn't all those examples also work just as easily with torrents/magnet-links? I think in all those cases a central server distribution model has unfortunately been "good enough" for the majority of users (even though their data is mined and ads are injected) When it comes to FOSS, I personally don't understand why something like a package manager isn't P2P by default. It feels very aligned with the hacker culture - a la "A Declaration of the Independence of Cyberspace". Virtually nobody uses it, so the solutions are half baked, clunky and not integrated with every day workflows (ex: browsers don't support anything P2P). Something like libcurl can't pull a torrent from the web
- anthk 3y ago
  P2P works better for ISO based distros with full offline backups or maybe LTS based distros with small repos as Hyperbola. Having a full repo of Trisquel with continuous updates would not work well.
  - geokon 3y ago
    Oh right, that's a very good point. I've only use LTS for years. Forgot about rolling releases haha
- sseagull 3y ago
  > Wouldn't all those examples also work just as easily with torrents/magnet-links? My (possibly wrong) feeling is that IPFS is meant for smaller files, and torrents for bigger collections. So with the example of a package manager, you could download each individual package via IPFS, whereas a torrent would make sense if you wanted to download an archive of them all. Not to say torrents couldn't work, but it just kinda feels like not the intended use case. (The above feeling come from reading about IPFS a few times throughout the years and toying with it for a while).
- rakoo 3y ago
  There has been attempts at doing torrent-based package distribution, see the notes of debian testing it here: https://wiki.debian.org/DebTorrent https://wiki.debian.org/DebTorrent. There are basically 2 problems: Packages are a lot of small files, and not everyone is interested in them all. Torrents are good for large content that everyone wants. The two are in opposition and there needs to be a balance: do you need to make One Unique Torrent with everything, but then peers have to individually ask each and every other peer if they have the content they're looking for, making it unscalable ? Or do you make one torrent per package, meaning peers have to track thousands of torrents ? (That's the direction IPFS took, which is why it's extremely inefficient and resource-heavy). DebTorrent has made the choice to go in the middle. The real problem, to me, is that content-addressed content is the anti-privacy protocol: since you ask everyone "do you have this content", you're basically telling everyone what you want. In the case of installing packages, this means telling everyone what the version of your packages is, and is potentially a security issue. But the real "problem" is that it fails at being much more better than the existing solution. Bandwidth is cheap, storage is cheap, so hosting everything on a few independent but centralized servers is good enough.
rvz 3y ago
> I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it. Netflix uses IPFS. [0] [0] https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distribution https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-conta...
orblivion 3y ago
> Linux package distribution FWIW it seems like NixOS at least tried or is trying: https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/ https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/
- LanternLight83 3y ago
  Guix has had some developments in this direction too. https://libreplanet.org/wiki/Group:Guix/GSoC-2023 https://libreplanet.org/wiki/Group:Guix/GSoC-2023 https://issues.guix.gnu.org/52555 https://issues.guix.gnu.org/52555

sandbx 3y ago

libgen uses it

timtom39 3y ago
In practice libgen uses torrents. The available seed nodes for IPFS are... few. This is largely due to the software/protocol being pretty bad compared to battle tested torrents.
- arsome 3y ago
  Perhaps if you're downloading full dumps, but distribution of individual books is often done via IPFS.

diggan 3y ago

> there is also the legal problem that IPFS conflicts with copyright. Redistributing anything is illegal by default I think you might misunderstand how IPFS works, or possibly confuse it with Freenet. When you use IPFS, nothing is automatically distributed, unless they happen to know the hash of the content you've added locally. It's not until someone starts to explicitly request data from you, that you start sharing anything.

grumbel 3y ago
Everything in your local cache is redistributed, meaning every single website you view through IPFS ends up getting potentially redistributed. Can you guarantee that each and every item you view on the Web allows redistribution?
- jerbear4328 3y ago
  I don't know how it works legally, but if you publish your own content to a distributed protocol like IPFS you should expect it to be distributed. If content that shouldn't be redistributed is put on the public IPFS, it should be the person who added it's fault, imo.
  - grumbel 3y ago
    The person who added it, is not tracked. A file can be added by anybody at anytime and anywhere and it will get the same content address. It's up to the person doing the redistribution to ensure it's legal, but IPFS gives them no tools to do so.
- amelius 3y ago
  Perhaps it should use the fact that any file A can be written as the XOR of two files B and C, where B and C look like complete noise. If you only host B and another person only hosts C, then there can be no copyright infringements, while you can still reconstruct A by simply XORing the other files together. Of course at the expense of having to store and transfer twice the amount of data.
  - fulladder 3y ago
    This is engineer logic. This kind of thing would never fly in court. If you want an example, go back and look at what happened to Aereo back in 2014. They came up with a clever hack to work around broadcast TV copyright. It went to the U.S. Supreme Court and got shot down.
    - 3y ago

jrwr 3y ago

IPFS also has issues with large datasets, anything larger then a few TBs cases nodes to explode and indexing takes forever.

realPubkey 3y ago
Also the lookup time for files is too slow to use it on anything user facing.

rglullis 3y ago

I was working on the same building as the team from Protocol Labs. I talked with a couple of devs and it seems like they never even considered that most people who would be interested in running a node would like to have control over who gets access to the pinned files. I think I opened a ticket asking for an ACL system, but it got closed. I really had high hopes for it, but I realized that all I really want is an object storage with content addressable urls.

deleted 3y ago
[deleted]
pclmulqdq 3y ago
ACLs are fundamentally incompatible with blockchains. Your only solution is to encrypt those files, and have off-chain key management if you ever want to revoke access.
- rglullis 3y ago
  IPFS and distributed filesystems have nothing to do with blockchains.
  - mhluongo 3y ago
    Filecoin, the thing funding IPFS, certainly does :)
    - dartos 3y ago
      Sure, but IPFS, the technology, doesn’t.
  - pclmulqdq 3y ago
    IPFS is entwined with (and pretty much 100% backed by) Filecoin, which is a blockchain. Introducing a point of incompatibility would be a nonstarter.
    - 3y ago

tutfbhuf 3y ago

> Has it? I have never seen anybody using IPFS in the wild Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time, often via an IPFS gateway such as Cloudflare's IPFS gateway, so you don't even notice it's using IPFS. These shadow libraries have millions of users per day, especially academics.

shuckles 3y ago
I’ve recently observed that both IPFS and the Cloudflare gateways are extremely unreliable on Libgen.
- poopsmithe 3y ago
  That's besides the point. The point being made was that IPFS is being used in production. You bring up a valid point though. From my experience, IPFS suffers from a tragedy of the commons issue. It's performance is generally terrible because the free providers are overloaded.
  - shuckles 3y ago
    Not really: > Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time I would like to see evidence this is actually true. My theory is that IPFS's "success" in hosting Libgen is roughly turning ICO proceeds into free hosting. Paying for dedicated servers might have been more cost efficient and, given the performance issues, more robust!
- beaugunderson 3y ago
  100%, and same for Standard Template Construct. There are basically no reliable IPFS gateways with book or paper content. They all get struck from the more reliable gateways immediately.
  - shuckles 3y ago
    Unfortunate behavior in an apocalypse-proof distributed filesystem.

geokon 3y ago

Wouldn't all those examples also work just as easily with torrents/magnet-links? I think in all those cases a central server distribution model has unfortunately been "good enough" for the majority of users (even though their data is mined and ads are injected) When it comes to FOSS, I personally don't understand why something like a package manager isn't P2P by default. It feels very aligned with the hacker culture - a la "A Declaration of the Independence of Cyberspace". Virtually nobody uses it, so the solutions are half baked, clunky and not integrated with every day workflows (ex: browsers don't support anything P2P). Something like libcurl can't pull a torrent from the web

anthk 3y ago
P2P works better for ISO based distros with full offline backups or maybe LTS based distros with small repos as Hyperbola. Having a full repo of Trisquel with continuous updates would not work well.
- geokon 3y ago
  Oh right, that's a very good point. I've only use LTS for years. Forgot about rolling releases haha
sseagull 3y ago
> Wouldn't all those examples also work just as easily with torrents/magnet-links? My (possibly wrong) feeling is that IPFS is meant for smaller files, and torrents for bigger collections. So with the example of a package manager, you could download each individual package via IPFS, whereas a torrent would make sense if you wanted to download an archive of them all. Not to say torrents couldn't work, but it just kinda feels like not the intended use case. (The above feeling come from reading about IPFS a few times throughout the years and toying with it for a while).
rakoo 3y ago
There has been attempts at doing torrent-based package distribution, see the notes of debian testing it here: https://wiki.debian.org/DebTorrent https://wiki.debian.org/DebTorrent. There are basically 2 problems: Packages are a lot of small files, and not everyone is interested in them all. Torrents are good for large content that everyone wants. The two are in opposition and there needs to be a balance: do you need to make One Unique Torrent with everything, but then peers have to individually ask each and every other peer if they have the content they're looking for, making it unscalable ? Or do you make one torrent per package, meaning peers have to track thousands of torrents ? (That's the direction IPFS took, which is why it's extremely inefficient and resource-heavy). DebTorrent has made the choice to go in the middle. The real problem, to me, is that content-addressed content is the anti-privacy protocol: since you ask everyone "do you have this content", you're basically telling everyone what you want. In the case of installing packages, this means telling everyone what the version of your packages is, and is potentially a security issue. But the real "problem" is that it fails at being much more better than the existing solution. Bandwidth is cheap, storage is cheap, so hosting everything on a few independent but centralized servers is good enough.

rvz 3y ago

> I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it. Netflix uses IPFS. [0] [0] https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distribution https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-conta...

orblivion 3y ago

> Linux package distribution FWIW it seems like NixOS at least tried or is trying: https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/ https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/

LanternLight83 3y ago
Guix has had some developments in this direction too. https://libreplanet.org/wiki/Group:Guix/GSoC-2023 https://libreplanet.org/wiki/Group:Guix/GSoC-2023 https://issues.guix.gnu.org/52555 https://issues.guix.gnu.org/52555