5 ms·

The IPFS mirror really gives a substantial performance boost. In the past, the files had to be fetched from a tired server far away in Russia, the speed can be

The IPFS mirror really gives a substantial performance boost. In the past, the files had to be fetched from a tired server far away in Russia, the speed can be as low as 0.5 Mbps in my experience (depends on your ISP and connectivity), now it can be lightning fast thanks to P2P, 2 Mbps+, more than adequate to download a 100 MiB+ file. You don't even have to install IPFS (but you probably should, to maintain decentralization), books can be downloaded straight via HTTPS from Cloudflare's IPFS reverse proxy (called a "gateway"), cloudflare-ipfs.com. The IPFS network itself cannot be taken down, but I wonder when will the publishers discover IPFS web gateways, and start filing DMCA takedown requests against them, and what are the gateways gonna do? Will Cloudflare implement a huge copyright blocklist or even terminate its service (or worse, send a list of IP addresses to Prentice Hall or Elsevier)? What about the semi-official ipfs.io gateway? And the smaller independent IPFS gateways (there are currently 10+ on the public web)? Will they be DMCAed out of existence?

traverseda 6y ago
In my experience IPFS is only fast when you're using it through something like Cloudflare's IPFS proxy (which is basically a caching proxy). I haven't found IPFS to be actually usable any time I've tried running a node myself. Is that unusual?
- segfaultbuserr 6y ago
  Interesting, didn't notice that before. I believed my downloads are too obscure to be cached by Cloudflare, but I may be wrong. I'll do a comparison once I fixed my server at home...
- the_duke 6y ago
  No. The last time I tried it (maybe 9-12 months ago) it was a real resource hog (including choking the network) and really slow.
- magila 6y ago
  This has been the state of IPFS for years now. The protocol and client haven't been developed to where they can work reliably in a real P2P scenario. IPFS only appears to work because most requests go strait to one of the big gateways which already has the content cached. Development of IPFS slowed to a crawl when most of Protocol Labs switched to working on Filecoin. It remains to be seen if either Filecoin or IPFS will ever be viable for their intended purpose.
  - georgyo 6y ago
    I'm not sure you are right about this. The following link will stream big buck bunny directly from the network with full P2P support. It is not using a gateway node for the video content. https://bbb.fu.io/ https://bbb.fu.io/ You can run the following in the console to see all your peers for await (const peer of await node.swarm.peers()) { console.log(peer.addr.toString()) } The console currently get's spammed pretty hard because they haven't implemented filtering out plain ws (non-wss) peers if you are connecting https. But it will use webrtc to find peers, and find all wss nodes in the network.
    - magila 6y ago
      For me at least the vast majority of the data is being downloaded from various servers under bootstrap.libp2p.io or preload.ipfs.io. These are clearly dedicated servers not true peers. The contact info for these servers appears to be retrieved via the DNS rather than the DHT. This isn't surprising because browser clients can't participate in the DHT, but the DHT is currently one of the biggest weak points of IPFS. IPFS's DHT is so chatty that it consumes over a megabit of throughput just to maintain an idle node. Lookups are excruciatingly slow and often fail completely.
      - georgyo 6y ago
        Sorta, a peer can advertise dns addresses, and the browser can participate in the dht as a client. The example above uses webrtc-star for browser to browser discovery. $ ipfs dht findpeer 12D3KooWJxNHY6zE1KnzFdBJrTMCbhCVNtSLvjp7qR5Wsp49DnUC /dnsaddr/ipfs.scalable.io /dns4/ipfs.scalable.io/udp/4001/quic /dns6/ipfs.scalable.io/tcp/443/wss /dns6/ipfs.scalable.io/udp/4001/quic /dns4/ipfs.scalable.io/tcp/443/wss
- swills 6y ago
  Are you opening the required ports (4001, 8080)? I found it was like that too until I did opened the ports.
- StavrosK 6y ago
  I have to agree. I've had cases where I had to pin data that was on the node already and IPFS just hung the pin for minutes, failed, and then the GC removed the data later on. This is just one of the issues I had, other things include IPFS not being able to discover content on nodes that were already connected to it, IPFS just failing all my queued pins at once for some reason, etc. I regret ever building a pinning platform on IPFS. The worst thing is that, when I expressed my frustration on Twitter about an IPFS release completely breaking IPNS updates, the founder called me a Karen.
mathnmusic 6y ago
And what about when browsers (like Brave) start supporting IPFS natively, and gateways are not needed any more?
- a1369209993 6y ago
  Browsers still don't support (better-established, though I'm not sure older offhand) TOR or bittorent yet, so I'm not terribly optimistic.
an_opabinia 6y ago
> send a list of IP addresses to Prentice Hall or Elsevier I don’t know, do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue? Even the lawyers would be wrong on that one.
- segfaultbuserr 6y ago
  > do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue? I'm not sure about the implication of your words. But, clearly, Elsevier has already tried to take legal actions against LibGen previously, and once in a while, LibGen's domain names are still being blocked and routinely replaced, just like Sci-Hub. I'd still say the reason it's still running is a jurisdiction issue, or let's say it's a geopolitical issue, it's using Russia to shield itself away from US influences. But an IPFS gateway hosted in the United States by Cloudflare does not enjoy these legal and political benefits.
tutfbhuf 6y ago
IPFS gateways are similar to Tor exit relays, they just route the traffic to the public internet. Tor exit relays still exist, and haven't been DCMAed out of existence so far. One could also imagine IPFS over Tor.
- georgyo 6y ago
  You're almost right. The IPFS gateway nodes also cache the content. IE, they are both storing, hosting, and sharing to other IPFS nodes.
  - mpfundstein 6y ago
    how can one validate that the cached version is indeed bit equal to the one stored within ipfs?
    - openfuture 6y ago
      You hash it, everything is content addressed.
    - georgyo 6y ago
      The address of the file is the hash of the file. (Really the hash of the DAG of the file) So you could validate it if you wanted to. Because the URL _is_ the hash, if you did validate it you could be very sure that it is correct. If a webserver hosts a file, how can you be sure that file is correct? Sometimes the ship a sums file next to it. At best this can find corruption, but it would not find malicious files.
- mintplant 6y ago
  IPFS-over-Tor has been trapped in development hell for 5 years. https://github.com/ipfs/notes/issues/37 https://github.com/ipfs/notes/issues/37
  - segfaultbuserr 6y ago
    If IPFS-over-Tor is in the development hell, IPFS-over-I2P is in the development limbo. https://github.com/ipfs/notes/issues/124 https://github.com/ipfs/notes/issues/124
jakobdabo 6y ago
Am I understanding it right that DMCA takedown requests apply to a URL? Maybe IPFS can introduce a feature to generate one-time URLs, e.g. `oneTimeURL = f(realURL);`. Websites then can generate new URLs for example every hour or even for each client request. Copyright bots will file a DMCA request to a URL which only they have.
- judge2020 6y ago
  In general DMCA takedown requests reference a URL to locate infringing content, but the actual requests ask that the content itself be removed and not just the URL.
- ALittleLight 6y ago
  I believe this is similar logic to what MegaUpload used. i.e. They would allow users to easily create links to content, and if DMCA'd would invalidate the links but leave the content alone (and the users free to generate arbitrarily many more links to keep sharing copyrighted content). It kind of worked for MegaUpload. Their founder got rich and a version of the company still exists. However, the founder is also gradually losing a long court battle and may find himself extradited to the US where he will face significant jail time.
matheusmoreira 6y ago
They'll probably lobby the governments for more laws. It's a politico-technological arms race. Governments make laws. People don't agree with them so they create technology that subverts the government. The state must then become even more tyrannical just to maintain the same level of power. We'll end up with either an uncontrollable population or a totalitarian government.

traverseda 6y ago

In my experience IPFS is only fast when you're using it through something like Cloudflare's IPFS proxy (which is basically a caching proxy). I haven't found IPFS to be actually usable any time I've tried running a node myself. Is that unusual?

segfaultbuserr 6y ago
Interesting, didn't notice that before. I believed my downloads are too obscure to be cached by Cloudflare, but I may be wrong. I'll do a comparison once I fixed my server at home...
the_duke 6y ago
No. The last time I tried it (maybe 9-12 months ago) it was a real resource hog (including choking the network) and really slow.
magila 6y ago
This has been the state of IPFS for years now. The protocol and client haven't been developed to where they can work reliably in a real P2P scenario. IPFS only appears to work because most requests go strait to one of the big gateways which already has the content cached. Development of IPFS slowed to a crawl when most of Protocol Labs switched to working on Filecoin. It remains to be seen if either Filecoin or IPFS will ever be viable for their intended purpose.
- georgyo 6y ago
  I'm not sure you are right about this. The following link will stream big buck bunny directly from the network with full P2P support. It is not using a gateway node for the video content. https://bbb.fu.io/ https://bbb.fu.io/ You can run the following in the console to see all your peers for await (const peer of await node.swarm.peers()) { console.log(peer.addr.toString()) } The console currently get's spammed pretty hard because they haven't implemented filtering out plain ws (non-wss) peers if you are connecting https. But it will use webrtc to find peers, and find all wss nodes in the network.
  - magila 6y ago
    For me at least the vast majority of the data is being downloaded from various servers under bootstrap.libp2p.io or preload.ipfs.io. These are clearly dedicated servers not true peers. The contact info for these servers appears to be retrieved via the DNS rather than the DHT. This isn't surprising because browser clients can't participate in the DHT, but the DHT is currently one of the biggest weak points of IPFS. IPFS's DHT is so chatty that it consumes over a megabit of throughput just to maintain an idle node. Lookups are excruciatingly slow and often fail completely.
    - georgyo 6y ago
      Sorta, a peer can advertise dns addresses, and the browser can participate in the dht as a client. The example above uses webrtc-star for browser to browser discovery. $ ipfs dht findpeer 12D3KooWJxNHY6zE1KnzFdBJrTMCbhCVNtSLvjp7qR5Wsp49DnUC /dnsaddr/ipfs.scalable.io /dns4/ipfs.scalable.io/udp/4001/quic /dns6/ipfs.scalable.io/tcp/443/wss /dns6/ipfs.scalable.io/udp/4001/quic /dns4/ipfs.scalable.io/tcp/443/wss
swills 6y ago
Are you opening the required ports (4001, 8080)? I found it was like that too until I did opened the ports.
StavrosK 6y ago
I have to agree. I've had cases where I had to pin data that was on the node already and IPFS just hung the pin for minutes, failed, and then the GC removed the data later on. This is just one of the issues I had, other things include IPFS not being able to discover content on nodes that were already connected to it, IPFS just failing all my queued pins at once for some reason, etc. I regret ever building a pinning platform on IPFS. The worst thing is that, when I expressed my frustration on Twitter about an IPFS release completely breaking IPNS updates, the founder called me a Karen.

mathnmusic 6y ago

And what about when browsers (like Brave) start supporting IPFS natively, and gateways are not needed any more?

a1369209993 6y ago
Browsers still don't support (better-established, though I'm not sure older offhand) TOR or bittorent yet, so I'm not terribly optimistic.

an_opabinia 6y ago

> send a list of IP addresses to Prentice Hall or Elsevier I don’t know, do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue? Even the lawyers would be wrong on that one.

segfaultbuserr 6y ago
> do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue? I'm not sure about the implication of your words. But, clearly, Elsevier has already tried to take legal actions against LibGen previously, and once in a while, LibGen's domain names are still being blocked and routinely replaced, just like Sci-Hub. I'd still say the reason it's still running is a jurisdiction issue, or let's say it's a geopolitical issue, it's using Russia to shield itself away from US influences. But an IPFS gateway hosted in the United States by Cloudflare does not enjoy these legal and political benefits.

tutfbhuf 6y ago

IPFS gateways are similar to Tor exit relays, they just route the traffic to the public internet. Tor exit relays still exist, and haven't been DCMAed out of existence so far. One could also imagine IPFS over Tor.

georgyo 6y ago
You're almost right. The IPFS gateway nodes also cache the content. IE, they are both storing, hosting, and sharing to other IPFS nodes.
- mpfundstein 6y ago
  how can one validate that the cached version is indeed bit equal to the one stored within ipfs?
  - openfuture 6y ago
    You hash it, everything is content addressed.
  - georgyo 6y ago
    The address of the file is the hash of the file. (Really the hash of the DAG of the file) So you could validate it if you wanted to. Because the URL _is_ the hash, if you did validate it you could be very sure that it is correct. If a webserver hosts a file, how can you be sure that file is correct? Sometimes the ship a sums file next to it. At best this can find corruption, but it would not find malicious files.
mintplant 6y ago
IPFS-over-Tor has been trapped in development hell for 5 years. https://github.com/ipfs/notes/issues/37 https://github.com/ipfs/notes/issues/37
- segfaultbuserr 6y ago
  If IPFS-over-Tor is in the development hell, IPFS-over-I2P is in the development limbo. https://github.com/ipfs/notes/issues/124 https://github.com/ipfs/notes/issues/124

jakobdabo 6y ago

Am I understanding it right that DMCA takedown requests apply to a URL? Maybe IPFS can introduce a feature to generate one-time URLs, e.g. `oneTimeURL = f(realURL);`. Websites then can generate new URLs for example every hour or even for each client request. Copyright bots will file a DMCA request to a URL which only they have.

judge2020 6y ago
In general DMCA takedown requests reference a URL to locate infringing content, but the actual requests ask that the content itself be removed and not just the URL.
ALittleLight 6y ago
I believe this is similar logic to what MegaUpload used. i.e. They would allow users to easily create links to content, and if DMCA'd would invalidate the links but leave the content alone (and the users free to generate arbitrarily many more links to keep sharing copyrighted content). It kind of worked for MegaUpload. Their founder got rich and a version of the company still exists. However, the founder is also gradually losing a long court battle and may find himself extradited to the US where he will face significant jail time.

matheusmoreira 6y ago

They'll probably lobby the governments for more laws. It's a politico-technological arms race. Governments make laws. People don't agree with them so they create technology that subverts the government. The state must then become even more tyrannical just to maintain the same level of power. We'll end up with either an uncontrollable population or a totalitarian government.