3 ms·

I do wonder why IA does not maintain a IPFS instance, or if they do, why they're not more popular? There's tons of IPFS mirror services out there that operate a

I do wonder why IA does not maintain a IPFS instance, or if they do, why they're not more popular? There's tons of IPFS mirror services out there that operate at reasonable speeds. One issue I've run into with IA is old enough websites that there's JS or CSS that just wont render, what I'm not sure about is, can we retroactively fix such things? Would be nice to be able to un-ruin the code somehow if they exported everything possible at the time. Edit: Would be really neat if you could click on a domain while on IA, and a desktop client downloads as many WAR files in a slower priority download queue, as many as you're interested in, with higher priority pages first, and then you can view it fully offline.

stavros 1y ago
Because nobody pins on IPFS. It's basically http with extra steps, at this point.
TechSquidTV 1y ago
They do torrents. I was looking into this recently as well, considering building an Activity Pub alternative to IA. I came to what I assume is the same conclusion that IA came to. No one uses IPFS. For the average user, it is significantly more difficult to get started. For the experienced user, the ecosystem of tools around IPFS is extremely small. All in all, IPFS offers very little benefit over torrents in practice and has a much smaller user pool.
- outside1234 1y ago
  IPFS is a great idea poorly executed. Content addressable storage is a great idea, but it is so difficult to use in practice for real world scaled scenarios (larger than one hard disk drive).
- kevincox 1y ago
  The problems with the torrents is that they can be updated if the file changes (sometimes small metadata changes) and now your seeders can't be found. Maybe if they also kept a list of old hashes so that you could at least manually try to recover data from the older torrent?
  - Lammy 1y ago
    This is outdated information. These issues have been solved by various BitTorrent Enhancement Proposals. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old torrent. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents: https://www.bittorrent.org/beps/bep_0039.html https://www.bittorrent.org/beps/bep_0039.html https://www.bittorrent.org/beps/bep_0046.html https://www.bittorrent.org/beps/bep_0046.html If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html https://www.bittorrent.org/beps/bep_0052.html This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrents with which it shares files, so if you already have an old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html https://www.bittorrent.org/beps/bep_0038.html
    - NoMoreNicksLeft 1y ago
      Have any of these even started to be implemented in any client/library? It's been years.
    - deleted 1y ago
      [deleted]
komali2 1y ago
I spent a bit of time trying to find it just now but I swear I read a super long blog or comment or something by someone at archive.org where they concluded essentially that IPFS just "isn't ready" or wasn't feasible for their needs because it's super slow and they didn't see how that couldn't be the case when they consider the volume of transactions they need to do (they didn't see an optimization path). I wish I could find that article! edit: https://github.com/internetarchive/dweb-archive/blob/master/docs/archive_architecture_ipfs.md https://github.com/internetarchive/dweb-archive/blob/master/...

stavros 1y ago

Because nobody pins on IPFS. It's basically http with extra steps, at this point.

TechSquidTV 1y ago

They do torrents. I was looking into this recently as well, considering building an Activity Pub alternative to IA. I came to what I assume is the same conclusion that IA came to. No one uses IPFS. For the average user, it is significantly more difficult to get started. For the experienced user, the ecosystem of tools around IPFS is extremely small. All in all, IPFS offers very little benefit over torrents in practice and has a much smaller user pool.

outside1234 1y ago
IPFS is a great idea poorly executed. Content addressable storage is a great idea, but it is so difficult to use in practice for real world scaled scenarios (larger than one hard disk drive).
kevincox 1y ago
The problems with the torrents is that they can be updated if the file changes (sometimes small metadata changes) and now your seeders can't be found. Maybe if they also kept a list of old hashes so that you could at least manually try to recover data from the older torrent?
- Lammy 1y ago
  This is outdated information. These issues have been solved by various BitTorrent Enhancement Proposals. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old torrent. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents: https://www.bittorrent.org/beps/bep_0039.html https://www.bittorrent.org/beps/bep_0039.html https://www.bittorrent.org/beps/bep_0046.html https://www.bittorrent.org/beps/bep_0046.html If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html https://www.bittorrent.org/beps/bep_0052.html This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrents with which it shares files, so if you already have an old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html https://www.bittorrent.org/beps/bep_0038.html
  - NoMoreNicksLeft 1y ago
    Have any of these even started to be implemented in any client/library? It's been years.
  - deleted 1y ago
    [deleted]

komali2 1y ago

I spent a bit of time trying to find it just now but I swear I read a super long blog or comment or something by someone at archive.org where they concluded essentially that IPFS just "isn't ready" or wasn't feasible for their needs because it's super slow and they didn't see how that couldn't be the case when they consider the volume of transactions they need to do (they didn't see an optimization path). I wish I could find that article! edit: https://github.com/internetarchive/dweb-archive/blob/master/docs/archive_architecture_ipfs.md https://github.com/internetarchive/dweb-archive/blob/master/...