8 ms·
I like IPFS, I really do, but whenever I try to use it, it's either too slow to become usable or sometimes it plain doesn't work. I pinned a whole bunch of file
by jeroenhd 5y ago
I like IPFS, I really do, but whenever I try to use it, it's either too slow to become usable or sometimes it plain doesn't work. I pinned a whole bunch of files on IPFS a while back to experiment with it and the system seems to work, but every time I try to fetch those resources from a location that hasn't cached the content yet, it takes several seconds to show me the HTML/JSON/PNG files.
HTTP may be inefficient for document storage, but IPFS is inefficient for almost everything else.
I like the concepts behind IPFS but it's simply not practical to use the system in its current form. I hope my issues will get resolved at some point but I can't shake the thought that it'll die a silent death like so many attempts to replace the internet before it.
- zozbot234 5y agoSounds like it's working fine for you. "Several seconds" of lag is nothing for an "Inter-Planetary File System", in fact it's on par with other decentralised P2P networks.
- Taek 5y agoSkynet fetches files in under 100ms, you can definitely get a decentralized system going as fast as the centralized web if you build it right. The main challenge for me with this comment is that you can't expect distributed/decentralized networks to win if you set an expectation that "things will just be slower than the normal web". Nobody is going to migrate to that.
- amelius 5y ago> Skynet I don't know Skynet. I first checked Google, got a Wikipedia link describing a movie, then checked the Wikipedia disambiguation page, but got nothing. https://en.wikipedia.org/wiki/Skynet https://en.wikipedia.org/wiki/Skynet Also, why would a project duplicate the efforts of IPFS rather than contribute to it?
- Taek 5y agohttps://siasky.net/ https://siasky.net/ and https://docs.siasky.net/ https://docs.siasky.net/ IPFS has chosen an architecture which fundamentally keeps it non-performant, Skynet is built from the ground up in a different way, and gets 10-100x improvements on performance for content-addressed links, and 100-1000x improvements on performance for dynamic lookups (IPNS)
- CharlesW 5y ago> "Several seconds" of lag is nothing for an "Inter-Planetary File System", in fact it's on par with other decentralised P2P networks. That's good enough for kicking off batch file transfers (assuming you mean P2P networks like BitTorrent), but there's no evidence that people will tolerate a slow web, and lots of evidence that they won't.
- leesalminen 5y agoI agree that the http caching layers are currently needed to achieve decent UX. But it’s also possible that won’t be necessary forever. The network could expand and get to the point where resolution time comes down to accessible levels. Only time will tell I suppose.
- lillecarl 5y agoWhen downloading content using a magnet link downloads are usually quite slow started, while torrent files usually start directly. It's not a fair comparison since the torrents are from a private tracker with high quality peers, but it's noticeable that the DHT stuff is slower. Not sure how cjdns solves that. Will the network be faster the more people join it? Is centralized infra inevitable for fast things because of all assumptions and insight a centralized provider can use? BTC is also slow, I don't know of any cryptocurrency that can provide VISA transaction volumes, even if they use more power than some countries alone.
- Taek 5y agoCryptocurrency is different from storage because cryptocurrency needs to provide global consistency guarantees. Skynet is a decentralized network with lookup times that have a p50 TTFB of under 200ms. It achieves this by looking things up directly on hosts rather than routing through a DHT. There's a bit of overhead to accomplish this (around 200kb of extra bandwidth per lookup), but for a smooth web experience that tradeoff is more than worthwhile.
- Gigachad 5y agoOn what basis do you think this is likely? What part of IPFS gets faster when more users are online? If the file exists on a system in the network, it should make a connection and start fetching the file within milliseconds ideally. It's not like we are distributing multi gb files where more seeders means more speed, this is all just slowness in setting up a connection. Even torrents take 1-10 seconds to initiate downloads with an absolutely massive network.
- 5y ago
- spicybright 5y agoWonder if it's practical to "buffer" popular content on IPFS by copying it to normal HTTP servers. Requesting an IPFS document would query a few popular repositories, then revert back to normal IPFS if it's not found. These buffer servers would also track what's popular and shuffle around what they store accordingly.
- jeroenhd 5y agoI think this is exactly what Cloudflare's and ipfs.io's web proxies do. They won't cache your stuff forever, but they'll cache it as long as someone requests the content again before the content gets removed from cache. The downside of this approach is that it only works with popular nodes and you'd be back to the old, centralised internet architecture for all real use cases. I don't think you can accurately gauge what is and isn't popular in a P2P network like IPFS. You never have a view of the entire network, after all. There's also the problem of running such a system. Who pays for the system's upkeep and do we trust them? If we'd use Cloudflare's excellent features, who says Cloudflare won't intentionally uncache a post criticising their centralisation of the internet, forcing the views they disagree with to the slow net while the views they agree with get served blazingly fast. I don't think such a system would work well if we intend to keep the decentralised nature of IPFS alive. Explicit caching leads to centralization, that's the exact reason caching works. Instead, the entire network needs a performance boost. I don't know where the performance challenges in IPFS lie, but I'm sure there's ways to improve the system. Getting more people to run and use IPFS would obviously work, but then you'd still only be caching only popular content. Edit: actually, I don't really want to see caching happen through popularity of the service either, because as it stands IPFS essentially shares your entire browsing history with the world by either requesting documents in plain text or even caching the documents you've just read. I wonder if that IPFS-through-Tor section on their website ever got filled in, because the last few times I've checked that was just a placeholder in their documentation.
- zinodaur 5y agoHow much were you paying for your IPFS pin? E.g., if you are getting something via HTTP, there's a server somewhere with that content just waiting for you to request it, typically stored on an SSD, etc. V.s. IPFS pins which are typically packed on to massive disks shared with lots of other people IDK a whole lot about IPFS though. Maybe it was the metadata resolving / DHT lookup or whatever that was super slow. BitTorrent latency was always pretty high, but it didn't matter because throughput was also high
- wsjtho55 5y agoWireguard and the usual tools for file search and retrieval works fine. Do we need all that comes with IPFS? Not just technically, but the user training and pivot of technical doers? So many of these projects feel like programmer vanity projects, there’s really little difference between them and a guy on the corner telling me why Protestants are wrong, join his flock. That it’s a technical project not entirely ephemeral nonsense doesn’t matter; solutions exist already we just don’t implement that way.
- stavros 5y agoThis echoes my experience as well, and I (used to) run a pinning service.
- noman-land 5y agoI love IPFS. It's one of my favorite recent technologies, but I think people have unrealistic expectations about such a young idea. Decentralized tech doesn't work well until the network effects build up. IPFS has the interesting quality that the more popular a piece of content is, the easier it is to get ahold of. If millions of people were using IPFS, the most popular content would be being served by many thousands of people, and finding it and downloading it would be extremely fast. Then subsequent viewings would be instantaneous because you can cache it for life. This leaves interesting incentives for monetizing pinning and caching for less popular content. It makes sense if you ask me. If I love a piece of music so much that I'm willing to give it to others for free then everyone benefits from being able to access it easier. Content that people care about organically becomes more resilient and nearly impossible to remove. Content that no one cares about is slow and inefficient because it has to be hauled out of cold storage the one time a year anyone cares. If someone thinks that content is more important than people are giving it credit for they can host it or pay for someone else to do it. If you have a website and you have "fans" that subscribe to you and help pin all your stuff, then your stuff becomes faster and easier to get. Your "fans" can even get paid for helping to serve your content. So, to me, it's early days for IPFS, and the way to make it better is to try to build apps that increase its usage, so the power of the network effects is felt.
- wly_cdgr 5y ago> IPFS has the interesting quality that the more popular a piece of content is, the easier it is to get ahold of. That sounds like the worst of the current internet, but even worse
- deleted 5y ago[deleted]
- guntars 5y agoTry browsing the IPFS example "website". I opens for me under a few hundred milliseconds. ipfs cat /ipfs/QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc/readme
- Taek 5y agoThe IPFS network tends to run quickly if the file you are fetching is stored in the cache of either ipfs.io or in the cache of cloudflare. Everything else has lookup times of 30-60 seconds, sometimes more. DHTs just aren't a good choice for massive data systems if you need low latency.
- jeroenhd 5y agoI think that's an unfair example because those files come pre-pinned or at least pre-loaded with most IPFS installs. I've just pinned a 12MiB file filled with random bytes on one of my servers (`dd if=/dev/urandom of=test.dat bs=1M count=12; ipfs add test.dat; ipfs pin <hash that came out>`). The server has a 50mbps uplink, so transferring the file to my laptop should take about two seconds. Dumping this blog's contents over IPFS takes the server about 3 seconds (first time load) so the network seems to be in working order, at least when downloading data. `ipfs swarm peers` lists about 800 known peers. On the server itself, `ipfs cat /ipfs/redacted > /tmp/test.dat` runs in about a second, which is all perfectly acceptable overhead for a transfer that'll take two to three seconds anyway. On my laptop, I've tried to get the file but I just cancelled it after waiting for 16 minutes. Halfway throughout the wait, I've tried opening the file through the ipfs.io proxy, which finally gave me the file after a few minutes, but no such luck yet if I retry the ipfs command. I don't know if it's the random file, the size, or something different, but if I'm launching a blog or publishing documents on IPFS, visitors should not be expected to wait five to ten minutes for the data to load. "After the first twenty visitors it'll get faster" is not a solution to this problem, because there won't be twenty visitors to help the network cache my content. Maybe I'm expecting too much here; maybe the files shouldn't be expected to be available within half an hour, or before Cloudflare caches it. Maybe there's something wrong with my laptop's setup (I haven't done any port forwarding and I'm behind a firewall). Either way, if I follow the manual but can still buy a domain, set up DNS and hosting on my VPS and send a link to a friend faster than I can get the file through P2P, I don't think IPFS will ever get off the ground. Fifteen minutes is an awful lot of time for a data transfer these days! Edit: actually, now it seems ipfs.io and cloudflare have picked up the file in their caches. Data transfer is up to normal speed now. If you want to try to replicate my experiment, I've just uploaded a new test file to /ipfs/QmbBD872kjfoutAmTKFCxTCApw9LBB9qxxRyXpEGYzsqMH. Edit 2: I realized that by saying I downloaded the file and that the file is random, I just announced my personal IP address to the world through the IPFS hash, so I removed it. That was pretty dumb of me, and also a pretty clear problem of IPFS in my book.
- dvcrn 5y agoI keep IPFS companion turned on constantly but when I hit a site that's getting loaded through IPFS it often takes so long that I end up turning IPFS off so it just fetches it from the central server