14 ms·

Ask HN: Why don't more software projects use BitTorrent for downloads?

• lerigner 5y ago
  Twitter using BitTorrent on the Backend (2010): >The project uses BitTorrent technology and is called Murder https://www.bittorrent.com/blog/2010/02/09/twitter-using-bittorrent-on-the-backend/ https://www.bittorrent.com/blog/2010/02/09/twitter-using-bit...
• fxtentacle 5y ago
  A) If you don't use the cloud, bandwidth is so dirt cheap that nobody cares. B) Some ISPs will throttle you heavily if you upload too much. C) Some users still have to pay fees for upload traffic.
• ksec 5y ago
  Without Browser Support BT Requires additional client. >which I imagine could incur substantial hosting costs over time. And bandwidth is extremely cheap. ( Look at Cloudflare ) Lots of sponsorship and help for Open Source project or if you want to host it on Github for free. Bandwidth is also getting cheaper. May be there is a floor somewhere, but we dont seems to have that in sight.
• wink 5y ago
  Because I dislike it so much as a user that I never got into it and never would have thought to propose it. In Germany, before Fibre (and lol who has that), upload speeds for DSL and Cable have usually been only 10-20% of download speeds, and I know a lot of people who have stayed away from it for that reason alone. Yes, you can limit it, but it kinda kills some of the whole thing. Also the hassle with port forwarding etc.pp. I think I didn't have a client installed on any machine for the last 10 years.
  • Pooge 5y ago
    > upload speeds for DSL and Cable have usually been only 10-20% of download speeds Why or how is that an issue? Anybody that contributes to the seeding, however small, matters. Because files are cut into a lot of pieces, there is no competition for the "faster upload speed". > Also the hassle with port forwarding That, I can understand, but it's fairly easy to open a port. And if your router supports it you can enable UPnP. I am the exact opposite of you: as I user, I absolutely love it and it has to be one of my favorite piece of technology.
    • wink 5y ago
      > Why or how is that an issue? Anybody that contributes to the seeding, however small, matters. Because files are cut into a lot of pieces, there is no competition for the "faster upload speed". If it's a localized thing, for example a game client in your language, and everyone has shitty upload speed, the critical mass would need to be 10x as big so that not everyone has a shitty download rate. Maybe a few single cases soured me on the experience. > port forwarding. it's easy if you do 'port X to machine Y' but like in the above example, try 3 computers in the network who need the same 20GB download. it sucks. (esp if it's not /a file/ you can then locally distribute, but it's a game company's launcher)
      • Pooge 5y ago
        > critical mass would need to be 10x as big so that not everyone has a shitty download rate Yes, but those who are downloading have - in the best case scenario - a better bandwidth and can therefore seed even more than before. From my experience, the scenario where nobody is seeding anymore - which wouldn't happen in this case - is much worse than the scenario where everybody has a slow upload speed. The more people download, the more they seed. Sure, there's a bottleneck at the start of the life of the torrent (essentially the creator of the torrent), but after it really evens out. > it's easy if you do 'port X to machine Y' but like in the above example, try 3 computers in the network who need the same 20GB download I'm sorry, I don't think I understood correctly. I'm going to answer as I understood: in that case, wouldn't you just attribute different ports to your local devices?
• linkdd 5y ago
  When distributing over torrent, you need at least one seeder with a good connection. You obviously. So you still need to host a server with your package on it, and configure a torrent client to seed it. I know a lot of HTTP/FTP storage space providers, or I can just put my binary on Github (Github Pages, or Release asset), or host it on a CDN like netlify. I think there is more solutions than real numbers between 0 and 1. But I don't know a single "torrent seedbox" provider. Setting it up yourself is not complicated, but since it's DIY, this alone explains why it's not more wide-spread.
  • charcircuit 5y ago
    You don't need a seedbox. You just make the torrent file reference the normal HTTP download.
  • pabs3 5y ago
    There is a way that you can add a URL to a torrent file as a seed location, then you always have a backup seed for when there are no torrent seeds. https://en.wikipedia.org/wiki/BitTorrent_%28protocol%29#Web_seeding https://en.wikipedia.org/wiki/BitTorrent_%28protocol%29#Web_...
  • kelseyfrog 5y ago
    Can't you use S3 to seed a torrent? https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTorrent.html https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjec...
    • miked85 5y ago
      They seem to have deprecated support [1] [1] https://news.ycombinator.com/item?id=27524549 https://news.ycombinator.com/item?id=27524549
    • banana_giraffe 5y ago
      As mentioned, S3's torrent server is no longer a thing. When it worked, it also was never that great. The WebSeed AWS ran was slow, the tracker was always problematic as well.
• cjbprime 5y ago
  For the least friction, you could make the download page use Webtorrent, which is a BitTorrent bridge using WebRTC datachannels.
• PaulHoule 5y ago
  It's not really worth it for small downloads. I looked at one of my projects and it is an 880kb download. If you are talking a gigabyte or more there is a big advantage in terms of (1) a completely reliable download, and (2) being able to split up the download into multiple streams. At home I have two ADSL lines and a load balancer and with bittorrent I get double-speed downloads.
  • stevenwoo 5y ago
    Why don't large companies use it for updates? I have a lot of trouble downloading the 10+ gigabyte IOS downloads on DSL for some reason but torrenting works fine for larger files and stopping and resuming does not break torrenting unlike Apple's update process.
    • llui85 5y ago
      Musescore is planning to use P2P sharing to download sample libraries (>1gb) and updates in version 4 (coming in the next few months).
    • PaulHoule 5y ago
      I don't know how they do it but when I download from Steam there are multiple streams and I get about 200% of the throughput of a single ADSL.
      • duffyjp 5y ago
        I don't know if this would be of interest to you, but https://lancache.net/ https://lancache.net/ is really fun to setup. I assigned an old (unreliable) 500gb SSD to a Linux VM and run Lancache on it. A couple tweaks to Pi Hole and every system in your house will use with no configuration (and fallback to Steam if it's down). If you only have a single gaming system it won't matter but if you have a few it's fun to watch the second one get 100MB/s Steam downloads in a house with 20MB/s internet.
      • Arie 5y ago
        Steam downloads are plain HTTP in separate file chunks. Easy to cache and load balance across connections.
• ralusek 5y ago
  The best downside that I've heard, at least for doing this automatically, is that not all users have cheap/available upload bandwidth. As long as you're able to opt in as a seeder, it's great.
• garbagecoder 5y ago
  Until everyone has synchronous internet it won’t need to be a thing. I’ve pointed out many times that between BitTorrent, tor, and bitcoin we’ve already done alleged web3, which is what we used to call the Dark Web.
  • unforeseen9991 5y ago
    Plus a whole number of other distributed projects over the years. Everything web3 is just hyped ideas that are already out there, in some cases for 10+ years.
    • familymart 5y ago
  • betwixthewires 5y ago
    That's a good point, you're right. I hadn't thought of it this way. What we want already exists, we just have to use it.
• wmf 5y ago
  There's a ton of free HTTP mirroring available for open source; BitTorrent really has no advantage. Bandwidth is so cheap in general that BitTorrent is not needed.
• Froedlich 5y ago
  A lot of ISPs nuke anything that looks like a torrent and send angry messages to their customers.
  • amir734jj 5y ago
    Can confirm this.
    • copperx 5y ago
      Country? Provider? Type of download?
  • LeoPanthera 5y ago
    Is that actually true? Because I thought it was "content" owners hunting for torrents containing their stuff and then sending DMCA complaints (or local equivalent) to the ISPs of everyone in that particular swarm. Are there really ISPs that just blanket block all torrents? I've never used one that does.
    • Shared404 5y ago
      There's some local to me that don't necessarily block torrents, but will get upset. Mostly just means I set up a VPN through Linode or whatever.
  • banana_giraffe 5y ago
    And some corporate environments might be ok with non-business uses of Internet data, but uncomfortable talks will be had if they detect BitTorrent.
• malermeister 5y ago
  Blizzard did this for a while for their WoW updates [0]. Not sure if they still do though. [0] https://wowpedia.fandom.com/wiki/Blizzard_Downloader https://wowpedia.fandom.com/wiki/Blizzard_Downloader
• pabs3 5y ago
  Unfortunately web browsers don't support it, so it requires an external client to download.
  • TheFreim 5y ago
    The Brave browser has a BitTorrent client built in, though I've never used it. Not a major player in the browser market, though.
  • chc4 5y ago
    There is WebTorrent, which uses WebRTC. instant.io uses it for file sharing, for example.
    • pabs3 5y ago
      I got the impression that WebTorrent users can't download from BitTorrent users and vice versa, is that true?
      • sergiotapia 5y ago
        Last I checked webtorrent users can download from webtorrent users. So they are isolated like that.
        • hiimshort 5y ago
          If I remember correctly, there is a server bridge that enables communication between webtorrent and normal bittorrent clients.
          • baobabKoodaa 5y ago
            Why would you use BitTorrent to distribute files P2P if you're just going to place a server between the peers? Wouldn't it be better to just have the server distribute the files in that case?
• pabs3 5y ago
  A lot of Linux based OSes also have BitTorrent downloads.
• LeoPanthera 5y ago
  Windows 10 claims it distributes updates in a peer-to-peer manner. Or at least it used to. Does that still happen?
  • waste_monk 5y ago
    Delivery Optimisation? Yes, that's still a thing. It doesn't have to be internet based, you can turn it down to just peers on your local network, or disable it entirely (if you have WSUS infrastructure and no need for it).
• _8j50 5y ago
  How is it better than HTTPS+CDNs both performance wise and implementation/maintainance complexity wise?
• gbraad 5y ago
  Blocked or otherwise impeded in some places: eg, unable to request peers from a tracker.
• HellsMaddy 5y ago
  GitHub/GitLab releases makes it easy, free, and has virtually no overhead to distribute software to millions of users globally. Setting up torrents and making sure they stay seeded costs time and money, and torrents are less convenient for most users.
  • newaccount74 5y ago
    We've had a few users complain that they had issues downloading large files from Github -- either S3 was blocked alltogether (China) or just very slow (Australia). That was a couple of years ago, don't know what the situation is like today. I think offering BT as an alternative would be a good idea, especially for downloads > 100MB.
  • nhoughto 5y ago
    Yep came to say the same, because a lot of software is already hosted in a lower friction way by GitHub and similar. So unless you have a need/want to host it yourself and maybe benefit from the optimizations then why bother?
  • dagmx 5y ago
    Those aren't necessarily mutually exclusive though. The Git hosts could also provide their releases by seeding them as an option.
  • sodality2 5y ago
    You can add the web URL seed to the torrent, pretty cool feature.
• d--b 5y ago
  Raspberry pi OS has both. Regarding hosting, I am not exactly sure that it changes anything. You still have to seed the torrent, so the file needs to be hosted somewhere. What is reduced is the bandwidth, but you rarely pay for bandwidth. I do like the torrent option, but it’s a little more work for the person who downloads.
• psuter 5y ago
  Facebook reportedly was using Bittorrent to distribute their own application across their servers. I remember outreach presentations where they mentioned this over 10 years ago, and this article seems to have some corroborating details: https://arstechnica.com/information-technology/2012/04/exclusive-a-behind-the-scenes-look-at-facebook-release-engineering/ https://arstechnica.com/information-technology/2012/04/exclu...
• soperj 5y ago
  This is something I've wondered specifically for Musescore 4. They're releasing their Orchestral library as a separate 7.5gb download, but instead of using bit-torrent, they're making you download a download manager. Seems so bizarre.
  • bombcar 5y ago
    Could the download manager be a torrent client in disguise?
    • soperj 5y ago
      Why would they do that though?
    • cuteboy19 5y ago
      If it's not p2p then there is no point to bittorrent
• u801e 5y ago
  Maybe not for individual software projects, but I would like to see something like this for installing and updating packages via the OS's package manager. Given the fact that package repositories have mirror servers, it should definitely be possible.
  • perihelions 5y ago
    Sounds like something that would have significant security impact, wouldn't it?
    • Jtsummers 5y ago
      BT is resistant to content spoofing. So unless the users download arbitrary .torrent files from untrusted sources, it will be hard to inject malware this way. You'd need to find hash collisions for the contents of the torrent. It would have some privacy implications, as you end up announcing "I'm using this OS!" to your peers (well, it's a high probability that you are at least).
      • perihelions 5y ago
        The same's true for ordinary package managers, but they've come with CVE's that allowed circumventing checksum validation [0]. Software mirror security doesn't seem to me (non-expert) a trivial or solved problem. [0] https://news.ycombinator.com/item?id=8330386 https://news.ycombinator.com/item?id=8330386
• banana_giraffe 5y ago
  I suspect a big part of the reason is "more work for very little gain" since most users will just download over HTTP given the choice. I'd love to see some stats from somewhere like Internet Archive that offers both, it'd be great to be proven wrong, but I suspect a number basically 0% after rounding of their data is downloaded via peer-to-peer BitTorrent traffic.
• shiado 5y ago
  Many ISPs give customers much better download speeds than upload speeds. I remember personally having problems using bittorrent and seeding too much destroying the network for all using it.
• WatchDog 5y ago
  There are so many free CDNs and mirrors, especially for hosting open source. Also most open source projects aren't particularly bandwidth hungry, why bother?
• gorgoiler 5y ago
  Spotify used to use BitTorrent, or something closely related. Back in the early days, colleagues in my office would be falsely accused by IT of file sharing, when the only crime they were really committing was listening to Jack Johnson.
  • crtasm 5y ago
    If that was the very early days they were quite possibly sharing unlicensed tracks: https://torrentfreak.com/spotifys-beta-used-pirate-mp3-files-some-from-pirate-bay-170509/ https://torrentfreak.com/spotifys-beta-used-pirate-mp3-files...
• jldugger 5y ago
  I'm pretty sure OSS gets a free ride on mirrors, because of peering agreements: if traffic between networks is equal, nobody has a case that they're the stronger party on negotiations, so no money changes hands. If there's an asymmetry, theres' an argument that one side has more demand or need for peering and use that as leverage to demand payment. So smaller ISPs that want to balance out a lopsided consumer traffic, will set up mirrors to protect themselves from this. Since OSS is free to redistribute, a loose federation of mirrors has set up to better match ISPs who want to send more bytes with projects that need to send a lot of bytes. > Are there any big downsides to distributing torrents as opposed to traditional FTP/HTTP Analytics is the big reason Mozilla shut down their community FTP mirrors, and bit torrent doesn't really solve that portion.
• pugworthy 5y ago
  Valve hired Bram Cohen back in 2004, and AFAIK it is/was core to Steam's distribution system.
• bbulkow 5y ago
  Bittorrent clients for windows and mac are generally malware infested pits of adware. Wherever i try to be virtuous and use one, i have to clean whatever i installed. I am sure i am not the only one. Why not build your own install using the bittorrent libraries? The security model is more complicated, for one. If i was an architect and proposed that, i would have to understand the attack vectors and present why they are than the well known solution. And taking control of the software update channel is a massive risk. Second, i have less control of the user experience. With http, i can pay for the right amount of bandwidth, or time on a cdn. If i implement bittorrent, i still have to have buy capacity, i just have a more complicated model for how much to buy. Suggested updates can be spread over time - and need to be, for canary purposes. I think Android often pushes an app update over 4 or 5 days, by default? Steady state infra capacity, or even better, low priority which and be interrupted, is cheap. Given the complexity and business risk (people can't download our software! Our binary got hijacked! Two code paths to test?) And the inexpensive nature of mirrors, and the competence of cdns, there are rarely causes where it would make sense.
  • Mandatum 5y ago
    Deluge, qBittorrent, Transmission, oh my!
  • AlchemistCamp 5y ago
    Blizzard has used BT for binary downloads for years. None of your fears came to pass.
  • alophawen 5y ago
    Linux distros/archive.org and other legit users of BitTorrent tend to use HTTP seeds [1], so you can still calculate for bandwidth costs. Commercial software like World of Warcraft used BitTorrent in their updater for years [2]. 1: https://www.bittorrent.org/beps/bep_0019.html https://www.bittorrent.org/beps/bep_0019.html 2: https://wowpedia.fandom.com/wiki/Blizzard_Downloader https://wowpedia.fandom.com/wiki/Blizzard_Downloader
• kjeetgill 5y ago
  A few companies I've worked at have used torrents to roll out new service binaries (multi-gb tarballs of Java jars) and periodic index deployments using torrents. The biggest advantage is just diffusing a thundering heard without increasing latency considerably. Shout-out to tTorrent.