21 ms·
I work with a slowly changing dataset that's about 100GB to download in full. A few people a week download it. I've considered adding a torrent download, becau
by Symbiote 8y ago
I work with a slowly changing dataset that's about 100GB to download in full. A few people a week download it.
I've considered adding a torrent download, because it includes built-in verification of the download. A common problem is users reporting that their download over HTTP is corrupt, but I'm not sure if they'd be able or want to use Bittorrent.
(Also, for many users the download is probably fine, but they can't open it in Excel. Bittorrent won't help that. )
- solarkraft 8y agoFor 100GB of important data I'd be willing to buy an extra hard drive. I'd expect your users to be willing to set up a torrent client. It's not even difficult.
- duckerude 8y agoIt's not that difficult, but it can be scary. The few times I've published files over bittorrent I've had to reassure people that torrenting itself isn't any more illegal than other download methods. It's also not clear ahead of time how difficult it's going to be. If torrenting is the only way, some people won't bother.
- Symbiote 8y agoThat's what I'm concerned with; tarnishing a good reputation with a "scary" word. Bandwidth is no problem for us, we have a faster connection than all the users. Users in Africa and Latin America would probably benefit most, but I'd need to research whether they'd be prepared to use Bittorrent before implementing this.
- bo1024 8y agoThe torrent download sounds like a great application for this -- hopefully it can take some strain off your servers as well. Linux distributions mostly do this. Only thing is how you want to deal with updates to the dataset.
- keypusher 8y agoYou might want to look into something like syncthing. It uses a distributed protocol like torrents but also supports updating files to new versions while only resyncing the diff and sharing folders. Open source, donation-funded. https://syncthing.net/ https://syncthing.net/