5 ms·
Torrents are available here: http://gen.lib.rus.ec/scimag/repository_torrent/ http://gen.lib.rus.ec/scimag/repository_torrent/ Apparently it was 55TB two years
by wongarsu 6y ago
Torrents are available here: http://gen.lib.rus.ec/scimag/repository_torrent/ http://gen.lib.rus.ec/scimag/repository_torrent/
Apparently it was 55TB two years ago. It's probably still somewhere in the realm of a smallish NAS and I bet a couple people on /r/datahoarders have it all downloaded.
Hosting is a different matter. I'm not sure if there's some effort to have it available on IPFS or DAT
- biggc 6y ago> 55 TB ... > smallish What a time to be alive
- wongarsu 6y ago55 TB fits comfortably on 4 16TB HDDs. You can fit around 8 drives in a single tabletop NAS before you have to go either to multiple enclosures or rack mounting. Using that as a metric I would call anything below 100TB smallish. Storage is indeed progressing a lot.
- generalizations 6y agoThat much storage would still cost somewhere around $1500-$2000. Impressively low, but still in the range of a very dedicated hobbyist.
- agumonkey 6y agoClearly .. tons of people spend more than that on gears of way less value (think GPUs for leisure purposes only). Have a tiny online donation setup and you could get full sponsorship or half easily.
- vbezhenar 6y agoI think that kind of donation exposure would open you to some legal risks. Now you're some kind of visible commercial entity, not just some random IP address in the tracker response.
- agumonkey 6y agoThese days people do fundraising for all sorts of reason and get money. One could always invent a good story to avoid explaining the real reason behind the gears.
- Pixionus 6y agoFor most of the lay users I help in the community 4tb is immense. For most of my colleagues (I work on Kubernetes cluster backbends for large companies running streaming services you likely use, among other 'big' data companies) 20-50tb is if you run your own all time backups (usually with ZFS and the like so divide that by 3 for total usable space). The guys that are running IPtv or other sketchy video streaming and scraping software are hovering around 100TB-200TB in their NAS and none of them actually run user grade hardware... I'm honestly very surprised to hear it referred to as 'smallish'.. At most I'd say 100TB is about middle of the road now days for media horders and small for media content producers. Now my buddy that worked on the CEPH storage backed at CERN would laugh at any of these numbers... but that's a different ballpark all together..
- pmiller2 6y agoYou can get more than 8 without going to external enclosures. I have a tower case with 15 drive bays in it.
- dzhiurgis 6y agoThere are 30/50/100 TB SSD’s that would fit your coat pocket
- snvzz 6y agoThere's progress indeed, but 4x 16TB isn't cutting it. You'd need extra disks for redundancy, extra space for filesystem overhead (including data and metadata checksums), and a sizable amount of ECC memory.
- vbezhenar 6y agoYou don't strictly need ECC memory for that specific task. Torrents have checksums for every file chunk, so even if some bit corrupted because of RAM, it's enough to re-check the files and re-download corrupted pieces if needed. That said, complete re-check of 64 TB at 250 MB/s would take at least 71 hour. So ECC RAM definitely won't hurt.
- jacquesm 6y agoIt's about double that now.
- TheRealSteel 6y agoThanks. So there's a monolithic torrent, but we still need to implement a way to search for and download one specific paper, in a decentralized way?
- __turbobrew__ 6y agoIsn’t that the torrents for libgen, not scihub? I’m not intimately familiar with the interactions between libgen and sci hub but the wiki page for sci hub says it hasn’t been using libgen as a storage backend for a few years now.
- tpxl 6y agoAre there any guidelines on what to download if you want to host? I could throw a couple hundred gigs of storage and a decent uplink at it, but if I download and never reach a good ratio I did more harm than good.