5 ms·

Anyone remember Napster? There were extremely obscure musical recordings that were instantly found on Napster that are impossible to find even today. It took a

Anyone remember Napster? There were extremely obscure musical recordings that were instantly found on Napster that are impossible to find even today. It took a long time for anyone to rebuild that data to the level of usefulness that Napster had. If Sci-Hub is forced to shut down due to legal pressure, technical problems, running out of money, or the founder losing interest, it would take a long time for anyone to duplicate Sci-Hub if they started from zero. I think a good way to keep the cause alive is to backup or mirror the 85,483,811 papers hosted on Sci-Hub. That is, if she's willing to allow it (maybe it's already being done, but how widely?). In the case of Sci-Hub, you'd need between 85 and 850 TB for each backup or mirror depending on whether the average paper size is 1 MB or 10 MB.

hobom 5y ago
I think there was some reddit data hoarder initiative that did that a few months ago.
vb6sp6 5y ago
Napster was ok. Oink on the other hand...
- Cyph0n 5y ago
  Or its spiritual successor, What.cd
  - omega3 5y ago
    The closure of what.cd should be up there with the burning of the library of Alexandria.
twobitshifter 5y ago
Isn’t there a torrent available? I thought I had heard that scihub is immortal because of torrented copies.
- deleted 5y ago
  [deleted]
- mkl 5y ago
  Only the existing bits. New publications wouldn't be included, so eventually the archive would become less relevant and stop being shared. There's significant cost for the required storage (currently 77TB according to andyxor's comment).
  - jacquesm 5y ago
    108T 104T 3,9T 97% /home/scihub
  - qiqitori 5y ago
    77TB (or 104 TB) sounds expensive, but many people reading this could buy that kind of storage with just one month's salary. (SeaGate IronWolf Pro 16 TB, currently $521 on Amazon)
    - capableweb 5y ago
      77 / 16 = 5.5 (6 disks) at $521 per disk turns into $3126. Seems only 11 countries in the world has a higher salary than $3126[1], and probably people wouldn't buy something like that with their entire monthly salary, so at least $5000 a month is required to be able to just dispose $3126 for some drives. That leaves 5 countries with a average salary higher than $5000. Granted, the source might not be the best, nor using average salaries as a signal either, but to say that many people can spend something like that on a whim seems ignorant at best. - [1] https://www.worlddata.info/average-income.php https://www.worlddata.info/average-income.php
      - dkarp 5y ago
        They said "many people reading this" and HN's biggest audience is US tech workers. It wouldn't be ignorant to say that many US tech workers can afford that.
      - 5y ago
twobitshifter 5y ago
Yes p2p really had the largest catalog and still surpasses what’s available on Spotify or Apple Music. You could get obscure live recordings, the smallest bands, everything was there. If you consider bootleg recordings, the volume of music produced is many times larger than what any one centralized service could offer.
andyxor 5y ago
There are 850 scihub torrents, each containing 100,000 scientific articles, to a total of 85M articles (77TB) http://libgen.rs/scimag/repository_torrent/ http://libgen.rs/scimag/repository_torrent/ index for all the papers in these torrents (SQL db): https://libgen.rs/dbdumps/scimag.sql.gz https://libgen.rs/dbdumps/scimag.sql.gz also see https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_mission_for_scihub_and_open_science_we_are/ https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_...
- wolverine876 5y ago
  But are they searchable, by users with little technical ability? Will they be updated with new papers if Sci-Hub shuts down?
  - pjerem 5y ago
    Nope, but at least, they aren’t lost. They are just there, ready to be made available by the potential successor of Sci-Hub. That’s what matters.
  - aabaker99 5y ago
    It's not too hard to find the article's doi (programatically) for newer articles. Once you have the doi you have all the important article metadata like author, publish date, etc. I imagine this is all that's required to make the articles searchable. However, this is certainly more challenging for older articles.
nxpnsv 5y ago
For me soulseek was amazing to find obscure recordings.
nvarsj 5y ago
> I think a good way to keep the cause alive is to backup or mirror the 85,483,811 papers hosted on Sci-Hub. There's already an effort where the data is broken up into 100GB chunks [1]. Looks like the entire database is duplicated now at least 10 times. [2] > Anyone remember Napster? There were extremely obscure musical recordings that were instantly found on Napster that are impossible to find even today. That existed (and still exists) - there was oink.cd, then what.cd, and now RED. But each incarnation has lost things. Each time you burn down the library, you're bound to lose something. It's disappointing there isn't a legal way of cultural preservation. We need something like a crowd sourced digital library. [1] https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_mission_for_scihub_and_open_science_we_are/ https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_... [2] https://phillm.net/torrent-health-frontend/stats-filtered-table.php?propname%5B%5D=seeders&comp%5B%5D=%3C&value%5B%5D=12&propname%5B%5D=type&comp%5B%5D===&value%5B%5D=scimag https://phillm.net/torrent-health-frontend/stats-filtered-ta...
efreak 5y ago
As far as backups for future generations go, I'd hope archive.org has a backup of it all, regardless of whether or not they make it available. I certainly hope they have a program for keeping copyrighted (or otherwise private) data backed up for safekeeping for future generations without making it publicly (or privately) accessible; sort of a digital time capsule, if you will, that's kept securely locked until a future date.

hobom 5y ago

I think there was some reddit data hoarder initiative that did that a few months ago.

vb6sp6 5y ago

Napster was ok. Oink on the other hand...

Cyph0n 5y ago
Or its spiritual successor, What.cd
- omega3 5y ago
  The closure of what.cd should be up there with the burning of the library of Alexandria.

twobitshifter 5y ago

Isn’t there a torrent available? I thought I had heard that scihub is immortal because of torrented copies.

deleted 5y ago
[deleted]
mkl 5y ago
Only the existing bits. New publications wouldn't be included, so eventually the archive would become less relevant and stop being shared. There's significant cost for the required storage (currently 77TB according to andyxor's comment).
- jacquesm 5y ago
  108T 104T 3,9T 97% /home/scihub
- qiqitori 5y ago
  77TB (or 104 TB) sounds expensive, but many people reading this could buy that kind of storage with just one month's salary. (SeaGate IronWolf Pro 16 TB, currently $521 on Amazon)
  - capableweb 5y ago
    77 / 16 = 5.5 (6 disks) at $521 per disk turns into $3126. Seems only 11 countries in the world has a higher salary than $3126[1], and probably people wouldn't buy something like that with their entire monthly salary, so at least $5000 a month is required to be able to just dispose $3126 for some drives. That leaves 5 countries with a average salary higher than $5000. Granted, the source might not be the best, nor using average salaries as a signal either, but to say that many people can spend something like that on a whim seems ignorant at best. - [1] https://www.worlddata.info/average-income.php https://www.worlddata.info/average-income.php
    - dkarp 5y ago
      They said "many people reading this" and HN's biggest audience is US tech workers. It wouldn't be ignorant to say that many US tech workers can afford that.
    - 5y ago

twobitshifter 5y ago

Yes p2p really had the largest catalog and still surpasses what’s available on Spotify or Apple Music. You could get obscure live recordings, the smallest bands, everything was there. If you consider bootleg recordings, the volume of music produced is many times larger than what any one centralized service could offer.

andyxor 5y ago

There are 850 scihub torrents, each containing 100,000 scientific articles, to a total of 85M articles (77TB) http://libgen.rs/scimag/repository_torrent/ http://libgen.rs/scimag/repository_torrent/ index for all the papers in these torrents (SQL db): https://libgen.rs/dbdumps/scimag.sql.gz https://libgen.rs/dbdumps/scimag.sql.gz also see https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_mission_for_scihub_and_open_science_we_are/ https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_...

wolverine876 5y ago
But are they searchable, by users with little technical ability? Will they be updated with new papers if Sci-Hub shuts down?
- pjerem 5y ago
  Nope, but at least, they aren’t lost. They are just there, ready to be made available by the potential successor of Sci-Hub. That’s what matters.
- aabaker99 5y ago
  It's not too hard to find the article's doi (programatically) for newer articles. Once you have the doi you have all the important article metadata like author, publish date, etc. I imagine this is all that's required to make the articles searchable. However, this is certainly more challenging for older articles.

nxpnsv 5y ago

For me soulseek was amazing to find obscure recordings.

nvarsj 5y ago

> I think a good way to keep the cause alive is to backup or mirror the 85,483,811 papers hosted on Sci-Hub. There's already an effort where the data is broken up into 100GB chunks [1]. Looks like the entire database is duplicated now at least 10 times. [2] > Anyone remember Napster? There were extremely obscure musical recordings that were instantly found on Napster that are impossible to find even today. That existed (and still exists) - there was oink.cd, then what.cd, and now RED. But each incarnation has lost things. Each time you burn down the library, you're bound to lose something. It's disappointing there isn't a legal way of cultural preservation. We need something like a crowd sourced digital library. [1] https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_mission_for_scihub_and_open_science_we_are/ https://www.reddit.com/r/DataHoarder/comments/nc27fv/rescue_... [2] https://phillm.net/torrent-health-frontend/stats-filtered-table.php?propname%5B%5D=seeders&comp%5B%5D=%3C&value%5B%5D=12&propname%5B%5D=type&comp%5B%5D===&value%5B%5D=scimag https://phillm.net/torrent-health-frontend/stats-filtered-ta...

efreak 5y ago

As far as backups for future generations go, I'd hope archive.org has a backup of it all, regardless of whether or not they make it available. I certainly hope they have a program for keeping copyrighted (or otherwise private) data backed up for safekeeping for future generations without making it publicly (or privately) accessible; sort of a digital time capsule, if you will, that's kept securely locked until a future date.