4 ms·

I'm curious how Sci-hub's approach compares to the What.cd/Redacted approach. IIUC Sci-hub has scooped up science docs through a good enough UX that it was abl

I'm curious how Sci-hub's approach compares to the What.cd/Redacted approach. IIUC Sci-hub has scooped up science docs through a good enough UX that it was able to leverage the goodwill of science folks to upload docs (plus whatever other methods it has used to scoop up docs), and it uses a public blitzkrieg-style distribution mechanism. I.e., I guess if one had a big enough harddrive and a fast enough internet connection, one could start downloading the lib right now and see if they win the race against the copyright holders. On the other hand, the What.cd/Redacted approach seems to use Bittorrent ratios to create a private-tracker economy. New users get a few gigs free download on joining. But apparently because a) there's a 1:1 upload/download ratio, and b) a few first-mover fat cats are sitting on enormous ratios, this means there is a scramble by everyone else to upload new FLACs to build up their ratio so they can continue to be able to download FLACs. It seems that would mean the library-in-its-entirety cannot be easily replicated at will. Yet the tracker was apparently already nuked off the internet as What.cd and reappeared later as Redacted. Was any data lost between the two services? Oh yeah, there's also apparently another approach in rutracker, which seems to be blitzkrieg to add content and publish, at the (apparent?) cost of quality of content. It's really a shame that the nerdy, completist domain of digital archiving through torrents isn't covered by fair use. Perhaps we could exclude the most recent 10 years of music so that the hopeful young musician streamers can get paid a few hundred dollars for millions of streams and then receive the silver lining of fair use protection against a label refusing to release one of their albums.

soulofmischief 4y ago
Unfortunately, a lot of data is currently yet to be restored. Some of that music might not surface again for decades to come.
MontyCarloHall 4y ago
Sci-Hub was interesting because until 2021, it could automatically add any article not in its database by querying proxy servers set up at universities that actually subscribed to the journals, which would then download the PDF from the journal’s website and forward it to Sci-Hub. (This was the approach most academics took in the pre-Sci-Hub days; they’d email friends at other universities and ask them if they had access to a given article.) Sadly, Sci-Hub took down this “magic proxy” to try and win a court case in India, which its creator thinks might legitimize it elsewhere. It’s a huge shame, because it means that many obscure articles are now inaccessible via Sci-Hub. Ironically, a pure proxy-based Sci-Hub that didn’t host any articles on its own might actually be legal in certain countries, since it’s not actually hosting any copyrighted content itself. It definitely would be a lot easier to host and a lot harder to shut down; indeed, it could be completely decentralized.
- pdntspa 4y ago
  The Pirate Bay has never hosted any content; yet many, many cases have been decided against them. Why would that work for sci-hub?
  - genewitch 4y ago
    Courts can make dumb decisions, either by bias or bad arguments. Proper lawyer-ing can cover for bad decisions some of the time, and perhaps losing some case and winning others is more important. realistically everyone wants to know where they stand. There's certain "arranged" understandings when it comes to science (and all academic) publishing, and navigating that to provide a uniform interface to the actual documents is the "hard part" - although realistically, a file is a file, and distribution should be "easy". recently i needed to check what domains i had purchased through various domain name providers (namecheap, godaddy), and i nearly lost access to my godaddy account because of the TOTP/verification stages. I consider cellphone numbers ephemeral, as i consider all phone numbers. I've been told that a dozen numbers in twice as many years is too many, but i refuse to get led around by the nose by cellular providers. So TOTP only works if i have access to the - few - phone numbers i had access to when i signed up. Several services (including some mentioned) used to have voice TOTP, but no longer do, and use an SMS bridge. Some want me to have an app on a phone *with a(n) (e)SIM*. my email service is attached to a domain name that i very nearly lost access to management for. I was complaining about an old gmail address i had and i decided to look at my overall domain health and found it lacking. someone who is - forgive this - "balls deep" in a degree just wants whatever they publish to work for their advisors and graders. Obviously research assistants and research students in a masters or doctorate program are held to a slightly higher standard of "publish or perish" because of the way grants are written and assigned. It's all garbage, and has been for at least 15 years. sorry for my meandering, but i felt these two annoyances are related.
  - MontyCarloHall 4y ago
    I didn’t say that all countries would deem Sci-Hub legal, just that some might. Indeed, The Pirate Bay is perfectly legal in several countries.
- 4y ago
loeg 4y ago
> Yet the tracker was apparently already nuked off the internet as What.cd and reappeared later as Redacted. You imply there is some continuity in the operation between these two trackers, but I don’t believe that’s the case. What.cd shut down. Subsequently, redacted (passtheheadphones) and apollo started, appealing to the same userbase. Neither of those trackers were privy to what.cd’s databases.
- genewitch 4y ago
  in the sets of what.cd and waffles.fm and Redacted, approximate the intersection of the sets. I like music a lot. I have a lot of vinyl and weird CDs, too. However, i can't be assed to rip to whatever draconian style-guide some of these private trackers want. So it's a matter of being a member of several servers and finding something that either isn't listed or seeded and "filling" or creating a torrent with some other tracker's set of files. this is rewarding the wrong behavior. If i remember some song i heard in 1996, i should just be able to get it. It would be nice if all of the people who were involved in the creation and publication of the song got rewarded, somehow, but that's just not how art works in capitalism. I say this as someone who has personally released 11 CDs and a further 6 CDs in collaboration, of music. I haven't been paid a penny or more for anything i've ever produced in "art". I don't consider this a downside. People who know me and know i write music appreciate my music. People who don't know me will miss out. That's all there is to it.
jumelles 4y ago
> Was any data lost between the two services? Undoubtedly.
hatware 4y ago
Bounty systems are a pretty common way to get insane upload ratios so that you can archive.
- throwhn0000001 4y ago
  >insane upload ratios not with most RED bounties you can fill. if you only sort by biggest bounties, you get albums that realistically can't be filled, or that would require serious money and effort to find.. rare asia specific releeases and stuff. you can do specific requests if you have accounts for streaming platforms but nobody makes bounty requests for those
luckylion 4y ago
> Was any data lost between the two services? Yes, absolutely, a lot. Even though some people have automated their setup very well and have been downloading (and uploading on the newer trackers) a lot, it's just a giant amount of content, it's unlikely for any one person to have it all, and coordination was very limited back then. The birthday release of the torrent db of what.cd includes 2.6m torrents in 1.2m groups (aka individual releases), in total weighing in at 588TB (or 421TB is you discard mp3, but there was content that hadn't been available in FLAC). That's doable on an SWE salary today if you're dedicated, but what.cd was shut down in 2016, and you'd still need to deal with the ratio system during collection.
- throwhn0000001 4y ago
  after wcd fell some private sftp and dc+ servers came up for about 100 of the top seeding and hoarding members who were all familiar with each other. people regularly shared new content and filled out mussing music in their collections. i dont think it exists anymore but it might. i did not have access but knew several people that did who got me content to complete music sets the combined amount of content on those servers was probably around 100TB but most likely more
  - luckylion 4y ago
    Oh, certainly, with some coordination there's a lot you can achieve, and it would probably be much more than 100TB. A friend of mine had barely above 1TB seeding and iirc that was somewhere in the top 450-500, so top 100 probably had much more. It's still a lot to store (and maintain) individually, so unless they're still going with that private exchange, each tracker end could mean someone not migrating to the next installment and their part of the library being lost. The Eye is doing a lot of public-ish work in that area (seen with very mixed feelings in the tracker scene, because apparently they're not just archivists but also jerks).
throwhn0000001 4y ago
Redacted's ratio economy is fucked. There's groups of people using high speed seedboxes that grab every new upload to build up ratio in a normal way, if you can call it that. Everyone else is left with scraps, trickling data out to whoever comes later. Or maybe you get lucky and you're the lone seeder, you get a 1:1 copy. You just better hope that was on a 24-Bit FLAC release, given how big they can get. There's many many threads on https://old.reddit.com/r/trackers https://old.reddit.com/r/trackers about REDs economy problems. OPS has a bonus point system and suffers less, but has worse content than RED. My opinion only though rutracker is all about putting content out there. A lot of stuff is mislabelled or lower quality, less retention, identified wrong, seeded slowly. But you can sure find a lot of oddities there. > was any data lost yes, absolutely. I've hundreds of albums not on RED, and I can't bother to reupload them. it's a total waste of time when you know they're unseeded in a week if not for me hanging on. time better spent finding new music and sharing everything on soulseek instead.
- relaxing 4y ago
  What saves the economy is the generous freeleech and gift token events. (Currently one running if you haven’t logged in this week.) It all serves the purpose of getting users to be good citizens participating in the community, and not just snatch and run.
  - orbital-decay 4y ago
    In theory. In practice, most just can't be bothered using private trackers because of this. Hard to get into, hard to keep up - you have to be a dedicated archivist just to use them. Which is the polar opposite of why people have been torrenting in the past - piracy had less friction than using licensed content. Rutracker works differently, they aren't afraid of snatch-and-run. They managed to build a community out of those dedicated archivists. So the volunteers keep the rare content afloat, and the tracker can also be open and usable by ordinary people as well (which keeps popular content afloat). This approach also has its flaws, of course.
  - margalabargala 4y ago
    > generous freeleech and gift token events That's what saved the what.cd economy, which had generous freeleech and gift token events. RED does it differently, and has stingy and ineffective freeleech and gift token events. On wcd, staff picks were freeleech, and that ensured people were able to download and seed, and there was free liquidity. RED's lack of large freeleech events has meant that most people on the site don't have much buffer, and can't just download what they like. It's functionally a forced recession due to lack of liquidity. Basically an internet version of this: https://yathartharora.substack.com/p/53-the-babysitting-recession https://yathartharora.substack.com/p/53-the-babysitting-rece...
eimrine 4y ago
> What.cd/Redacted approach seems to use Bittorrent ratios to create a private-tracker economy. New users get a few gigs free download on joining. Isn't that "bittorrent ratio" easy to cheat? I remember a good old times where I had to download some popular files just for ratio, then (beginning of 10's everybody starts to cheat and some of the biggest trackers turned to forever free leach.
- ShowalkKama 4y ago
  >Isn't that "bittorrent ratio" easy to cheat? Yes[0] but you have to be careful else you might get caught 0: https://wiki.installgentoo.com/wiki/Ghostleech https://wiki.installgentoo.com/wiki/Ghostleech
- 29083011397778 4y ago
  Other users also report their upload and download. Ergo, if you say you uploaded 10gig, it would show up in other users' download. This is tracked and checked, and you'll be kicked if you try to play the system this way.

soulofmischief 4y ago

Unfortunately, a lot of data is currently yet to be restored. Some of that music might not surface again for decades to come.

MontyCarloHall 4y ago

Sci-Hub was interesting because until 2021, it could automatically add any article not in its database by querying proxy servers set up at universities that actually subscribed to the journals, which would then download the PDF from the journal’s website and forward it to Sci-Hub. (This was the approach most academics took in the pre-Sci-Hub days; they’d email friends at other universities and ask them if they had access to a given article.) Sadly, Sci-Hub took down this “magic proxy” to try and win a court case in India, which its creator thinks might legitimize it elsewhere. It’s a huge shame, because it means that many obscure articles are now inaccessible via Sci-Hub. Ironically, a pure proxy-based Sci-Hub that didn’t host any articles on its own might actually be legal in certain countries, since it’s not actually hosting any copyrighted content itself. It definitely would be a lot easier to host and a lot harder to shut down; indeed, it could be completely decentralized.

pdntspa 4y ago
The Pirate Bay has never hosted any content; yet many, many cases have been decided against them. Why would that work for sci-hub?
- genewitch 4y ago
  Courts can make dumb decisions, either by bias or bad arguments. Proper lawyer-ing can cover for bad decisions some of the time, and perhaps losing some case and winning others is more important. realistically everyone wants to know where they stand. There's certain "arranged" understandings when it comes to science (and all academic) publishing, and navigating that to provide a uniform interface to the actual documents is the "hard part" - although realistically, a file is a file, and distribution should be "easy". recently i needed to check what domains i had purchased through various domain name providers (namecheap, godaddy), and i nearly lost access to my godaddy account because of the TOTP/verification stages. I consider cellphone numbers ephemeral, as i consider all phone numbers. I've been told that a dozen numbers in twice as many years is too many, but i refuse to get led around by the nose by cellular providers. So TOTP only works if i have access to the - few - phone numbers i had access to when i signed up. Several services (including some mentioned) used to have voice TOTP, but no longer do, and use an SMS bridge. Some want me to have an app on a phone *with a(n) (e)SIM*. my email service is attached to a domain name that i very nearly lost access to management for. I was complaining about an old gmail address i had and i decided to look at my overall domain health and found it lacking. someone who is - forgive this - "balls deep" in a degree just wants whatever they publish to work for their advisors and graders. Obviously research assistants and research students in a masters or doctorate program are held to a slightly higher standard of "publish or perish" because of the way grants are written and assigned. It's all garbage, and has been for at least 15 years. sorry for my meandering, but i felt these two annoyances are related.
- MontyCarloHall 4y ago
  I didn’t say that all countries would deem Sci-Hub legal, just that some might. Indeed, The Pirate Bay is perfectly legal in several countries.
4y ago

loeg 4y ago

> Yet the tracker was apparently already nuked off the internet as What.cd and reappeared later as Redacted. You imply there is some continuity in the operation between these two trackers, but I don’t believe that’s the case. What.cd shut down. Subsequently, redacted (passtheheadphones) and apollo started, appealing to the same userbase. Neither of those trackers were privy to what.cd’s databases.

genewitch 4y ago
in the sets of what.cd and waffles.fm and Redacted, approximate the intersection of the sets. I like music a lot. I have a lot of vinyl and weird CDs, too. However, i can't be assed to rip to whatever draconian style-guide some of these private trackers want. So it's a matter of being a member of several servers and finding something that either isn't listed or seeded and "filling" or creating a torrent with some other tracker's set of files. this is rewarding the wrong behavior. If i remember some song i heard in 1996, i should just be able to get it. It would be nice if all of the people who were involved in the creation and publication of the song got rewarded, somehow, but that's just not how art works in capitalism. I say this as someone who has personally released 11 CDs and a further 6 CDs in collaboration, of music. I haven't been paid a penny or more for anything i've ever produced in "art". I don't consider this a downside. People who know me and know i write music appreciate my music. People who don't know me will miss out. That's all there is to it.

jumelles 4y ago

> Was any data lost between the two services? Undoubtedly.

hatware 4y ago

Bounty systems are a pretty common way to get insane upload ratios so that you can archive.

throwhn0000001 4y ago
>insane upload ratios not with most RED bounties you can fill. if you only sort by biggest bounties, you get albums that realistically can't be filled, or that would require serious money and effort to find.. rare asia specific releeases and stuff. you can do specific requests if you have accounts for streaming platforms but nobody makes bounty requests for those

luckylion 4y ago

> Was any data lost between the two services? Yes, absolutely, a lot. Even though some people have automated their setup very well and have been downloading (and uploading on the newer trackers) a lot, it's just a giant amount of content, it's unlikely for any one person to have it all, and coordination was very limited back then. The birthday release of the torrent db of what.cd includes 2.6m torrents in 1.2m groups (aka individual releases), in total weighing in at 588TB (or 421TB is you discard mp3, but there was content that hadn't been available in FLAC). That's doable on an SWE salary today if you're dedicated, but what.cd was shut down in 2016, and you'd still need to deal with the ratio system during collection.

throwhn0000001 4y ago
after wcd fell some private sftp and dc+ servers came up for about 100 of the top seeding and hoarding members who were all familiar with each other. people regularly shared new content and filled out mussing music in their collections. i dont think it exists anymore but it might. i did not have access but knew several people that did who got me content to complete music sets the combined amount of content on those servers was probably around 100TB but most likely more
- luckylion 4y ago
  Oh, certainly, with some coordination there's a lot you can achieve, and it would probably be much more than 100TB. A friend of mine had barely above 1TB seeding and iirc that was somewhere in the top 450-500, so top 100 probably had much more. It's still a lot to store (and maintain) individually, so unless they're still going with that private exchange, each tracker end could mean someone not migrating to the next installment and their part of the library being lost. The Eye is doing a lot of public-ish work in that area (seen with very mixed feelings in the tracker scene, because apparently they're not just archivists but also jerks).

throwhn0000001 4y ago

Redacted's ratio economy is fucked. There's groups of people using high speed seedboxes that grab every new upload to build up ratio in a normal way, if you can call it that. Everyone else is left with scraps, trickling data out to whoever comes later. Or maybe you get lucky and you're the lone seeder, you get a 1:1 copy. You just better hope that was on a 24-Bit FLAC release, given how big they can get. There's many many threads on https://old.reddit.com/r/trackers https://old.reddit.com/r/trackers about REDs economy problems. OPS has a bonus point system and suffers less, but has worse content than RED. My opinion only though rutracker is all about putting content out there. A lot of stuff is mislabelled or lower quality, less retention, identified wrong, seeded slowly. But you can sure find a lot of oddities there. > was any data lost yes, absolutely. I've hundreds of albums not on RED, and I can't bother to reupload them. it's a total waste of time when you know they're unseeded in a week if not for me hanging on. time better spent finding new music and sharing everything on soulseek instead.

relaxing 4y ago
What saves the economy is the generous freeleech and gift token events. (Currently one running if you haven’t logged in this week.) It all serves the purpose of getting users to be good citizens participating in the community, and not just snatch and run.
- orbital-decay 4y ago
  In theory. In practice, most just can't be bothered using private trackers because of this. Hard to get into, hard to keep up - you have to be a dedicated archivist just to use them. Which is the polar opposite of why people have been torrenting in the past - piracy had less friction than using licensed content. Rutracker works differently, they aren't afraid of snatch-and-run. They managed to build a community out of those dedicated archivists. So the volunteers keep the rare content afloat, and the tracker can also be open and usable by ordinary people as well (which keeps popular content afloat). This approach also has its flaws, of course.
- margalabargala 4y ago
  > generous freeleech and gift token events That's what saved the what.cd economy, which had generous freeleech and gift token events. RED does it differently, and has stingy and ineffective freeleech and gift token events. On wcd, staff picks were freeleech, and that ensured people were able to download and seed, and there was free liquidity. RED's lack of large freeleech events has meant that most people on the site don't have much buffer, and can't just download what they like. It's functionally a forced recession due to lack of liquidity. Basically an internet version of this: https://yathartharora.substack.com/p/53-the-babysitting-recession https://yathartharora.substack.com/p/53-the-babysitting-rece...

eimrine 4y ago

> What.cd/Redacted approach seems to use Bittorrent ratios to create a private-tracker economy. New users get a few gigs free download on joining. Isn't that "bittorrent ratio" easy to cheat? I remember a good old times where I had to download some popular files just for ratio, then (beginning of 10's everybody starts to cheat and some of the biggest trackers turned to forever free leach.

ShowalkKama 4y ago
>Isn't that "bittorrent ratio" easy to cheat? Yes[0] but you have to be careful else you might get caught 0: https://wiki.installgentoo.com/wiki/Ghostleech https://wiki.installgentoo.com/wiki/Ghostleech
29083011397778 4y ago
Other users also report their upload and download. Ergo, if you say you uploaded 10gig, it would show up in other users' download. This is tracked and checked, and you'll be kicked if you try to play the system this way.