9 ms·

When Megaupload (and some other sharehosters) died, quite a lot of interesting things just disappeared from the net. I'm talking about things like small tools

When Megaupload (and some other sharehosters) died, quite a lot of interesting things just disappeared from the net. I'm talking about things like small tools that were shared on e.g. xda-developers before Github came, about fan-mods for games etc... The 'big' ones continued living, but if you now e.g. search for a special kernel/ROM for your G1/ADP1 you are mostly out of luck. It's sad that there's basically no way for an organisation like archive.org to archive things from sharehosters given the unclear (or quite clearly black/gray) law situation and also the missing cooperation with the sharehosters themselves.

throwawayaway 12y ago
torrents are a reasonable solution for sharing those kind of programs.
- JoshTriplett 12y ago
  Not really; nothing guarantees that someone keeps seeding them forever.
  - thisjepisje 12y ago
    "Reasonable", not "absolutely infallible".
    - vacri 12y ago
      Unseeded torrents are pretty common, especially for less popular items.
      - slang800 12y ago
        In my experience, unseeded torrents are less common than Rapidshare links that have been removed by their user or taken down after being reported. However, I've never gathered statistics on it, so this is purely anecdotal.
  - jsprogrammer 12y ago
    Nothing guarantees that anything keeps serving files forever. BT at least distributes responsibility and allows anyone to easily keep the files hosted.
    - ratsmack 12y ago
      I've often wondered if Freenet, or a similar encrypted network file system would be a viable solution. This, of course, requires a significant number of computers to contribute full time.
      - jsprogrammer 12y ago
        What's wrong with BT? It can operate encrypted and has the DHT to address/discover content.
      - wmf 12y ago
        AFAIK Freenet is basically a cache with no incentive system, so content will expire. There are some cryptocurrency-oriented storage projects but they all seem to be in way over their heads.
        lmm 12y ago
        I think the fundamental economics for a cryptocurrency approach don't lean towards preserving obscure things that don't interest many people. Storage cost is always going to be proportional to the amount of data being stored, but income is always going to be proportional to the number of people who want the data.
        pgeorgi 12y ago
        In Freenet content will expire unless it's accessed regularly. If you want to make sure your files are around, just poll them every other week (which may or may not be already supported by the software), which is a simple "proof of work" scheme: As long as you care, the files are reasonably safe to be around. When you (and everyone else) stops caring, they'll slowly disappear in favour of more popular stuff.
  - rakoo 12y ago
    I'd expect a team like xda-developers to run a seed of their content; it's the best way to make sure it is still available.
    - chii 12y ago
      the point is that a service like rapidshare doesn't cost the user any money (and instead, get ads). Running a torrent server is expensive, and require dedicated hardware too. You can't really seed from the webhost after all.
      - pgeorgi 12y ago
        You can seed from regular webhosts - as long as it's a regular http query to a regular http server, not some temporary http query hidden behind several layers of interstitial ads for the premium account.
      - rakoo 12y ago
        The user can use seedboxes, they're far from being as difficult to use as dedicated hardware. They sometimes have some free tier that I expect would be more than enough for a team like xda-developers (think 5GB). It's true that they wouldn't be able to earn money through this, though.
toomuchtodo 12y ago
I wish the Internet Archive had an embargo feature, where you could push data in and while it wouldn't be served, it would be stored until a later date when copyright issues could be worked out.
- saganus 12y ago
  This is an interesting idea. It would work kind of like an escrow, right? Where the "release/publish clause" could be tied to copyright expiration or other legal events like you say. The problem I see is, how do you filter/reject the amount of stuff you would have to store without even being public? AFAIK, the IA has issues storing so much of the already-public data, so storing "dark" (non-public/unpublished) stuff would potentially mean lots of cruft and garbage so to speak. However it does sound very interesting. I hope we can some day achieve truly permanent data storage systems were we could just dump all of this and not worry much about it again. Edit: Thinking about it a bit more, how feasible does the following sound: Anyone interested in helping the IA could buy a sort of Drobo/NAS that is able to store only IA stuff (ala Freenet). Everything is encrypted of course, and then only way to access the files is when the escrow trigger fires off, the private key is released at the IA archive and then every owner of the IA-box will have access to that particular part of the archive (as well as regular IA users through web). It's kind of like an HDD preloaded full of torrents, and then the differences or new additions can be streamed to your local IA-box as needed. You could even filter what kind of stuff would you like to help the IA archive. For example, I'm a big fan of movies so I prioritize that category (up to a certain % so that no one category is forgotten). Does anyone know if anything resembles this? I mean, I could very well leave a low-powered NAS to help the IA serve their content, store it for later use, etc. And I imagine (hope) that a lot of other people would too. It would be a way of donating electricity, space to a worthy cause.
  - btown 12y ago
    Would tape storage, a la Amazon Glacier, be an efficient way to archive these, since they don't need to be accessed except with an offline process?
stusmall 12y ago
>I'm talking about things like small tools that were shared on e.g. xda-developers before Github came, about fan-mods for games etc... The 'big' ones continued living, but if you now e.g. search for a special kernel/ROM for your G1/ADP1 you are mostly out of luck. This is something that has always sketched me out about the Android rooting and moding scene. "Download this suspect binary from rapid share and run it as root" seemed to be a cornerstone of it.
- yablak 12y ago
  This is generally taken care of using checksums, assuming you trust the source. Of course, trusting the source is the hard part. CyanogenMod, for example, has a chain of trust for their releases; though they host their own binaries.
  - comex 12y ago
    It is too bad that these went down, but yech, sharing binaries or even entire OS images made of open source code... what if you want to change one configuration option or line of code while keeping whatever special modifications were made? What if you want to combine the changes in X fancy ROM with Y? (Never mind that the kernel is GPL.) I'm not an Android user myself, but I will nevertheless suggest that optimally these things would be distributed as source on GitHub, with a deterministic build system guaranteed to be able to reproduce binaries in the future, and only secondarily as binaries.
    - ianlevesque 12y ago
      I agree but an Android build takes hours and hours and has a complex setup. Even a pretty dedicated Android user would prefer a binary.
      - pakled_engineer 12y ago
        I build them on AWS now, mainly since it takes me 5hrs to download the source compared to a few minutes on AWS. Anybody who knows of a better cloud building service? I consistently fail to estimate instance costs.
DanBC 12y ago
I have fond memories of Rapidshare. Even the cats.
- Rondom 12y ago
  Haha, just told a friend about those cat-captchas... For the ones that do not know what is meant: http://www.geeksaresexy.net/2008/04/24/rapidshare-captcha-will-drive-you-crazy/ http://www.geeksaresexy.net/2008/04/24/rapidshare-captcha-wi...
visarga 12y ago
When MP3.com was shut down, I shed a tear :-( It was an amazing collection predating the YouTube's of today.
- Kiro 12y ago
  Yeah, I feel you. Some of my favorite tunes were lost. I've yet to find them somewhere else.
- normloman 12y ago
  The second and third albums I bought in my life were from mp3.com. That site had a huge impact on where I am today (teaching music, playing in a band). The people who bought it had a responsibility to keep that music alive. It's a shame they didn't. Some of that music is lost forever.
thizzbuzz 12y ago
the same is true within the styles of music that I like. It was compounded by the fact the other centralized file hosts, like Rapidshare and Mediafire, were forced to delete TONS of content on the same day that Megaupload folded. There are many obscure demos, rehearsals, etc. that disappeared from the internet and most haven't reappeared since. I knew some bloggers who had uploaded probably in the range of 5000-10000 old metal demos, and these guys were careful to not post copyrighted material, but it seemed like if they got even a single strike against their account, everything was deleted. I hope someone imaged those servers, otherwise a lot of that content might be lost forever.
neotek 12y ago
I wonder if it's feasible to build a hosting site that strictly forbids (and actively removes) copyright infringing material, intended more for this kind of file hosting than for piracy. Monetising would be a challenge, since rapidshare et al mainly made money by selling premium accounts, which are generally only attractive for piracy purposes.
- wmf 12y ago
  If you want to share something legit, Dropbox/Google Drive/etc. seem to work fine and have working business models.
  - neotek 12y ago
    Both require registration and are far more than just file hosts. The thing I like about rapidshare and other file hosts is that you can just drop a file on it and a minute later get a link. No thinking or effort required, no activation emails (or endless onboarding emails for that matter), no passwords to remember.
- icebraining 12y ago
  There's no such thing as "copyright infringing material", it depends on how has uploaded it. If I download an awesome ROM from megaupload using the link shared by its creator and upload it myself to megaupload to get my own link, the very same file is both infringing and not infringing copyright. This is what people who were appalled that megaupload would only remove one of the links to a deduplicated file, instead of removing the file itself, failed (or refused) to understand.
morenoh149 12y ago
+1 give the internet archive your money guys. http://archive.org/donate/ http://archive.org/donate/
Juha 12y ago
I think it was never very optimal to share roms through megaupload, but it seemed to be the easiest option. I'm happy to see more rom devs starting to use the needrom.com. They have rom hosting as well as commenting/rating system, it's not perfect but at least they have lots of android roms available.

throwawayaway 12y ago

torrents are a reasonable solution for sharing those kind of programs.

JoshTriplett 12y ago
Not really; nothing guarantees that someone keeps seeding them forever.
- thisjepisje 12y ago
  "Reasonable", not "absolutely infallible".
  - vacri 12y ago
    Unseeded torrents are pretty common, especially for less popular items.
    - slang800 12y ago
      In my experience, unseeded torrents are less common than Rapidshare links that have been removed by their user or taken down after being reported. However, I've never gathered statistics on it, so this is purely anecdotal.
- jsprogrammer 12y ago
  Nothing guarantees that anything keeps serving files forever. BT at least distributes responsibility and allows anyone to easily keep the files hosted.
  - ratsmack 12y ago
    I've often wondered if Freenet, or a similar encrypted network file system would be a viable solution. This, of course, requires a significant number of computers to contribute full time.
    - jsprogrammer 12y ago
      What's wrong with BT? It can operate encrypted and has the DHT to address/discover content.
    - wmf 12y ago
      AFAIK Freenet is basically a cache with no incentive system, so content will expire. There are some cryptocurrency-oriented storage projects but they all seem to be in way over their heads.
      - lmm 12y ago
        I think the fundamental economics for a cryptocurrency approach don't lean towards preserving obscure things that don't interest many people. Storage cost is always going to be proportional to the amount of data being stored, but income is always going to be proportional to the number of people who want the data.
      - pgeorgi 12y ago
        In Freenet content will expire unless it's accessed regularly. If you want to make sure your files are around, just poll them every other week (which may or may not be already supported by the software), which is a simple "proof of work" scheme: As long as you care, the files are reasonably safe to be around. When you (and everyone else) stops caring, they'll slowly disappear in favour of more popular stuff.
- rakoo 12y ago
  I'd expect a team like xda-developers to run a seed of their content; it's the best way to make sure it is still available.
  - chii 12y ago
    the point is that a service like rapidshare doesn't cost the user any money (and instead, get ads). Running a torrent server is expensive, and require dedicated hardware too. You can't really seed from the webhost after all.
    - pgeorgi 12y ago
      You can seed from regular webhosts - as long as it's a regular http query to a regular http server, not some temporary http query hidden behind several layers of interstitial ads for the premium account.
    - rakoo 12y ago
      The user can use seedboxes, they're far from being as difficult to use as dedicated hardware. They sometimes have some free tier that I expect would be more than enough for a team like xda-developers (think 5GB). It's true that they wouldn't be able to earn money through this, though.

toomuchtodo 12y ago

I wish the Internet Archive had an embargo feature, where you could push data in and while it wouldn't be served, it would be stored until a later date when copyright issues could be worked out.

saganus 12y ago
This is an interesting idea. It would work kind of like an escrow, right? Where the "release/publish clause" could be tied to copyright expiration or other legal events like you say. The problem I see is, how do you filter/reject the amount of stuff you would have to store without even being public? AFAIK, the IA has issues storing so much of the already-public data, so storing "dark" (non-public/unpublished) stuff would potentially mean lots of cruft and garbage so to speak. However it does sound very interesting. I hope we can some day achieve truly permanent data storage systems were we could just dump all of this and not worry much about it again. Edit: Thinking about it a bit more, how feasible does the following sound: Anyone interested in helping the IA could buy a sort of Drobo/NAS that is able to store only IA stuff (ala Freenet). Everything is encrypted of course, and then only way to access the files is when the escrow trigger fires off, the private key is released at the IA archive and then every owner of the IA-box will have access to that particular part of the archive (as well as regular IA users through web). It's kind of like an HDD preloaded full of torrents, and then the differences or new additions can be streamed to your local IA-box as needed. You could even filter what kind of stuff would you like to help the IA archive. For example, I'm a big fan of movies so I prioritize that category (up to a certain % so that no one category is forgotten). Does anyone know if anything resembles this? I mean, I could very well leave a low-powered NAS to help the IA serve their content, store it for later use, etc. And I imagine (hope) that a lot of other people would too. It would be a way of donating electricity, space to a worthy cause.
- btown 12y ago
  Would tape storage, a la Amazon Glacier, be an efficient way to archive these, since they don't need to be accessed except with an offline process?

stusmall 12y ago

>I'm talking about things like small tools that were shared on e.g. xda-developers before Github came, about fan-mods for games etc... The 'big' ones continued living, but if you now e.g. search for a special kernel/ROM for your G1/ADP1 you are mostly out of luck. This is something that has always sketched me out about the Android rooting and moding scene. "Download this suspect binary from rapid share and run it as root" seemed to be a cornerstone of it.

yablak 12y ago
This is generally taken care of using checksums, assuming you trust the source. Of course, trusting the source is the hard part. CyanogenMod, for example, has a chain of trust for their releases; though they host their own binaries.
- comex 12y ago
  It is too bad that these went down, but yech, sharing binaries or even entire OS images made of open source code... what if you want to change one configuration option or line of code while keeping whatever special modifications were made? What if you want to combine the changes in X fancy ROM with Y? (Never mind that the kernel is GPL.) I'm not an Android user myself, but I will nevertheless suggest that optimally these things would be distributed as source on GitHub, with a deterministic build system guaranteed to be able to reproduce binaries in the future, and only secondarily as binaries.
  - ianlevesque 12y ago
    I agree but an Android build takes hours and hours and has a complex setup. Even a pretty dedicated Android user would prefer a binary.
    - pakled_engineer 12y ago
      I build them on AWS now, mainly since it takes me 5hrs to download the source compared to a few minutes on AWS. Anybody who knows of a better cloud building service? I consistently fail to estimate instance costs.

DanBC 12y ago

I have fond memories of Rapidshare. Even the cats.

Rondom 12y ago
Haha, just told a friend about those cat-captchas... For the ones that do not know what is meant: http://www.geeksaresexy.net/2008/04/24/rapidshare-captcha-will-drive-you-crazy/ http://www.geeksaresexy.net/2008/04/24/rapidshare-captcha-wi...

visarga 12y ago

When MP3.com was shut down, I shed a tear :-( It was an amazing collection predating the YouTube's of today.

Kiro 12y ago
Yeah, I feel you. Some of my favorite tunes were lost. I've yet to find them somewhere else.
normloman 12y ago
The second and third albums I bought in my life were from mp3.com. That site had a huge impact on where I am today (teaching music, playing in a band). The people who bought it had a responsibility to keep that music alive. It's a shame they didn't. Some of that music is lost forever.

thizzbuzz 12y ago

the same is true within the styles of music that I like. It was compounded by the fact the other centralized file hosts, like Rapidshare and Mediafire, were forced to delete TONS of content on the same day that Megaupload folded. There are many obscure demos, rehearsals, etc. that disappeared from the internet and most haven't reappeared since. I knew some bloggers who had uploaded probably in the range of 5000-10000 old metal demos, and these guys were careful to not post copyrighted material, but it seemed like if they got even a single strike against their account, everything was deleted. I hope someone imaged those servers, otherwise a lot of that content might be lost forever.

neotek 12y ago

I wonder if it's feasible to build a hosting site that strictly forbids (and actively removes) copyright infringing material, intended more for this kind of file hosting than for piracy. Monetising would be a challenge, since rapidshare et al mainly made money by selling premium accounts, which are generally only attractive for piracy purposes.

wmf 12y ago
If you want to share something legit, Dropbox/Google Drive/etc. seem to work fine and have working business models.
- neotek 12y ago
  Both require registration and are far more than just file hosts. The thing I like about rapidshare and other file hosts is that you can just drop a file on it and a minute later get a link. No thinking or effort required, no activation emails (or endless onboarding emails for that matter), no passwords to remember.
icebraining 12y ago
There's no such thing as "copyright infringing material", it depends on how has uploaded it. If I download an awesome ROM from megaupload using the link shared by its creator and upload it myself to megaupload to get my own link, the very same file is both infringing and not infringing copyright. This is what people who were appalled that megaupload would only remove one of the links to a deduplicated file, instead of removing the file itself, failed (or refused) to understand.

morenoh149 12y ago

+1 give the internet archive your money guys. http://archive.org/donate/ http://archive.org/donate/

Juha 12y ago

I think it was never very optimal to share roms through megaupload, but it seemed to be the easiest option. I'm happy to see more rom devs starting to use the needrom.com. They have rom hosting as well as commenting/rating system, it's not perfect but at least they have lots of android roms available.