7 ms·

I imagine at least somebody will download to a server and host them all there. Might grab a new 1TB drive into one of my servers and do it if I'm bored enough..

by corin_ 16y ago

I imagine at least somebody will download to a server and host them all there. Might grab a new 1TB drive into one of my servers and do it if I'm bored enough...

jacquesm 16y ago
I've been doing that for the last 12 months, and the collection is significantly larger than that. http://www.reocities.com/ http://www.reocities.com/
- bobf 16y ago
  Great idea and name!
- pavel_lishin 16y ago
  Have you grabbed the torrent they're offering?
  - jacquesm 16y ago
    I would be very surprised if there is anything in there that is not in reocities yet but I will certainly do a comparison. I'll have to run that against all the deletion requests as well for this content because chances are fairly large that lots of it has already been removed at the request of the owners.
    - a1k0n 16y ago
      Have you tried de-duping files to save space? I imagine there's a finite number of MIDI files and "under construction" GIFs that could just be symlinked to save a ton of space...
      - jacquesm 16y ago
        Space is not the problem, I've got plenty of that. The bigger problem is that the underlying filesystem is having some problems dealing with the total number of files. Total disk space usage on the array that holds reocities is about 10.5 T (that includes a master copy though) and runs to 233 million files.
        bobf 16y ago
        Have you considered implementing a NoSQL solution to store the files? I've successfully done something similar with HBase.
        jacquesm 16y ago
        No, but that's a good idea, I'll definitely look in to that when the next round of reocities improvements rolls around.
  - jacquesm 16y ago
    On second reading, I don't actually see a torrent. Did I miss something?
- arst 16y ago
  While I think the web is probably a better place with reocities than without it, slapping ads on this content feels a tad bit slimy to me. I'm curious to hear your take on the ethical and legal implications of it.
  - jacquesm 16y ago
    The ads are actually a boost for a HN'er. And let's suppose for a second that they were not (as they have not been in the past), reocities has cost a fairly large amount of money to date (instead of made money, as you suggest), not a single person that has asked me to remove the content has ever commented on the presence of the ads, and neither has anybody that has found their stuff again because I backed it up. On the contrary, the reactions have been almost 100% positive with a very few exceptions.
    - arst 16y ago
      (instead of made money, as you suggest) I apologize if that's how you read my post; I certainly didn't intend to suggest that or to belittle the amount of work and money that you've put into this project.
      - jacquesm 16y ago
        If you put up half of the money that went in to it sight unseen then we'll call it even , deal ? ;)
    - pbhjpbhj 16y ago
      This is from the Google stable of morals - if people don't know you're profiting from their copyright then it's OK. Presumably you have no legal right to redistribute any of the Geocities stuff? The fact you spent time and money on it is neither here nor there, that's not a moral|legal argument. Presumably there have been some exceptions (you intimate that), what sort of percentage does that amount to? If you calculate that as a true reflection of the whole population then how many people's copyright do you suppose you've infringed, knowingly, against their wishes? If someone takes your published material and slaps ads on it and republishes without your knowledge is that all good with you? (I'm guessing you may say yes here!). It's not just about economics, it's probably as much about moral rights. People believe that their Geocities content died and was buried. Of course they are pretty minor "offences" (or at least appears so) across a large population - akin to being a spammer or somesuch. Actually scratch that this is simply like copying others blog posts on a massive scale.
- vyrotek 16y ago
  Ah ha! So here's where you've all been hiding. http://www.reocities.com/TimesSquare/1266/ http://www.reocities.com/TimesSquare/1266/
  - samatman 16y ago
    That page is by a young 'hacker' named Simon Liu. I wonder if it is this Simon Liu: http://www.computer.org/portal/web/csdl/doi/10.1109/MITP.2009.109 http://www.computer.org/portal/web/csdl/doi/10.1109/MITP.200... "Surviving Distributed Denial of Service Attacks" See? This is why reocities is awesome. The Simon Lius of the world should have access to their old Descent scores.
- mhd 16y ago
  And after only two random clicks, I'm at a (bad) foot fetish site. Well, it was either that, or a "Mr. T ate my <censored>"…
- brc 16y ago
  I checked, my very first website is thus far not there. I'll wait until you're finished, and if it doesn't appear I can send the offline backup. I've been carting it around for 10 years, laughing at the animated graphics and frames.
- a_m_kelly 16y ago
  I'd love to know what sort of infrastructure you're running this on. I'm in a course for my Masters for library school that deals with similar sorts of problems in maintaining and preserving for the "long term" digital materials. From your site it sounds like you wrote a script using wget to harvest the files and another to check them against versions that were still up. What do you do on the server end now to ensure that the files are still working correctly? Are you running periodic checksums on them or the like? Finally, are you looking for any help from an interested novice?
  - jacquesm 16y ago
    I have a large database table that stores the md5 hashes of all the files and there is a script that can compare all of the contents of the site with the hashes in the files (and with a second copy if that's what it would come to). Some bitrot is inevitable but I think it's under control for now. As for help, yes, but right now I'm pretty swamped in other stuff, the next round of work on reocities will likely come after the new year.
    - daeken 16y ago
      Have you considered using something like MogileFS? It'd be perfect for this sort of situation. Let me know if you're interested in this or have any questions -- I've dealt a good bit with systems like this in the past, and would love to give you a hand.
      - jacquesm 16y ago
        You know where to find me :) And yes, of course I'm interested. But right now no time.
- afterburner 16y ago
  It is truly amazing to see one of my old sites reappear in this form. Also, there appears to be another mirror, oocities.com
- flipbrad 16y ago
  are you worried about copyright infringement suits?
  - jacquesm 16y ago
    Not one bit. It was a situation where the common good outweighed the benefits of respecting copyright law handily, besides that the only thing that changed is the machine where the content was hosted. I do not pass it off as my own, have a very clearly established procedure for removing the data at the request of the copyright holder. Think of it as a hosting provider going out of business and a new one taking over the corpse of the old one. That it happened without contacting the owners is simply because the vast majority of the owners does not have contact information to begin with. See the reocities faq for some more info on this: http://www.reocities.com/newhome/faq.html http://www.reocities.com/newhome/faq.html

by corin_ 16y ago

I imagine at least somebody will download to a server and host them all there. Might grab a new 1TB drive into one of my servers and do it if I'm bored enough...

jacquesm 16y ago

I've been doing that for the last 12 months, and the collection is significantly larger than that. http://www.reocities.com/ http://www.reocities.com/

bobf 16y ago
Great idea and name!
pavel_lishin 16y ago
Have you grabbed the torrent they're offering?
- jacquesm 16y ago
  I would be very surprised if there is anything in there that is not in reocities yet but I will certainly do a comparison. I'll have to run that against all the deletion requests as well for this content because chances are fairly large that lots of it has already been removed at the request of the owners.
  - a1k0n 16y ago
    Have you tried de-duping files to save space? I imagine there's a finite number of MIDI files and "under construction" GIFs that could just be symlinked to save a ton of space...
    - jacquesm 16y ago
      Space is not the problem, I've got plenty of that. The bigger problem is that the underlying filesystem is having some problems dealing with the total number of files. Total disk space usage on the array that holds reocities is about 10.5 T (that includes a master copy though) and runs to 233 million files.
      - bobf 16y ago
        Have you considered implementing a NoSQL solution to store the files? I've successfully done something similar with HBase.
        jacquesm 16y ago
        No, but that's a good idea, I'll definitely look in to that when the next round of reocities improvements rolls around.
- jacquesm 16y ago
  On second reading, I don't actually see a torrent. Did I miss something?
arst 16y ago
While I think the web is probably a better place with reocities than without it, slapping ads on this content feels a tad bit slimy to me. I'm curious to hear your take on the ethical and legal implications of it.
- jacquesm 16y ago
  The ads are actually a boost for a HN'er. And let's suppose for a second that they were not (as they have not been in the past), reocities has cost a fairly large amount of money to date (instead of made money, as you suggest), not a single person that has asked me to remove the content has ever commented on the presence of the ads, and neither has anybody that has found their stuff again because I backed it up. On the contrary, the reactions have been almost 100% positive with a very few exceptions.
  - arst 16y ago
    (instead of made money, as you suggest) I apologize if that's how you read my post; I certainly didn't intend to suggest that or to belittle the amount of work and money that you've put into this project.
    - jacquesm 16y ago
      If you put up half of the money that went in to it sight unseen then we'll call it even , deal ? ;)
  - pbhjpbhj 16y ago
    This is from the Google stable of morals - if people don't know you're profiting from their copyright then it's OK. Presumably you have no legal right to redistribute any of the Geocities stuff? The fact you spent time and money on it is neither here nor there, that's not a moral|legal argument. Presumably there have been some exceptions (you intimate that), what sort of percentage does that amount to? If you calculate that as a true reflection of the whole population then how many people's copyright do you suppose you've infringed, knowingly, against their wishes? If someone takes your published material and slaps ads on it and republishes without your knowledge is that all good with you? (I'm guessing you may say yes here!). It's not just about economics, it's probably as much about moral rights. People believe that their Geocities content died and was buried. Of course they are pretty minor "offences" (or at least appears so) across a large population - akin to being a spammer or somesuch. Actually scratch that this is simply like copying others blog posts on a massive scale.
vyrotek 16y ago
Ah ha! So here's where you've all been hiding. http://www.reocities.com/TimesSquare/1266/ http://www.reocities.com/TimesSquare/1266/
- samatman 16y ago
  That page is by a young 'hacker' named Simon Liu. I wonder if it is this Simon Liu: http://www.computer.org/portal/web/csdl/doi/10.1109/MITP.2009.109 http://www.computer.org/portal/web/csdl/doi/10.1109/MITP.200... "Surviving Distributed Denial of Service Attacks" See? This is why reocities is awesome. The Simon Lius of the world should have access to their old Descent scores.
mhd 16y ago
And after only two random clicks, I'm at a (bad) foot fetish site. Well, it was either that, or a "Mr. T ate my <censored>"…
brc 16y ago
I checked, my very first website is thus far not there. I'll wait until you're finished, and if it doesn't appear I can send the offline backup. I've been carting it around for 10 years, laughing at the animated graphics and frames.
a_m_kelly 16y ago
I'd love to know what sort of infrastructure you're running this on. I'm in a course for my Masters for library school that deals with similar sorts of problems in maintaining and preserving for the "long term" digital materials. From your site it sounds like you wrote a script using wget to harvest the files and another to check them against versions that were still up. What do you do on the server end now to ensure that the files are still working correctly? Are you running periodic checksums on them or the like? Finally, are you looking for any help from an interested novice?
- jacquesm 16y ago
  I have a large database table that stores the md5 hashes of all the files and there is a script that can compare all of the contents of the site with the hashes in the files (and with a second copy if that's what it would come to). Some bitrot is inevitable but I think it's under control for now. As for help, yes, but right now I'm pretty swamped in other stuff, the next round of work on reocities will likely come after the new year.
  - daeken 16y ago
    Have you considered using something like MogileFS? It'd be perfect for this sort of situation. Let me know if you're interested in this or have any questions -- I've dealt a good bit with systems like this in the past, and would love to give you a hand.
    - jacquesm 16y ago
      You know where to find me :) And yes, of course I'm interested. But right now no time.
afterburner 16y ago
It is truly amazing to see one of my old sites reappear in this form. Also, there appears to be another mirror, oocities.com
flipbrad 16y ago
are you worried about copyright infringement suits?
- jacquesm 16y ago
  Not one bit. It was a situation where the common good outweighed the benefits of respecting copyright law handily, besides that the only thing that changed is the machine where the content was hosted. I do not pass it off as my own, have a very clearly established procedure for removing the data at the request of the copyright holder. Think of it as a hosting provider going out of business and a new one taking over the corpse of the old one. That it happened without contacting the owners is simply because the vast majority of the owners does not have contact information to begin with. See the reocities faq for some more info on this: http://www.reocities.com/newhome/faq.html http://www.reocities.com/newhome/faq.html