2 ms·

The Internet Archive should archive stuff before a company enters financial hardship and can longer afford the outgoing bandwidth costs to have their entire dat

The Internet Archive should archive stuff before a company enters financial hardship and can longer afford the outgoing bandwidth costs to have their entire dataset replicated online. It makes sense to block those attempts. It strikes me that a similar fate would most likely have occurred if Everett was hit by a bus and his bank accounts eventually ran dry. We should identify historically significant single points-of-failure and make archives before the failures occur, insofar as is possible.

eropple 12y ago
Sure, but it's important to note that I'm not talking about blocking the scrapers. I'm talking about "we're willing to pay for an S3 export, Amazon will mail out a bunch of hard drives." Everett has steadfastly avoided contacting Archive Team or anyone else who wants to do this and I can't think of an ethically not-icky reason for that.
- cookiecaper 12y ago
  The not-icky reason is that he doesn't want to. He's not under any obligation to do it. Maybe we'd like it if he did this, but I don't think it's fair to assume bad or even just "ethically icky" motives. If all of twitpic's data is exported or copied elsewhere, he can no longer effectively profit off of it. We don't know that his intention is to just nuke the data and be done, he may be trying to salvage what he can monetarily (and the "we're getting acquired, just kidding" thing is probably good evidence that something like this is going on behind the scenes). I don't think there's anything wrong with this either. It's his service, and if his motive was profit, it makes perfect sense to shop the data around instead of just accepting a total loss. There's no reason twitpic should have to be run for altruistic purposes. Maybe dcurtis et al would have better luck getting a response if instead of saying "We'll cover the bandwidth costs", they said "We'll buy the data from you at a price commensurate with the years of effort and upkeep you've invested into an apparently massively important historical archive". I don't think it's fair to pretend like Noah Everett is obligated to sell his archive for the cost of bandwidth. In any case, at this point he's made it clear that he wants to wash his hands of the project in one way or another. It's his personal project and he is and should be free to do that. This is the worst possible time to try to do a bulk archive. I understand that we can't see into the future and can't always predict when something like this is going to go down, but I'd think we can probably improve our heuristics so that something as seemingly significant as twitpic can't just vanish into the ether next time.
  - deleted 12y ago
    [deleted]
  - eropple 12y ago
    > The not-icky reason is that he doesn't want to. Eight hundred million photos and all their comments. That's historical record. That's our historical record. No one jerk gets to destroy all of that. The "hurr, well, don't use it" is completely bonkers--I don't use it, but my history is on there, too. It's a record I didn't put there but matters to me as a citizen and as a member of society. Destroying a major part of the historical record for everybody is indefensibly shitty, full-stop, you don't get to say "well it's mine" when it's not. It's ours. Twitpic was allowed to profit by holding onto it awhile. Now it's time to do the right thing.
    - cookiecaper 12y ago
      It is his. This is what happens in an ethereal format like the internet, and it's something we'll have to adapt to -- in the history of mass publication, one wasn't able to just destroy all copies of a thing with the push of a button. Books and pamphlets could be hunted down, but almost always some would survive, because you can't remotely change the contents of the paper. In the digital age, you can. If there's a lesson to take from this, it's the dangers of the non-permanence of the web; we should be looking at solutions that can mitigate that danger. But I don't blame Everett at all for not just giving away his company's IP for $15k in bandwidth bills, and I don't believe he should be shamed into doing so. If I were him, I'd probably be contacting Sotheby's or another auctioneer that specializes in high-value goods and seeing about selling off the dataset to the highest bidder. Then these fellows can put their money where their mouth is and show us just how important the preservation of that data is.
      - eropple 12y ago
        This celebration of plutocracy gone wild is one of the more disgusting things I've read on Hacker News, and that's saying a lot. His corporation only exists through our societal forebearance. He has an obligation to the society that gave him the legal structure and protections thereof. Part of that is not deleting history just because he's in a snit.

eropple 12y ago

Sure, but it's important to note that I'm not talking about blocking the scrapers. I'm talking about "we're willing to pay for an S3 export, Amazon will mail out a bunch of hard drives." Everett has steadfastly avoided contacting Archive Team or anyone else who wants to do this and I can't think of an ethically not-icky reason for that.

cookiecaper 12y ago
The not-icky reason is that he doesn't want to. He's not under any obligation to do it. Maybe we'd like it if he did this, but I don't think it's fair to assume bad or even just "ethically icky" motives. If all of twitpic's data is exported or copied elsewhere, he can no longer effectively profit off of it. We don't know that his intention is to just nuke the data and be done, he may be trying to salvage what he can monetarily (and the "we're getting acquired, just kidding" thing is probably good evidence that something like this is going on behind the scenes). I don't think there's anything wrong with this either. It's his service, and if his motive was profit, it makes perfect sense to shop the data around instead of just accepting a total loss. There's no reason twitpic should have to be run for altruistic purposes. Maybe dcurtis et al would have better luck getting a response if instead of saying "We'll cover the bandwidth costs", they said "We'll buy the data from you at a price commensurate with the years of effort and upkeep you've invested into an apparently massively important historical archive". I don't think it's fair to pretend like Noah Everett is obligated to sell his archive for the cost of bandwidth. In any case, at this point he's made it clear that he wants to wash his hands of the project in one way or another. It's his personal project and he is and should be free to do that. This is the worst possible time to try to do a bulk archive. I understand that we can't see into the future and can't always predict when something like this is going to go down, but I'd think we can probably improve our heuristics so that something as seemingly significant as twitpic can't just vanish into the ether next time.
- deleted 12y ago
  [deleted]
- eropple 12y ago
  > The not-icky reason is that he doesn't want to. Eight hundred million photos and all their comments. That's historical record. That's our historical record. No one jerk gets to destroy all of that. The "hurr, well, don't use it" is completely bonkers--I don't use it, but my history is on there, too. It's a record I didn't put there but matters to me as a citizen and as a member of society. Destroying a major part of the historical record for everybody is indefensibly shitty, full-stop, you don't get to say "well it's mine" when it's not. It's ours. Twitpic was allowed to profit by holding onto it awhile. Now it's time to do the right thing.
  - cookiecaper 12y ago
    It is his. This is what happens in an ethereal format like the internet, and it's something we'll have to adapt to -- in the history of mass publication, one wasn't able to just destroy all copies of a thing with the push of a button. Books and pamphlets could be hunted down, but almost always some would survive, because you can't remotely change the contents of the paper. In the digital age, you can. If there's a lesson to take from this, it's the dangers of the non-permanence of the web; we should be looking at solutions that can mitigate that danger. But I don't blame Everett at all for not just giving away his company's IP for $15k in bandwidth bills, and I don't believe he should be shamed into doing so. If I were him, I'd probably be contacting Sotheby's or another auctioneer that specializes in high-value goods and seeing about selling off the dataset to the highest bidder. Then these fellows can put their money where their mouth is and show us just how important the preservation of that data is.
    - eropple 12y ago
      This celebration of plutocracy gone wild is one of the more disgusting things I've read on Hacker News, and that's saying a lot. His corporation only exists through our societal forebearance. He has an obligation to the society that gave him the legal structure and protections thereof. Part of that is not deleting history just because he's in a snit.