4 ms·

Hopefully it is just hidden and not deleted. But this is the main reason why alternative archive sites exists which ignore the original posters requests. Freque

Hopefully it is just hidden and not deleted. But this is the main reason why alternative archive sites exists which ignore the original posters requests. Frequently used to archive posts from public figures which are suspected to be attempted to be scrubbed later.

joe_the_user 5y ago
Got any examples of these alternative archives?
- SilverRed 5y ago
  This is the main one https://archive.is/ https://archive.is/ From the FAQ, they do not respect robots.txt since they only archive on request by a user and they do not remove archives unless they contain illegal content.
  - 1vuio0pswjnm7 5y ago
    HN commenters like to use archive.is but I always wonder if people are aware that archive.is is (a) blocked in some countries^1 and (b) may block access to itself in a country if it feels threatened.^2 There is also the issue of EDNS subnet.^3 archive.is tries to require it; it wants to know what location a request is coming from. In addition to EDNS, archive.is inserts the IP address and geolocation of the incoming request into the HTML of the returned page as a tracking pixel.^4 Thus archive.is does some things archive.org does not do besides just ignoring robots.txt One of the things archive.org does that archive.is does not do is that archive.org inserts an HTTP response header intended to disable Chrome FLoC.^5 I add this header for all sites in a local proxy; however I do not see many sites adding it as a courtesy. Thanks archive.org for doing that. 1. https://en.wikipedia.org/wiki/Archive.is https://en.wikipedia.org/wiki/Archive.is 2. https://www.reddit.com/r/KotakuInAction/comments/3e29vm/archiveis_possibly_banned_in_finland/ https://www.reddit.com/r/KotakuInAction/comments/3e29vm/arch... 3. https://webapps.stackexchange.com/questions/135222/why-does-1-1-1-1-not-resolve-archive-is https://webapps.stackexchange.com/questions/135222/why-does-... 4. https://news.ycombinator.com/item?id=27498902 https://news.ycombinator.com/item?id=27498902 5. permissions-policy: interest-cohort=()
- v0x 5y ago
  There’s archive.is but I get the sense that the major use case for that is getting around paywalls as opposed to permanently archiving a page - indeed, since they host content that the site owner probably doesn’t want them to, it would stand to reason the service would not be likely to stand the yet of time. But I could be wrong.
- mellosouls 5y ago
  A couple of pointers to the wider world of web archiving: https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-Community https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-... https://github.com/iipc/awesome-web-archiving https://github.com/iipc/awesome-web-archiving
fwn 5y ago
As far as I was able to experience it's just hidden and not deleted. I have to keep an old domain indefinitely to host a robots.txt just to keep sensitive personal data hidden that little me foolishly published on the open internet. But I'm not complaining. The internet archive is a great gift. Using it with a bookmarklet really feels like a super power.
- mercora 5y ago
  it sounds a lot like this would need some kind of delegation mechanism where you could point to a different URL in-time before abandoning the place. or maybe some kind of sealing using a cryptographic function that lets you proof your are the owner of the current/previous content and also would proof you are not the owner of the newer content while this proof could be used to release the ban if ever needed.
techrat 5y ago
> Hopefully it is just hidden and not deleted. Hidden. Even when you request for them to remove stuff. Had domain, stuff got archived, asked for them to remove it, added robots.txt. Domain lapsed. Someone else picked it up. their robots.txt now permissive, old stuff that I requested for them to remove is now visible.
- mavhc 5y ago
  Wonder how you'd overcome that flaw, is there a history of domain name ownership?
  - newswasboring 5y ago
    Just delete the things when requested. No need to make it more complicated than that.
    - mavhc 5y ago
      When requested by who? The current owner of the domain? Do they own what was on that domain 20 years ago? What if you lost your domain, but owned it in the past? Can you delete stuff from that era?
      - logbiscuitswave 5y ago
        I’ve requested old domains of mine to get removed from archive.org and at least as of a few years ago it was an all or nothing thing. There was no surgical way to say “only remove content from between 1995 and 2005 when I owned the domain”. Maybe things have changed since then, though.
  - account42 5y ago
    Maybe we need a whois.archive.org.
- throwslackforce 5y ago
  That's insidious. Does it even make sense to revive data that has been removed based on current configuration? Even if the owner is the same, allowing the site to be archived going forward isn't the same thing as permitting it retroactively.

joe_the_user 5y ago

Got any examples of these alternative archives?

SilverRed 5y ago
This is the main one https://archive.is/ https://archive.is/ From the FAQ, they do not respect robots.txt since they only archive on request by a user and they do not remove archives unless they contain illegal content.
- 1vuio0pswjnm7 5y ago
  HN commenters like to use archive.is but I always wonder if people are aware that archive.is is (a) blocked in some countries^1 and (b) may block access to itself in a country if it feels threatened.^2 There is also the issue of EDNS subnet.^3 archive.is tries to require it; it wants to know what location a request is coming from. In addition to EDNS, archive.is inserts the IP address and geolocation of the incoming request into the HTML of the returned page as a tracking pixel.^4 Thus archive.is does some things archive.org does not do besides just ignoring robots.txt One of the things archive.org does that archive.is does not do is that archive.org inserts an HTTP response header intended to disable Chrome FLoC.^5 I add this header for all sites in a local proxy; however I do not see many sites adding it as a courtesy. Thanks archive.org for doing that. 1. https://en.wikipedia.org/wiki/Archive.is https://en.wikipedia.org/wiki/Archive.is 2. https://www.reddit.com/r/KotakuInAction/comments/3e29vm/archiveis_possibly_banned_in_finland/ https://www.reddit.com/r/KotakuInAction/comments/3e29vm/arch... 3. https://webapps.stackexchange.com/questions/135222/why-does-1-1-1-1-not-resolve-archive-is https://webapps.stackexchange.com/questions/135222/why-does-... 4. https://news.ycombinator.com/item?id=27498902 https://news.ycombinator.com/item?id=27498902 5. permissions-policy: interest-cohort=()
v0x 5y ago
There’s archive.is but I get the sense that the major use case for that is getting around paywalls as opposed to permanently archiving a page - indeed, since they host content that the site owner probably doesn’t want them to, it would stand to reason the service would not be likely to stand the yet of time. But I could be wrong.
mellosouls 5y ago
A couple of pointers to the wider world of web archiving: https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-Community https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-... https://github.com/iipc/awesome-web-archiving https://github.com/iipc/awesome-web-archiving

fwn 5y ago

As far as I was able to experience it's just hidden and not deleted. I have to keep an old domain indefinitely to host a robots.txt just to keep sensitive personal data hidden that little me foolishly published on the open internet. But I'm not complaining. The internet archive is a great gift. Using it with a bookmarklet really feels like a super power.

mercora 5y ago
it sounds a lot like this would need some kind of delegation mechanism where you could point to a different URL in-time before abandoning the place. or maybe some kind of sealing using a cryptographic function that lets you proof your are the owner of the current/previous content and also would proof you are not the owner of the newer content while this proof could be used to release the ban if ever needed.

techrat 5y ago

> Hopefully it is just hidden and not deleted. Hidden. Even when you request for them to remove stuff. Had domain, stuff got archived, asked for them to remove it, added robots.txt. Domain lapsed. Someone else picked it up. their robots.txt now permissive, old stuff that I requested for them to remove is now visible.

mavhc 5y ago
Wonder how you'd overcome that flaw, is there a history of domain name ownership?
- newswasboring 5y ago
  Just delete the things when requested. No need to make it more complicated than that.
  - mavhc 5y ago
    When requested by who? The current owner of the domain? Do they own what was on that domain 20 years ago? What if you lost your domain, but owned it in the past? Can you delete stuff from that era?
    - logbiscuitswave 5y ago
      I’ve requested old domains of mine to get removed from archive.org and at least as of a few years ago it was an all or nothing thing. There was no surgical way to say “only remove content from between 1995 and 2005 when I owned the domain”. Maybe things have changed since then, though.
- account42 5y ago
  Maybe we need a whois.archive.org.
throwslackforce 5y ago
That's insidious. Does it even make sense to revive data that has been removed based on current configuration? Even if the owner is the same, allowing the site to be archived going forward isn't the same thing as permitting it retroactively.