6 ms·

I'm wondering why doesn't HN keeps a cached copy of each page that makes it to the front page (or maybe automatically post it to archive.is). I don't think it w

I'm wondering why doesn't HN keeps a cached copy of each page that makes it to the front page (or maybe automatically post it to archive.is). I don't think it would be hard to implement and would avoid this quite frequent problem.

konschubert 10y ago
Because of copyright issues. People like traffic to their actual domain, because it often means better search engine ranking in the future. On top of that, some websites serve ads, which means that traffic is proportional to revenue.
- mac01021 10y ago
  Could we pull up the real page from the actual domain in a full-window iframe and then replace it with the cached version if it fails to load?
  - majewsky 10y ago
    That will fail for a lot of websites since X-Frame-Options is becoming more common: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-...
- Qwertious 10y ago
  Ah, don't you just love it when there's a simple technical problem with a simple and valid technical solution, but there's a legal problem that stops it dead? Well, legal and whatever domain the "you viewed the ads but they werent tracked so we don't get money" problem falls into. Stupid artificial barriers which are there because they solve an economic problem, that wouldn't exist in a perfect world. Game theory, maybe.
  - leephillips 10y ago
    I have the technical ability to steal lots of stuff. But yes, there are legal (and moral) obstacles that keep me from doing it.
  - overcast 10y ago
    You'd be OK with putting a ton of work into something, and then having it cached/ripoffed somewhere else?
    - fao_ 10y ago
      Google and the Internet Archive already caches it. Also, if your work goes down for whatever (perhaps legal) reason, you know that people can still benefit from it. Caching it for when a site is overloaded is not ripping it off, just like sending copies of works to the British Library for archival purposes is not ripping the original work off.
      - otterley 10y ago
        Both of those sites can be instructed not to store copies of your pages for their caches or archives; and they will dutifully follow your instructions. Also, the library analogy doesn't work in copyright law. Copyright protects the act of copying, not the act of transferring an already-authorized copy of a work to someone else.
      - justinhensley 10y ago
        The trick in this solution is deciding when to flip the switch for the cached version. Perhaps a simple GET to the URL ever 60s. If you receive 5 non-200 responses in a row, then the HN link points to the archive.is version. Same method in reverse for bringing it back.
        fao_ 10y ago
        That sounds like it would DDoS the site. Even better: monitor the comments for "site is down", or just provide a link to google's cached version as a button (Even better than the 'web' link we have at the moment).
        justinsaccount 10y ago
        > That sounds like it would DDoS the site How would one request per minute from a single server DoS the site, let alone DDoS it?
  - quickConclusion 10y ago
    This reminds me of the bird complaining: "if only there was no air slowing me down, I would fly a lot faster".
  - mnutt 10y ago
    The solution described above is pretty much what Google's AMP does. It turns out that a lot of people don't like AMP because they see it as a monopolistic way of keeping users in Google's walled garden. HN doesn't quite have a monopoly on startup/tech news, but I think the same reasons apply.
    - sp332 10y ago
      Sites opt-in to AMP, so copyright is less of an issue there.
akerro 10y ago
This post has only 30upvotes right now, I'm surprised that website couldn't hold it. My WP blog hosted for ~3euro/month was a few times here on top with 600UV, on reddit, according to GA with 3600 people online at the same time... and it didn't die to my surprise. I would rather say that hosting is... not performing well.
- majewsky 10y ago
  That hinges on the assumption that most (or even a non-negligible amount of) visitors will upvote the article.
- teh_klev 10y ago
  Upvotes are probably not a good estimate of traffic from HN hitting a site for a multitude of reasons, for example: - not everyone upvotes - there's probably a substantial number of users who are content to read articles but have never had a login (I did for two years) - social media amplification.
  - akerro 10y ago
    I know, that's why I mentioned reddit too.
- pmlnr 10y ago
  That 503 looked like apache mod_security throttling.
- nine_k 10y ago
  Static pages vs something like WordPress?
nashashmi 10y ago
try using fullhn.com
- justinhensley 10y ago
  This is useful.

konschubert 10y ago

Because of copyright issues. People like traffic to their actual domain, because it often means better search engine ranking in the future. On top of that, some websites serve ads, which means that traffic is proportional to revenue.

mac01021 10y ago
Could we pull up the real page from the actual domain in a full-window iframe and then replace it with the cached version if it fails to load?
- majewsky 10y ago
  That will fail for a lot of websites since X-Frame-Options is becoming more common: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-...
Qwertious 10y ago
Ah, don't you just love it when there's a simple technical problem with a simple and valid technical solution, but there's a legal problem that stops it dead? Well, legal and whatever domain the "you viewed the ads but they werent tracked so we don't get money" problem falls into. Stupid artificial barriers which are there because they solve an economic problem, that wouldn't exist in a perfect world. Game theory, maybe.
- leephillips 10y ago
  I have the technical ability to steal lots of stuff. But yes, there are legal (and moral) obstacles that keep me from doing it.
- overcast 10y ago
  You'd be OK with putting a ton of work into something, and then having it cached/ripoffed somewhere else?
  - fao_ 10y ago
    Google and the Internet Archive already caches it. Also, if your work goes down for whatever (perhaps legal) reason, you know that people can still benefit from it. Caching it for when a site is overloaded is not ripping it off, just like sending copies of works to the British Library for archival purposes is not ripping the original work off.
    - otterley 10y ago
      Both of those sites can be instructed not to store copies of your pages for their caches or archives; and they will dutifully follow your instructions. Also, the library analogy doesn't work in copyright law. Copyright protects the act of copying, not the act of transferring an already-authorized copy of a work to someone else.
    - justinhensley 10y ago
      The trick in this solution is deciding when to flip the switch for the cached version. Perhaps a simple GET to the URL ever 60s. If you receive 5 non-200 responses in a row, then the HN link points to the archive.is version. Same method in reverse for bringing it back.
      - fao_ 10y ago
        That sounds like it would DDoS the site. Even better: monitor the comments for "site is down", or just provide a link to google's cached version as a button (Even better than the 'web' link we have at the moment).
        justinsaccount 10y ago
        > That sounds like it would DDoS the site How would one request per minute from a single server DoS the site, let alone DDoS it?
- quickConclusion 10y ago
  This reminds me of the bird complaining: "if only there was no air slowing me down, I would fly a lot faster".
- mnutt 10y ago
  The solution described above is pretty much what Google's AMP does. It turns out that a lot of people don't like AMP because they see it as a monopolistic way of keeping users in Google's walled garden. HN doesn't quite have a monopoly on startup/tech news, but I think the same reasons apply.
  - sp332 10y ago
    Sites opt-in to AMP, so copyright is less of an issue there.

akerro 10y ago

This post has only 30upvotes right now, I'm surprised that website couldn't hold it. My WP blog hosted for ~3euro/month was a few times here on top with 600UV, on reddit, according to GA with 3600 people online at the same time... and it didn't die to my surprise. I would rather say that hosting is... not performing well.

majewsky 10y ago
That hinges on the assumption that most (or even a non-negligible amount of) visitors will upvote the article.
teh_klev 10y ago
Upvotes are probably not a good estimate of traffic from HN hitting a site for a multitude of reasons, for example: - not everyone upvotes - there's probably a substantial number of users who are content to read articles but have never had a login (I did for two years) - social media amplification.
- akerro 10y ago
  I know, that's why I mentioned reddit too.
pmlnr 10y ago
That 503 looked like apache mod_security throttling.
nine_k 10y ago
Static pages vs something like WordPress?

nashashmi 10y ago

try using fullhn.com

justinhensley 10y ago
This is useful.