4 ms·

What was the test to determine the dataloss?

by hunterb123 4y ago

Cerium 4y ago
I came to the same conclusion by observing that there are posts and comments from only eight hours ago.
- jbverschoor 4y ago
  So that means dataloss.. Probably restored from backup. Good news for people who were banned, or for posts that didn't get enough momentum :) edit: Was restored from backup.. so def. dataloss
  - joshuamorton 4y ago
    > So that means dataloss.. Probably restored from backup. If the server went down at XX:XX, and the backup they restored from is also from XX:XX, there isn't dataloss. If the server was down for 8 hours, the last data being 8 hours old isn't dataloss, it's correct.
  - dang 4y ago
    8 hours of downtime, but not data loss, since there was no data to lose during the downtime. Last post before we went down (2022-07-08 12:46:04 UTC): https://news.ycombinator.com/item?id=32026565 https://news.ycombinator.com/item?id=32026565 First post once we were back up (2022-07-08 20:30:55 UTC): https://news.ycombinator.com/item?id=32026571 https://news.ycombinator.com/item?id=32026571 (hey, that's this thread! how'd you do that, tpmx?) So, 7h 45m of downtime. What we don't know is how many posts (or votes, etc.) happened after our last backup, and were therefore lost. The latest vote we have was at 2022-07-08 12:46:05 UTC, which is about the same as the last post. There can't be many lost posts or votes, though, because I checked HN Search (https://hn.algolia.com/ https://hn.algolia.com/) just before we brought HN back up, and their most recent comment and story were behind ours. That means our last backup on the ill-fated server was taken after the last API update (HN Search relies on our API), and the API gets updated every 30 seconds. I'm not saying that's a rock-solid argument, but it suggests that 30 seconds is an upper bound on how much data we lost.
    - deleted 4y ago
      [deleted]
    - sillysaurusx 4y ago
      Curiosity got the better of me. Why was there a 6 ID gap between the last post and first post? The answer seems to be that admins were making posts, which is neat. (There was also one lonely Flexport job ad.) Is your backup system tied to your API? Algolia is a third party service, and streaming the latest HN data to Algolia seems pretty similar to streaming it to a backup system.
      - dang 4y ago
        I posted a bunch of test things and then deleted them.
        scott_s 4y ago
        I love this answer so much.
        sillysaurusx 4y ago
        I really wanted to ask “How did you post things if the server was down?” but perhaps some things are better left as mysteries.
        dang 4y ago
        The server was up for us before it was up for everybody else.
        O__________O 4y ago
        You could see them via HN’s API before they were deleted, nothing interesting; API was back up before the www.
        dang 4y ago
sillysaurusx 4y ago
Informal. My last upvote was pretty close to when HN went down, so I expected my karma to go down, but it didn't. Also I remember the "Why we're going with Rails" story on the front page from before it went down.
- deleted 4y ago
  [deleted]

What was the test to determine the dataloss?

Cerium 4y ago

I came to the same conclusion by observing that there are posts and comments from only eight hours ago.

jbverschoor 4y ago
So that means dataloss.. Probably restored from backup. Good news for people who were banned, or for posts that didn't get enough momentum :) edit: Was restored from backup.. so def. dataloss
- joshuamorton 4y ago
  > So that means dataloss.. Probably restored from backup. If the server went down at XX:XX, and the backup they restored from is also from XX:XX, there isn't dataloss. If the server was down for 8 hours, the last data being 8 hours old isn't dataloss, it's correct.
- dang 4y ago
  8 hours of downtime, but not data loss, since there was no data to lose during the downtime. Last post before we went down (2022-07-08 12:46:04 UTC): https://news.ycombinator.com/item?id=32026565 https://news.ycombinator.com/item?id=32026565 First post once we were back up (2022-07-08 20:30:55 UTC): https://news.ycombinator.com/item?id=32026571 https://news.ycombinator.com/item?id=32026571 (hey, that's this thread! how'd you do that, tpmx?) So, 7h 45m of downtime. What we don't know is how many posts (or votes, etc.) happened after our last backup, and were therefore lost. The latest vote we have was at 2022-07-08 12:46:05 UTC, which is about the same as the last post. There can't be many lost posts or votes, though, because I checked HN Search (https://hn.algolia.com/ https://hn.algolia.com/) just before we brought HN back up, and their most recent comment and story were behind ours. That means our last backup on the ill-fated server was taken after the last API update (HN Search relies on our API), and the API gets updated every 30 seconds. I'm not saying that's a rock-solid argument, but it suggests that 30 seconds is an upper bound on how much data we lost.
  - deleted 4y ago
    [deleted]
  - sillysaurusx 4y ago
    Curiosity got the better of me. Why was there a 6 ID gap between the last post and first post? The answer seems to be that admins were making posts, which is neat. (There was also one lonely Flexport job ad.) Is your backup system tied to your API? Algolia is a third party service, and streaming the latest HN data to Algolia seems pretty similar to streaming it to a backup system.
    - dang 4y ago
      I posted a bunch of test things and then deleted them.
      - scott_s 4y ago
        I love this answer so much.
        sillysaurusx 4y ago
        I really wanted to ask “How did you post things if the server was down?” but perhaps some things are better left as mysteries.
        dang 4y ago
        The server was up for us before it was up for everybody else.
        O__________O 4y ago
        You could see them via HN’s API before they were deleted, nothing interesting; API was back up before the www.
        dang 4y ago

sillysaurusx 4y ago

Informal. My last upvote was pretty close to when HN went down, so I expected my karma to go down, but it didn't. Also I remember the "Why we're going with Rails" story on the front page from before it went down.

deleted 4y ago
[deleted]