4 ms·
Nah, you can snapshot every 15 minutes. The snapshot interval depends on the frequency of changes and their capacity, but it's up to them how to allocate these
by Kiboneu 7mo ago
Nah, you can snapshot every 15 minutes. The snapshot interval depends on the frequency of changes and their capacity, but it's up to them how to allocate these capacities... but it's definitely doable and there are real reasons for doing so. You can collapse deltas between snapshots after some time to make them last longer. I'd be surprised if they don't do that.
As an aside, snapshotting would have prevented a good deal of horror stories shared by people who give AI access to the FS. Well, as long as you don't give it root.......
- gchamonlive 7mo agoThe problem isn't the granularity of the backup but since the worm silently nukes pages, it's virtually impossible to reconcile the state before the attack and the current state, so you have to just forfeit any changes made since then and ask the contributors to do the leg work of reapplying the correct changes
- Kiboneu 7mo agoWhy would nuked pages matter? Snapshots capture everything and are not part of wikimedia software.
- gchamonlive 7mo agoThe nuke might be legitimate?
- wizzwizz4 7mo agoThat's not a lot of state lost. Destructive operations are easier to replay than constructive ones.
- gchamonlive 7mo agoIs Wikimedia overreacting then?
- wizzwizz4 7mo agoNo: from what I can tell, they're being conservative, which is appropriate here. Once you've pushed the "stop bad things happening" button, there's no need to rush.
- sobjornstad 7mo agoNowadays I refuse to do any serious work that isn't in source control anywhere besides my NAS that takes copy-on-write snapshots every 15 minutes. It has saved my butt more times than I can count.
- Kiboneu 7mo agoYeah same here. Earlier I had a sync error that corrupted my .git, somehow. no problem; I go back 15 minutes and copy the working version. Feels good to pat oneself in the back. Mine is sore, though. My E&O/cyber insurance likes me.
- john_strinlai 7mo ago>Nah, you can snapshot every 15 minutes. obviously you can. but, what is the actual snapshot frequency? like, what is the timestamp of the last known good snapshot? that is what matters. in any case, the comment you are replying to is a hypothetical, which correctly points out that even a day or two of lost edits is fine (not ideal, but fine). your reply doesnt engage with their comment at all.
- Kiboneu 7mo ago> the comment you are replying to is a hypothetical, which correctly points out that even a day or two of lost edits is fine (not ideal, but fine). your reply doesnt engage with their comment at all. I did engage, by pointing out that it wasn't relevant nor a realistic scenario for a competent sysadmin. (Did you read the OP?) That's a /you/ problem if you rely on infrequent backups, especially for a service with so much flux. > what is the actual snapshot frequency? like, what is the timestamp of the last known good snapshot? ? Why would I know what their internal operations are?
- john_strinlai 7mo ago>I did engage, by pointing out that it wasn't relevant nor a realistic scenario for a competent sysadmin. >Why would I know what their internal operations are? i mean... you must, right? you know that once-a-day snapshots is not relevant to this specific incident. you know that their sysadmins are apparently competent. i just assumed you must have some sort of insider information to be so confident.
- Kiboneu 7mo agoI think you are misreading my comments and made a bad assumption. The reason I'm confident is because this has been my bread and butter for a decade.
- john_strinlai 7mo ago>The reason I'm confident is because this has been my bread and butter for a decade. my decade of dealing with incompetent sysadmins and broken backups (if they even exist) has given me the opposite of confidence. but im glad you have had a different experience