4 ms·

We are just finishing building out our OSM infrastructure, and this morning in the shower I was thinking how nice it is that we can fix the maps by adding new c

We are just finishing building out our OSM infrastructure, and this morning in the shower I was thinking how nice it is that we can fix the maps by adding new construction and new roads (we do real estate for part of our state), and contribute this directly to OSM and then update tiles from OSM. Then I came in to read this story. Yesterday we were trying to figure out what our update strategy would be, if we would pull daily deltas, do quarterly updates, etc... We discussed the idea of vandalism and didn't have a good answer for how to prevent it. We thought about lagging behind the latest changes and then doing a "catch up" if vandalism was detected. Really what we'd like to do is make changesets based on the age of the change, except including our most recent changes (because our reputation is great :-), but I don't even know how to start doing that.

toast0 8y ago
The age of update heuristic is likely a good start; high profile vandalism like this is likely to be noticed in a reasonable amount of time. Vandalism in low profile areas is going to be harder to find though. I'm pretty sure you can get a stream of all the changes, so you would need to do something like have one copy of the database where you processed updates in near real time, and a second (production) copy where you had a delayed replication stream; in that stream, you'd want to look ahead at the real time database to calculate your stability heuristic. Alternatively, you could have the realtime database also populate a changed object queue -- anytime an object is changed pull it out of the middle of the queue and place it at the end again. If it makes it to the front, pull the current object data and place it in your production database. This should help with vandalism, but you do run the risk of accepting only parts of a coordinated edit of several objects. You could consider simply stopping replication into your production database in case anything ends up ultimately unstable, but I suspect it's hard to find a point in time where everything edited before was not edited again in the next 15 days (or whatever your threshold is). This is definitely a complex problem to solve, especially since you would want to have some escape hatches so your changes happen quickly, or you could flag upstream changes to apply quickly, or worst case, you might want to revert to an earlier version of the database. If you can figure out how to divide the data into different regions, you might have better luck, but sometimes object ids are moved around quite a bit. Geodata is already messy, and having a multitude of editors doesn't help with consistency etc.
Firefishy 8y ago
The vandalism was reverted in 2 hours over 20 days ago. If Mapbox had stayed current with the diffs we publish, the vandalism would have been gone in a blink.
- linsomniac 8y ago
  One of the news reports I read said the Mapbox press release said their AI flags over 70,000 suspicious edits a day as needing human review. Doesn't sound like much of an AI, and also sounds like the reason they were weeks behind in getting that update in.

toast0 8y ago

The age of update heuristic is likely a good start; high profile vandalism like this is likely to be noticed in a reasonable amount of time. Vandalism in low profile areas is going to be harder to find though. I'm pretty sure you can get a stream of all the changes, so you would need to do something like have one copy of the database where you processed updates in near real time, and a second (production) copy where you had a delayed replication stream; in that stream, you'd want to look ahead at the real time database to calculate your stability heuristic. Alternatively, you could have the realtime database also populate a changed object queue -- anytime an object is changed pull it out of the middle of the queue and place it at the end again. If it makes it to the front, pull the current object data and place it in your production database. This should help with vandalism, but you do run the risk of accepting only parts of a coordinated edit of several objects. You could consider simply stopping replication into your production database in case anything ends up ultimately unstable, but I suspect it's hard to find a point in time where everything edited before was not edited again in the next 15 days (or whatever your threshold is). This is definitely a complex problem to solve, especially since you would want to have some escape hatches so your changes happen quickly, or you could flag upstream changes to apply quickly, or worst case, you might want to revert to an earlier version of the database. If you can figure out how to divide the data into different regions, you might have better luck, but sometimes object ids are moved around quite a bit. Geodata is already messy, and having a multitude of editors doesn't help with consistency etc.

Firefishy 8y ago

The vandalism was reverted in 2 hours over 20 days ago. If Mapbox had stayed current with the diffs we publish, the vandalism would have been gone in a blink.

linsomniac 8y ago
One of the news reports I read said the Mapbox press release said their AI flags over 70,000 suspicious edits a day as needing human review. Doesn't sound like much of an AI, and also sounds like the reason they were weeks behind in getting that update in.