7 ms·

Mapbox CEO says the map calling New York City ‘Jewtropolis’ has been 100% fixed

63 points by yincrash 8y ago

village-idiot 8y ago
And this is why incorporating user submitted information is a tricky strategy. On one hand, there sure are a lot more users than employees. On the other hand, bad actors are often far more motivated and diligent than good faith actors, and they will try to screw over your system for no good reason.
- _sdegutis 8y ago
  I've come to the conclusion that at least 1% of users are actively malicious with at least enough free time, resources, and coordination of a small full time team of professionals. I created and hosted a free self-moderated community which went well for a little while, until at least a couple people became obsessed with spamming the site with extreme gore (see the Mr. OP comment in http://old.reddit.com/r/editfight http://old.reddit.com/r/editfight which is relatively tame compared to other things they've done) and evading every single method I had of stopping or banning them, until I had to just give up altogether and shut the site down. I feel bad for the moderators on imgur.com (where I got the majority of my users) because I read somewhere that many of them now have to go to therapy for some of the things they've seen in doing their job. Online communities and the moderation of them in the modern internet is problematic enough that I personally never want to tackle that again, not even by delegating it to someone else.
  - village-idiot 8y ago
    In the long run, I can’t help but wonder if creating the internet was a mistake. We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia.
    - myaccount4real 8y ago
      Mistake? At least there is visibility. I don't know what the answer to the issue is, but I know hiding it in the corner, unobserved by the masses is not the answer. This leads to NO social push back, as there is nothing to push back about.
    - xeromal 8y ago
      If you combine that, social media, and porn, how much of the internet is still 'wholesome'? I think niche forums are my favorite part of the internet left.
      - village-idiot 8y ago
        I genuinely think Social Media is going to get us all killed. Porn might prevent the next generation from being born, worst case.
        krapp 8y ago
        >Porn might prevent the next generation from being born, worst case. You know it's possible to watch porn and have sexual relationships with other people though, right?
        village-idiot 8y ago
        Worst case, not expected case.
        krapp 8y ago
        Worst case scenarios have to at least be possible. I don't believe it's possible that internet porn can or will 'prevent the next generation from being born', so I reject it as a valid worst case.
    - krapp 8y ago
      >We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia. We've also effectively given everyone else access to information, communication and opportunities to expand their worldview that have been more profoundly transformational than even the printing press. No, creating the internet was not a mistake, any more than any means of mass communication is a mistake because it can be misused as well as used wisely. Without the internet, global culture would be entirely controlled, centralized and limited by media conglomerates, digital rights management and distribution by physical media, and our means of mass communication limited to telephones. The same internet that grants freedom of expression to the "absolute worst elements of society" grants it to you and I as well. The internet is just a network, but I believe the good it's done for humanity has far outweighed the ill.
  - jakobegger 8y ago
    Hey! Just wanted to let you know that I thought Editfight was something special. I came by occasionally and drew something. Too bad it's gone now -- just a few days ago my 4 year old asked me if he could play the game where you draw something on the screen again. It's a bummer you had to shut it down, but I understand.
    - _sdegutis 8y ago
      Thanks for the kind words, and glad to hear it made an impression! The code is sitting here on my computer just waiting to be deployed again, but it would have to be somewhere those demonic trolls can find. That said, they don't go to HN and I do own the domain name for another year, so maybe a subdomain is in order... EDIT: (Apparently my HN account is rate-limited so I can't reply directly.) How about this: anyone who wants access, email me and I'll send you a link. The site is up and running. People can send the link to their friends, and it'll be a private HN-only sub-community.
      - ars 8y ago
        Is there a way to federate to HN logins? Only open it to HN members? Or at least invite only?
        sdegutis 8y ago
        I just put the site up in a subdomain specific to HN, check my post history for the link. Everyone from HN is invited, tell your friends. It'll be a mini party lol
        jakobegger 8y ago
        I found it! https://hn.editfight.com https://hn.editfight.com
  - Kagerjay 8y ago
    Reminds me of reddit r/place, first prominent artwork was Charizards penis source: was a moderator for reddit r/place
- RestlessMind 8y ago
  Society and Distributed Systems are similar in the sense that interesting stuff happens at the tail, and if you want to make them behave properly, you need to focus on those 1% (or 0.1% of the tail cases). For systems: queries of death, queries with highest serving latency, records with huge size compared to rest of the entries in DB, records with extremely high fan-in for your streaming pipeline, ... For society, a tiny minority of: users who would vandalize online data sets, refugees who commit terrorism / crimes, citizens committing crimes, recipients cheating welfare programs, people avoiding taxes, ... It is also the tail that imposes the maximum cost on your system / society.
  - glibassertion 8y ago
    > It is also the tail that imposes the maximum cost on your system / society. This is not true for every case. Excessive focus on tail risks (and moral hazard,) in fact, can be absurdly costly.
    - village-idiot 8y ago
      See: medicare fraud prevention.
- princekolt 8y ago
  That is true, but at the same time, Wikipedia wouldn't be possible without it. The CEO confirmed that their absuse-detection system immediately flagged the label, but apparently some human reviewer approved it. Which shows that even if your system is perfect, humans are always the weakest link.
  - village-idiot 8y ago
    Yeah, it's clear that this strategy can work, but a lot of people underestimate how much energy must be spent policing for anti-social behavior.
  - rmc 8y ago
    > Wikipedia wouldn't be possible without it OpenStreetMap wouldn't be possible without it either. :) For OSM, humans are the strongest link, they are often the best way to fix these kinds of changes. A strong local community is the solution. OSM fixed this vandalism within 2 hours.
yifanl 8y ago
Correct me if I'm missing something, but what's the usecase for user submitted data on the map? Do city names change so frequently that they need to be dynamically updated? I can see some use for it when you get to the neighbourhood level to account for local nicknames, but not full cities.
- mikestew 8y ago
  Do city names change so frequently that they need to be dynamically updated? Maybe that's the bug. It didn't occur to me until I read your comment [0] that the system could simply accept user-submitted data only above a certain level of detail. It won't eliminate the damage, but it would keep it more contained. [0] OTOH, GIS isn't my day-to-day like it would be if I worked at, say, MapBox.
  - maxerickson 8y ago
    So far human moderation has been a lot cheaper than implementing a reputation system for editing. (after all, someone has to be able to edit things like the label for New York) For instance, there's tens of millions of objects in OSM. Step one to locking some of them is deciding which ones. That alone is a big project.
- lucideer 8y ago
  The use case is that the entire globe is both incompletely mapped and also constantly changing: having it all editable is much easier than differentiating each and every case where some element might or might not change frequently. Such a system of differentiation of map elements by likelihood of change could be good in the long term, but it would be a huge undertaking to classify them.
- Cyclone_ 8y ago
  Helping label things like construction, store closures/openings and like you said local nicknames that might be hard to get data on from an official source.
  - 8y ago
lqet 8y ago
It continues to amaze me that OpenStreetMap is not massively plagued by this (I know that MapBox uses OSM as one of many sources, as mentioned in the article, but the changed name did not appear in the live OSM map, afaik, so I am guessing it came from somewhere else). In the past, the main explanatory argument was always that OSM's complicated editing process was the gatekeeper, but I don't think this is true anymore. With the OSM web editor, it is extremely simple to edit the map. It would only take a few clicks to immediately publish (for example) a highway in the form of a swastika to the live map, but despite using the map multiple times per day, both for private use and professionally, I have never encountered something like this.
- thinkingemote 8y ago
  It was spotted and fixed within 24 hours in OSM (about 30 days ago), that's the power of a community driven map. Mapbox don't keep their maps up to date, and probably used a snapshot of the data which happened to have gotten in the few hours before it was reverted.
  - lqet 8y ago
    Do you happen to know the OSM relation ID that was affected? The "New York City" relation [1] does not have anything related in its history [2] [1] https://www.openstreetmap.org/relation/175905 https://www.openstreetmap.org/relation/175905 [2] https://www.openstreetmap.org/api/0.6/relation/175905/history https://www.openstreetmap.org/api/0.6/relation/175905/histor...
    - untog 8y ago
      I wouldn't be surprised if it was purged from the history. Otherwise might it imply that New York City was once given that name, historically?
      - thinkingemote 8y ago
        OSM doesn't record historical changes of that kind (as in differentiating it from any other edit) and the nature of a change in that way. Although one might be able to infer actual geographical changes from comparisons. It just concerns itself with a snapshot of "now".
nicetryguy 8y ago
>>>/pol/
- deleted 8y ago
  [deleted]
sergiotapia 8y ago
Hahaha, do they have logs of who did this?
- deleted 8y ago
  [deleted]
- Firefishy 8y ago
  Yes, here is the changelog: https://www.openstreetmap.org/changeset/61555047?node_page=3 https://www.openstreetmap.org/changeset/61555047?node_page=3
  - deleted 8y ago
    [deleted]
  - rrdharan 8y ago
    And here’s the block of the vandal: https://www.openstreetmap.org/user_blocks/2141 https://www.openstreetmap.org/user_blocks/2141 Kind of interesting to see all the other active blocks in effect and the corresponding explanatory notes.
linsomniac 8y ago
We are just finishing building out our OSM infrastructure, and this morning in the shower I was thinking how nice it is that we can fix the maps by adding new construction and new roads (we do real estate for part of our state), and contribute this directly to OSM and then update tiles from OSM. Then I came in to read this story. Yesterday we were trying to figure out what our update strategy would be, if we would pull daily deltas, do quarterly updates, etc... We discussed the idea of vandalism and didn't have a good answer for how to prevent it. We thought about lagging behind the latest changes and then doing a "catch up" if vandalism was detected. Really what we'd like to do is make changesets based on the age of the change, except including our most recent changes (because our reputation is great :-), but I don't even know how to start doing that.
- toast0 8y ago
  The age of update heuristic is likely a good start; high profile vandalism like this is likely to be noticed in a reasonable amount of time. Vandalism in low profile areas is going to be harder to find though. I'm pretty sure you can get a stream of all the changes, so you would need to do something like have one copy of the database where you processed updates in near real time, and a second (production) copy where you had a delayed replication stream; in that stream, you'd want to look ahead at the real time database to calculate your stability heuristic. Alternatively, you could have the realtime database also populate a changed object queue -- anytime an object is changed pull it out of the middle of the queue and place it at the end again. If it makes it to the front, pull the current object data and place it in your production database. This should help with vandalism, but you do run the risk of accepting only parts of a coordinated edit of several objects. You could consider simply stopping replication into your production database in case anything ends up ultimately unstable, but I suspect it's hard to find a point in time where everything edited before was not edited again in the next 15 days (or whatever your threshold is). This is definitely a complex problem to solve, especially since you would want to have some escape hatches so your changes happen quickly, or you could flag upstream changes to apply quickly, or worst case, you might want to revert to an earlier version of the database. If you can figure out how to divide the data into different regions, you might have better luck, but sometimes object ids are moved around quite a bit. Geodata is already messy, and having a multitude of editors doesn't help with consistency etc.
mvexel 8y ago
Looks like https://www.openstreetmap.org/changeset/61555047 https://www.openstreetmap.org/changeset/61555047 may have been the source of this. It's the only changeset in the history of OSM for the area that contains the offending term. It's from August 10, and was reverted on August 11 in https://www.openstreetmap.org/changeset/61556585 https://www.openstreetmap.org/changeset/61556585
- Firefishy 8y ago
  Correct. The vandalism was reverted within 2 hours and a data update publish a minute or 2 later. Mapbox import the original vandalism change only today for some reason and there seems to have been a delay in importing the revert.

63 points by yincrash 8y ago

And this is why incorporating user submitted information is a tricky strategy. On one hand, there sure are a lot more users than employees. On the other hand, bad actors are often far more motivated and diligent than good faith actors, and they will try to screw over your system for no good reason.

_sdegutis 8y ago
I've come to the conclusion that at least 1% of users are actively malicious with at least enough free time, resources, and coordination of a small full time team of professionals. I created and hosted a free self-moderated community which went well for a little while, until at least a couple people became obsessed with spamming the site with extreme gore (see the Mr. OP comment in http://old.reddit.com/r/editfight http://old.reddit.com/r/editfight which is relatively tame compared to other things they've done) and evading every single method I had of stopping or banning them, until I had to just give up altogether and shut the site down. I feel bad for the moderators on imgur.com (where I got the majority of my users) because I read somewhere that many of them now have to go to therapy for some of the things they've seen in doing their job. Online communities and the moderation of them in the modern internet is problematic enough that I personally never want to tackle that again, not even by delegating it to someone else.
- village-idiot 8y ago
  In the long run, I can’t help but wonder if creating the internet was a mistake. We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia.
  - myaccount4real 8y ago
    Mistake? At least there is visibility. I don't know what the answer to the issue is, but I know hiding it in the corner, unobserved by the masses is not the answer. This leads to NO social push back, as there is nothing to push back about.
  - xeromal 8y ago
    If you combine that, social media, and porn, how much of the internet is still 'wholesome'? I think niche forums are my favorite part of the internet left.
    - village-idiot 8y ago
      I genuinely think Social Media is going to get us all killed. Porn might prevent the next generation from being born, worst case.
      - krapp 8y ago
        >Porn might prevent the next generation from being born, worst case. You know it's possible to watch porn and have sexual relationships with other people though, right?
        village-idiot 8y ago
        Worst case, not expected case.
        krapp 8y ago
        Worst case scenarios have to at least be possible. I don't believe it's possible that internet porn can or will 'prevent the next generation from being born', so I reject it as a valid worst case.
  - krapp 8y ago
    >We’ve effectively encouraged the absolute worst elements of society to come out and have a field day without the social pushback that had kept them in line for millennia. We've also effectively given everyone else access to information, communication and opportunities to expand their worldview that have been more profoundly transformational than even the printing press. No, creating the internet was not a mistake, any more than any means of mass communication is a mistake because it can be misused as well as used wisely. Without the internet, global culture would be entirely controlled, centralized and limited by media conglomerates, digital rights management and distribution by physical media, and our means of mass communication limited to telephones. The same internet that grants freedom of expression to the "absolute worst elements of society" grants it to you and I as well. The internet is just a network, but I believe the good it's done for humanity has far outweighed the ill.
- jakobegger 8y ago
  Hey! Just wanted to let you know that I thought Editfight was something special. I came by occasionally and drew something. Too bad it's gone now -- just a few days ago my 4 year old asked me if he could play the game where you draw something on the screen again. It's a bummer you had to shut it down, but I understand.
  - _sdegutis 8y ago
    Thanks for the kind words, and glad to hear it made an impression! The code is sitting here on my computer just waiting to be deployed again, but it would have to be somewhere those demonic trolls can find. That said, they don't go to HN and I do own the domain name for another year, so maybe a subdomain is in order... EDIT: (Apparently my HN account is rate-limited so I can't reply directly.) How about this: anyone who wants access, email me and I'll send you a link. The site is up and running. People can send the link to their friends, and it'll be a private HN-only sub-community.
    - ars 8y ago
      Is there a way to federate to HN logins? Only open it to HN members? Or at least invite only?
      - sdegutis 8y ago
        I just put the site up in a subdomain specific to HN, check my post history for the link. Everyone from HN is invited, tell your friends. It'll be a mini party lol
      - jakobegger 8y ago
        I found it! https://hn.editfight.com https://hn.editfight.com
- Kagerjay 8y ago
  Reminds me of reddit r/place, first prominent artwork was Charizards penis source: was a moderator for reddit r/place
RestlessMind 8y ago
Society and Distributed Systems are similar in the sense that interesting stuff happens at the tail, and if you want to make them behave properly, you need to focus on those 1% (or 0.1% of the tail cases). For systems: queries of death, queries with highest serving latency, records with huge size compared to rest of the entries in DB, records with extremely high fan-in for your streaming pipeline, ... For society, a tiny minority of: users who would vandalize online data sets, refugees who commit terrorism / crimes, citizens committing crimes, recipients cheating welfare programs, people avoiding taxes, ... It is also the tail that imposes the maximum cost on your system / society.
- glibassertion 8y ago
  > It is also the tail that imposes the maximum cost on your system / society. This is not true for every case. Excessive focus on tail risks (and moral hazard,) in fact, can be absurdly costly.
  - village-idiot 8y ago
    See: medicare fraud prevention.
princekolt 8y ago
That is true, but at the same time, Wikipedia wouldn't be possible without it. The CEO confirmed that their absuse-detection system immediately flagged the label, but apparently some human reviewer approved it. Which shows that even if your system is perfect, humans are always the weakest link.
- village-idiot 8y ago
  Yeah, it's clear that this strategy can work, but a lot of people underestimate how much energy must be spent policing for anti-social behavior.
- rmc 8y ago
  > Wikipedia wouldn't be possible without it OpenStreetMap wouldn't be possible without it either. :) For OSM, humans are the strongest link, they are often the best way to fix these kinds of changes. A strong local community is the solution. OSM fixed this vandalism within 2 hours.

yifanl 8y ago

Correct me if I'm missing something, but what's the usecase for user submitted data on the map? Do city names change so frequently that they need to be dynamically updated? I can see some use for it when you get to the neighbourhood level to account for local nicknames, but not full cities.

mikestew 8y ago
Do city names change so frequently that they need to be dynamically updated? Maybe that's the bug. It didn't occur to me until I read your comment [0] that the system could simply accept user-submitted data only above a certain level of detail. It won't eliminate the damage, but it would keep it more contained. [0] OTOH, GIS isn't my day-to-day like it would be if I worked at, say, MapBox.
- maxerickson 8y ago
  So far human moderation has been a lot cheaper than implementing a reputation system for editing. (after all, someone has to be able to edit things like the label for New York) For instance, there's tens of millions of objects in OSM. Step one to locking some of them is deciding which ones. That alone is a big project.
lucideer 8y ago
The use case is that the entire globe is both incompletely mapped and also constantly changing: having it all editable is much easier than differentiating each and every case where some element might or might not change frequently. Such a system of differentiation of map elements by likelihood of change could be good in the long term, but it would be a huge undertaking to classify them.
Cyclone_ 8y ago
Helping label things like construction, store closures/openings and like you said local nicknames that might be hard to get data on from an official source.
- 8y ago

lqet 8y ago

It continues to amaze me that OpenStreetMap is not massively plagued by this (I know that MapBox uses OSM as one of many sources, as mentioned in the article, but the changed name did not appear in the live OSM map, afaik, so I am guessing it came from somewhere else). In the past, the main explanatory argument was always that OSM's complicated editing process was the gatekeeper, but I don't think this is true anymore. With the OSM web editor, it is extremely simple to edit the map. It would only take a few clicks to immediately publish (for example) a highway in the form of a swastika to the live map, but despite using the map multiple times per day, both for private use and professionally, I have never encountered something like this.

thinkingemote 8y ago
It was spotted and fixed within 24 hours in OSM (about 30 days ago), that's the power of a community driven map. Mapbox don't keep their maps up to date, and probably used a snapshot of the data which happened to have gotten in the few hours before it was reverted.
- lqet 8y ago
  Do you happen to know the OSM relation ID that was affected? The "New York City" relation [1] does not have anything related in its history [2] [1] https://www.openstreetmap.org/relation/175905 https://www.openstreetmap.org/relation/175905 [2] https://www.openstreetmap.org/api/0.6/relation/175905/history https://www.openstreetmap.org/api/0.6/relation/175905/histor...
  - untog 8y ago
    I wouldn't be surprised if it was purged from the history. Otherwise might it imply that New York City was once given that name, historically?
    - thinkingemote 8y ago
      OSM doesn't record historical changes of that kind (as in differentiating it from any other edit) and the nature of a change in that way. Although one might be able to infer actual geographical changes from comparisons. It just concerns itself with a snapshot of "now".

nicetryguy 8y ago

>>>/pol/

deleted 8y ago
[deleted]

sergiotapia 8y ago

Hahaha, do they have logs of who did this?

deleted 8y ago
[deleted]
Firefishy 8y ago
Yes, here is the changelog: https://www.openstreetmap.org/changeset/61555047?node_page=3 https://www.openstreetmap.org/changeset/61555047?node_page=3
- deleted 8y ago
  [deleted]
- rrdharan 8y ago
  And here’s the block of the vandal: https://www.openstreetmap.org/user_blocks/2141 https://www.openstreetmap.org/user_blocks/2141 Kind of interesting to see all the other active blocks in effect and the corresponding explanatory notes.

linsomniac 8y ago

We are just finishing building out our OSM infrastructure, and this morning in the shower I was thinking how nice it is that we can fix the maps by adding new construction and new roads (we do real estate for part of our state), and contribute this directly to OSM and then update tiles from OSM. Then I came in to read this story. Yesterday we were trying to figure out what our update strategy would be, if we would pull daily deltas, do quarterly updates, etc... We discussed the idea of vandalism and didn't have a good answer for how to prevent it. We thought about lagging behind the latest changes and then doing a "catch up" if vandalism was detected. Really what we'd like to do is make changesets based on the age of the change, except including our most recent changes (because our reputation is great :-), but I don't even know how to start doing that.

toast0 8y ago
The age of update heuristic is likely a good start; high profile vandalism like this is likely to be noticed in a reasonable amount of time. Vandalism in low profile areas is going to be harder to find though. I'm pretty sure you can get a stream of all the changes, so you would need to do something like have one copy of the database where you processed updates in near real time, and a second (production) copy where you had a delayed replication stream; in that stream, you'd want to look ahead at the real time database to calculate your stability heuristic. Alternatively, you could have the realtime database also populate a changed object queue -- anytime an object is changed pull it out of the middle of the queue and place it at the end again. If it makes it to the front, pull the current object data and place it in your production database. This should help with vandalism, but you do run the risk of accepting only parts of a coordinated edit of several objects. You could consider simply stopping replication into your production database in case anything ends up ultimately unstable, but I suspect it's hard to find a point in time where everything edited before was not edited again in the next 15 days (or whatever your threshold is). This is definitely a complex problem to solve, especially since you would want to have some escape hatches so your changes happen quickly, or you could flag upstream changes to apply quickly, or worst case, you might want to revert to an earlier version of the database. If you can figure out how to divide the data into different regions, you might have better luck, but sometimes object ids are moved around quite a bit. Geodata is already messy, and having a multitude of editors doesn't help with consistency etc.

mvexel 8y ago

Looks like https://www.openstreetmap.org/changeset/61555047 https://www.openstreetmap.org/changeset/61555047 may have been the source of this. It's the only changeset in the history of OSM for the area that contains the offending term. It's from August 10, and was reverted on August 11 in https://www.openstreetmap.org/changeset/61556585 https://www.openstreetmap.org/changeset/61556585

Firefishy 8y ago
Correct. The vandalism was reverted within 2 hours and a data update publish a minute or 2 later. Mapbox import the original vandalism change only today for some reason and there seems to have been a delay in importing the revert.