3 ms·

That's the status quo. I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hit

That's the status quo. I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hits the benchmarks is acceptable. I don't regard this as a philosophical matter - let's accept any bot analysis that hits a certain threshold. 1) We can certainly set the threshold above the TIGER import in the US, which most people agree is positive, but very crufty. 2) It's the developing parts of the world that need automated road creation the most. The opposition seems to emanate from rich, European nations.

lucb1e 10y ago
Robot edits are great when they add data. When modifying or removing work that humans did, I'm much more hesitant. This is probably why there's opposition from "rich, European nations": the developing world has little data that anything could go wrong with. I'm not necessarily pro autonomous robots unless they either hit very high benchmarks, or it's purely for initial groundwork (or improving on previous bots' work). I totally see how it could be great for unmapped places where it could add grass/forest areas, houses, roads, rivers, etc. but tampering with existing data with no review queue (if not an approval queue, at least post-editing review)... I'm hesitant.
- andrewljohnson 10y ago
  Again, I wouldn't be philosophical about "When modifying or removing work that humans did, I'm much more hesitant." If a robotic system can demonstrate some high bar of precision in modifying human updates, then let the edits happen. If a human mapped a road, then that road changed places or perhaps changed from being two-way to one-way and a robot can detect that reliably, let the robot edit. That said, I don't think "robots changing human edits" is where the robots shine. Most human edits will be good, unless the data later changes. I agree that the low hanging fruit is 1) mapping new features, 2) cleaning up imports like TIGER, 3) providing "lint" type tools to give human editors feedback if their new edit might be wrong.
  - maxerickson 10y ago
    I'm curious how the lint aspect of DeepOSM will work in areas that have less room for improvement than the areas I looked at in Delaware and Maine. In those areas, the imported TIGER data is poor enough that a road by road analysis is overkill. The DeepOSM output flagged most of the roads. Given the capacity to run it, an interesting analysis would be an evaluation of the entire United States, maybe grouped by county or so. Then people in well mapped cities looking for a project could improve the worst county in their state, or the worst county in the US, etc.
  - manarth 10y ago
    > "Most human edits will be good, unless the data later changes" One concern is the date of the bot's data-source/satellite imagery. - Bot reviews satellite data, creates building. - Human notices that the building's actually been demolished, so removes it from OSM. - Bot reviews satellite data, recreates building. When I was in Georgia (the Eurasian country, not the USA state) in 2014, I was making changes to OSM and noticed that the satellite imagery was very different to what was on the ground. I couldn't find any way to work out how old the satellite data was, but my gut feel is that there will be some places where the pace of development is much faster than the rate of updates in satellite imagery.
mynewtb 10y ago
The TIGER import is about 10 years old and still not cleaned up. > It's the developing parts of the world that need automated road creation the most. The opposition seems to emanate from rich, European nations. I would guess that the majority of the OSM community is from rich, European nations. If people need open road data then why not let/make them create it themselves? Using tools to suggest changes is great but without local knowledge we get bad edits like for example some contributions by Facebook or Mapbox who seem to think their US offices know better than the locals...
zerebubuth 10y ago
> I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hits the benchmarks is acceptable. I think the benchmark rate is "indistinguishable from a human", a sort of OSM Turing test, perhaps. This would build in a few important safety features: 1. Human quality, which generally (but not always) means no egregious breakage. Paying some attention to the linters, for example. 2. Human presence, which means it's possible to have a conversation about the mapper's or bot's activities with that entity, or at least people in control of it. 3. Human timescales, so that it's possible to have that conversation before it's all over. > The opposition seems to emanate from rich, European nations. Personally, I think the difference in approach may reflect the difference between those who want to build a map as fast as possible and those who want to build a sustainable community (which makes and maintains a map). The push towards more bulk imports seems to emanate from the USA, with Facebook testing their import in Egypt. I wonder whether they had a discussion with the Egyptian community first, or just went ahead because they thought knew best what developing parts of the world need?

lucb1e 10y ago

Robot edits are great when they add data. When modifying or removing work that humans did, I'm much more hesitant. This is probably why there's opposition from "rich, European nations": the developing world has little data that anything could go wrong with. I'm not necessarily pro autonomous robots unless they either hit very high benchmarks, or it's purely for initial groundwork (or improving on previous bots' work). I totally see how it could be great for unmapped places where it could add grass/forest areas, houses, roads, rivers, etc. but tampering with existing data with no review queue (if not an approval queue, at least post-editing review)... I'm hesitant.

andrewljohnson 10y ago
Again, I wouldn't be philosophical about "When modifying or removing work that humans did, I'm much more hesitant." If a robotic system can demonstrate some high bar of precision in modifying human updates, then let the edits happen. If a human mapped a road, then that road changed places or perhaps changed from being two-way to one-way and a robot can detect that reliably, let the robot edit. That said, I don't think "robots changing human edits" is where the robots shine. Most human edits will be good, unless the data later changes. I agree that the low hanging fruit is 1) mapping new features, 2) cleaning up imports like TIGER, 3) providing "lint" type tools to give human editors feedback if their new edit might be wrong.
- maxerickson 10y ago
  I'm curious how the lint aspect of DeepOSM will work in areas that have less room for improvement than the areas I looked at in Delaware and Maine. In those areas, the imported TIGER data is poor enough that a road by road analysis is overkill. The DeepOSM output flagged most of the roads. Given the capacity to run it, an interesting analysis would be an evaluation of the entire United States, maybe grouped by county or so. Then people in well mapped cities looking for a project could improve the worst county in their state, or the worst county in the US, etc.
- manarth 10y ago
  > "Most human edits will be good, unless the data later changes" One concern is the date of the bot's data-source/satellite imagery. - Bot reviews satellite data, creates building. - Human notices that the building's actually been demolished, so removes it from OSM. - Bot reviews satellite data, recreates building. When I was in Georgia (the Eurasian country, not the USA state) in 2014, I was making changes to OSM and noticed that the satellite imagery was very different to what was on the ground. I couldn't find any way to work out how old the satellite data was, but my gut feel is that there will be some places where the pace of development is much faster than the rate of updates in satellite imagery.

mynewtb 10y ago

The TIGER import is about 10 years old and still not cleaned up. > It's the developing parts of the world that need automated road creation the most. The opposition seems to emanate from rich, European nations. I would guess that the majority of the OSM community is from rich, European nations. If people need open road data then why not let/make them create it themselves? Using tools to suggest changes is great but without local knowledge we get bad edits like for example some contributions by Facebook or Mapbox who seem to think their US offices know better than the locals...

zerebubuth 10y ago

> I'm in favor of a regime where there is a benchmark for the recall and precision rates for the automated analysis, and anything that hits the benchmarks is acceptable. I think the benchmark rate is "indistinguishable from a human", a sort of OSM Turing test, perhaps. This would build in a few important safety features: 1. Human quality, which generally (but not always) means no egregious breakage. Paying some attention to the linters, for example. 2. Human presence, which means it's possible to have a conversation about the mapper's or bot's activities with that entity, or at least people in control of it. 3. Human timescales, so that it's possible to have that conversation before it's all over. > The opposition seems to emanate from rich, European nations. Personally, I think the difference in approach may reflect the difference between those who want to build a map as fast as possible and those who want to build a sustainable community (which makes and maintains a map). The push towards more bulk imports seems to emanate from the USA, with Facebook testing their import in Egypt. I wonder whether they had a discussion with the Egyptian community first, or just went ahead because they thought knew best what developing parts of the world need?