16 ms·

Microsoft Releases 125M Building Footprints in the US as Open Data

405 points by seshagiric 8y ago

doikor 8y ago
I don't understand why this is needed. Here in Finland the government just gave this info out for free to anyone (CC 4.0). They have all the information anyway because they have need for it due to planning/building permits, defense, etc. The data is very accurate (for example I can zoom into my parents cottage and see the out house on it). Later on they used laser scanning from planes to scan the whole country for very accurate topological map too. The moment this information was released for free both OSM and Google Maps quality jumped a notch or two. Before this google maps only had roads but now it has small foot paths etc too. https://www.maanmittauslaitos.fi/en/e-services/open-data-file-download-service https://www.maanmittauslaitos.fi/en/e-services/open-data-fil...
- rcdmd 8y ago
  The USA is about 30x Finland's size by landarea and building data is primarily stored at the county or equivalent level of government (3000 of those). Imagine munging data from 3000 different counties, each with their own standards and systems of recording.
  - endorphone 8y ago
    It should be trivial. It should be standardized. It is absolutely bizarre that we need to rely upon commercial companies to map things like streets and buildings when this is very accurately tracked by government. In an ideal world OSM data grids could be delegated to the appropriate governments, where it could have perfect precision with changes, street closures, one ways, redirects, etc.
    - ISL 8y ago
      There's a huge difference between tracking for building permits and opening an API to the world. As an example -- in Seattle, sewer lines are still denoted with "sewer cards", digital images of pencil drawings on hundred-year-old paper. It is sufficient for sewer records, but utterly insufficient for OSM.
      - StudentStuff 8y ago
        In Washington State, nearly all of the counties provide daily or weekly data exports for lot lines, streets and more. OSM regularly imports this data, but if you go much outside Seattle and are using Google Maps, prepare for a rubbish experience, as Google hasn't bothered to source the data directly from the counties (thus there are many missing and incorrectly drawn roads in the San Juans) and they're relying on a 3rd party vendor that is running a few years behind on pulling county data. To put this in perspective, in Kitsap county alone they've renumbered nearly 300k lots over the past few years. Google Maps doesn't get those new addresses or road names until its trickled down through USPS (who they're required to notify) then eventually to their data vendor, leaving you with a spotty patchwork of places that Google users have updated to the correct addresses. Google had a good strategy with sourcing user contributions, but pulling the public domain data that is offered freely for download and is literally the canonical source isn't hard, a small team can manage it for OSM, why is Google paying a 3rd party vendor that gives them trash instead of the latest data?
        aaronbrethorst 8y ago
        This explains why I’ve had such a miserable experience navigating the peninsula with Google turn by turn directions. Thank you for the context.
    - toast0 8y ago
      This is sometimes accurately tracked by some government, but there are thousands of governments in this country. Getting thousands of organizations to a standard is pretty hard when they don't get a concrete benefit from it. At the same time, the federal government has no business knowing the shape of my buildings, thank you very much.
    - reaperducer 8y ago
      It should be trivial. It should be standardized. Your use of the word "trivial" indicates you've never worked on complex GIS systems. Whether it should be standardized is a matter of debate. Currently, each county or state or municipality uses the software and standard that suits its needs, and more importantly -- it's budget. The GIS needs of New York City are not the same as the GIS needs of Saint George, Utah. Saint George doesn't need skyscraper functions, and New York doesn't want to measure airports on top of mesas. The real world is messy, and so is pretty much every single GIS deployment. The real world doesn't digitize well.
      - oeh34thoens 8y ago
        Likewise, New York probably doesn't have the nuclear contamination that St. George, Utah does[0]! [0] <https://en.wikipedia.org/wiki/St._George%2C_Utah#Nuclear_contamination> https://en.wikipedia.org/wiki/St._George%2C_Utah#Nuclear_con...
      - endorphone 8y ago
        "Whether it should be standardized is a matter of debate." In the same way that vaccines are `debatable'. To export an accurate, up to date model of roads (and road data) and optionally building layouts should absolutely be the norm. Nor does a standardized export format necessitate using a single uniform GIS solution, of which I've had to inter-operate with a number. The real world digitizes spectacularly well, a lot of people just make poor choices and make excuses (or worse, buy nonsensical excuses) for not helping in getting there.
        reaperducer 8y ago
        The real world digitizes spectacularly well No, it doesn't. If it did, then the surveying industry would be out of business. Yet every time a building is permitted in the United States, surveys are done. Here's an example for a recent project surveyed in Chicago just this year: West Roosevelt Road; South Clark Street; a line beginning at a point 116 feet north of vacated West 16″‘ Street as measured along the west line of South Clark Street that is westerly 135.20 feet along the arc of a circle having a radius of 375.00 feet concave northerly and whose chord bears north 79 degrees 49 minutes 52 seconds west a distance of 135.20 feet; a line north 69 degrees 46 minutes 04 seconds west a distance of 101.85 feet; a line north 69 degrees 49 minutes 57 seconds west a distance of 26.00 feet; a line along the arc of a circle having a radius of 407.80 feet concave southerly and whose chord bears north 75 degrees 52 minutes 04 seconds west a distance of 85.51 feet a distance of westerly 85.67 feet; a line north 83 degrees 47 minutes 05 seconds west a distance of 164.45 feet; a line north 69 degrees 43 minutes 24 seconds west a distance of 25.16 feet; a line north 43 degrees 07 minutes 24 seconds west a distance of 31.91 feet to a point on the easterly dock line of the former South Branch of the Chicago River; a line south 46 degrees 47 minutes 47 seconds west along the easterly dock line of the former South Branch of the Chicago River a distance of 73.33 feet; a line south 89 degrees 54 minutes 55 seconds west a distance of 32.69 feet; a line south 49 degrees 36 minutes 35 seconds a distance of 46.38 feet; a line north 89 degrees 54 minutes 55 seconds east a distance of 296.25 feet; a line easterly along the arc of a circle having a radius of 375.00 feet concave southerly and whose chord bears south 78 degrees 32 minutes 39 seconds east a distance of 109.97 feet for a distance of 110.36 feet; a line south 69 degrees 46 minutes 04 seconds east a distance o f 136.90 feet; a line easterly along the arc of a circle having a radius of 391.00 feet concave northerly and whose chord bears south 79 degrees 33 minutes 50 seconds east a distance of 135.64 feet for a distance of 136.33 feet; South Clark Street; vacated West 16″‘ Street; a line 155.40 feet west of and parallel to South Clark Street; the north line of vacated West 16″” Street; and the South Branch of the Chicago River Digitize that. Source: https://www.chicagoarchitecture.org/2018/05/29/everything-there-is-to-know-so-far-about-the-78-in-chicagos-south-loop/ https://www.chicagoarchitecture.org/2018/05/29/everything-th...
    - rmc 8y ago
      Turning OSM into, essentially a view of local governments would remove the wiki aspect, where anyone can edit it. Additionally, there is enough problems getting the global OSM community to come up with one tagging scheme. If every country made their own stuff, it would not be one map.
      - endorphone 8y ago
        Of course it would remove the wiki aspect where a better, canonical source is available. For the same reason that my local library doesn't use a wiki library catalogue. That doesn't prevent the wiki aspect for non-canonical data. I hardly see tagging standardization as a big road block.
    - JumpCrisscross 8y ago
      > It should be standardized No thank you. New York City doesn’t need a federal bureaucracy to be checked into every time a building goes up or a pipeline gets laid. And the level of detail which is necessary for New York would be overkill for rural counties; they have better things to spend their tax dollars on.
      - endorphone 8y ago
        A building going up or a pipeline getting laid necessitates an enormous amount of government activities. The idea of having a simple standardized output would be just too onerous is absolute nonsense. Further, saying that the detail necessary for NYC is overkill for a rural county is just bunk. Yes, of course NYC would have more detail. Do you think a rural county is going to be overloaded plotting their dozen streets? The whole "better things with tax dollars" bit is really the cherry on the top.
        JumpCrisscross 8y ago
        > * The idea of having a simple standardized output would be just too onerous is absolute nonsense* This complicated thing is already so complicated, it couldn’t possibly hurt to add even more complexity!
  - tomarr 8y ago
    Where does this size argument arrive from because it comes up consistently on USA v elsewhere arguments? 95th percentile densities are no more than europe. The population of USA is more than 30x Finland and we are talking about laser scanning (by EPA or equivalent)
    - cabaalis 8y ago
      The United States is comprised of literally thousands of sovereign, independent governments. There are even independent limits on what rules these governments can impose upon each other. It is not a "subordinate" type of situation. That's why the size argument comes into play so often. It's not really a landmass or even population argument as much as is it a complexity argument.
      - mark_edward 8y ago
        People like to say that, but in practice it's not true and hasn't been for over a century.
        dvdhnt 8y ago
        Yep. As an example, higher level governments will attach stipulations to grants or other types of funding in order to “encourage” lower governments to do as they’d like them to. Our tax system is such that this almost always works.
        BurritoAlPastor 8y ago
        I sadly agree that federalism is de facto dead as a framework of policy and ideology. But as a legal and bureaucratic infrastructure, it has quite refused to die, for better or for worse. City, county, and state governments still do their thing, and anybody dealing with property, construction, zoning, etc in any capacity will still need to negotiate through each of those layers separately.
        robohamburger 8y ago
        I know building permits do seem to work like this in the US. There is probably a separate db (if its even a db) for each county. It is very much a data cleaning nightmare from my limited experience.
  - chiph 8y ago
    > primarily stored at the county or equivalent level That's assuming there are any records at all. In states like Idaho and Montana, once you get out of the bigger cities I very much doubt there are permits on file for most dwellings. Partly because few people care, and partly because the land owners resent any sort of "big government" telling them what they can and cannot do.
  - deleted 8y ago
    [deleted]
- jonknee 8y ago
  Local government rules much of the US which makes national data sets difficult to assemble. There are excellent topographic resources though: https://nationalmap.gov/ https://nationalmap.gov/
- golfer 8y ago
  That's great for Finland! That is a real asset to your country. The rest of the world doesn't necessarily work the same way Finland does, unfortunately.
- erk__ 8y ago
  When the danish goverment did it some years back, they used the data to recreate the whole of denmark in minecraft [0]. [0]:https://www.bloomberg.com/news/articles/2014-05-05/to-teach-and-delight-denmark-recreates-itself-in-minecraft https://www.bloomberg.com/news/articles/2014-05-05/to-teach-...
- ebikelaw 8y ago
  There are Finland-sized entities within the USA that also do this. For example Los Angeles County, twice as large as Finland. When you are pondering the question of why all USA is not as integrated as Finland, ask yourself also why Finland is not tightly integrated with Croatia. It is the same reason.
  - deklerk 8y ago
    > For example Los Angeles County, twice as large as Finland. Uhh. You're off by two orders of magnitude. Los Angeles county is 4751 sq mi [1]. Finland is 130,666 sq mi [2]. 1: https://en.m.wikipedia.org/wiki/Los_Angeles_County,_California https://en.m.wikipedia.org/wiki/Los_Angeles_County,_Californ... 2: https://en.m.wikipedia.org/wiki/Finland https://en.m.wikipedia.org/wiki/Finland
    - et2o 8y ago
      He or she meant by population I would assume. Approximately 5 vs. 10 million.
      - bbeonx 8y ago
        Right. If one interpretation is wildly wrong and there is another blindingly obvious interpretation that is correct and relevant... Why would we ever give someone the benefit of the doubt?
        TheSpiceIsLife 8y ago
        Or refer to the community guideline: Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. Which can be found here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      - slededit 8y ago
        Population isn't very relevant when we're speaking specifically of things that relate to land area like topological maps.
- cup-of-tea 8y ago
  The US government doesn't necessarily hold data with this level of detail. If they did it would be (in theory) in the public domain anyway. Remember the difference in size between the US and European countries. As for why other governments even have to "release" such information, it's historical and surprisingly hard to change. The UK government holds possibly the most detailed map in the world covering every inch of Britain. But after 2000 when GIS based products started to take off people discovered that none of this data was available to use. OSM was created specifically to give the UK some open geographical data because there simply wasn't any and campaigns to the government were taking too long (the government data still isn't fully open).
- microtheo 8y ago
  Take the size and complexity and federation into account... In Switzerland we have map.geo.admin.ch with a highly accurate representation of the levels and buildings. I love it
  - microtheo 8y ago
    The maps are an electronic version of the one we used in the military. They look gorgeous, retro and with a clever amount of details.
- junctioniv 8y ago
  Having done a lot of research into this for a project in the US, the issue is not that they aren’t available but that each county distributed their GIS data in a different format. Some have online portals with the GIs data, some have only paper records, and some counties are missing information all together. There are companies here that work to consolidate that information but it hey charge exorbitant fees to access the data.
- rmc 8y ago
  There are numerous "open data" licences, many of which are totally incompatible with OSM. CC-BY 4.0 in this case isn't straightforwardly importable into OSM (you need to get them to sign a waiver). So that's one reason to generate our own data, rather than using "open data" from a government.
deleted 8y ago
[deleted]
kozikow 8y ago
If anyone is interested in that type of data, our startup (https://tensorflight.com https://tensorflight.com) extracts information like footprints and 10 other property factors for insurance (including bing imagery). More info: https://tensorflight.com/catalogue#Objects-supported-by-TensorFlight https://tensorflight.com/catalogue#Objects-supported-by-Tens... .
- jollivier 8y ago
  Hey ! I'll look into it with a lot of interest. Your catalog link is not very obvious in your navigation (only in text corpus). You could make it more visible in the orange bottom navbar? Cheers
kerng 8y ago
>>The CNTK toolkit developed by Microsoft is open source and available on GitHub as well. The ResNet3 model is open source and available on GitHub. Our Bing Maps computer vision team will be presenting this work at the annual International State of the Map conference in Milan, Italy Nice!
davidmurdoch 8y ago
This is pretty great! This data is usually locked up in individual scraper-unfriendly county websites, sometimes behind a paywall, sometimes only by person, or even only by mail. And the data is often limited to parcel geometry, not building footprints. The other option is to use 3rd party services to purchase or lease the data. This data can cost hundreds of thousands of dollars for the entire US dataset.
jakobegger 8y ago
When I read the part about downloading the dataset from Github, my first reaction was "Can't Microsoft afford to host downloads themselves?", and then I realized they own Github now...
- jmspring 8y ago
  Seriously, the two aren’t related. Microsoft has had public projects on Github for a couple of years now with more important projects being made public regularly. Downvoted because the cynicism and not paying attention.
TekMol 8y ago
I wonder if I ever heard about Microsoft Maps before. What is the state of it? Who uses it? So I tried www.bing.com/maps. The first thing that surprises me is that it tries to set 5 cookies from google.com and one from www.google.com What is the reason for Microsoft to allow Google to track everybody who uses their maps?
- supuun 8y ago
  I visited it's website, but didn't notice any google trackers. I'm using Privacy Badger btw. Maps app is also automatically installed on every windows 10. I think they partnered heavily with here maps in the past.
- lucb1e 8y ago
  > What is the state of it? Who uses it? In the Netherlands the businesses (shops, restaurants, and general businesses) are more complete than OpenStreetMap, but not as complete as Google because everyone sees Google as the de facto standard and they push owners to add their info in the de facto Google search. Bing Maps is fast and responsive (unlike Google Maps on anything other than their own, proprietary platforms such as Google Chrome, the Google Maps app, or Google Earth), and has aerial imagery and traffic info (unlike OSM). Bonus game: turn off labels on satellite view, zoom out to world view, and try to find your house or other places. Surprisingly hard!
justinclift 8y ago
Out of curiosity, I wonder if this data dump includes "sensitive" buildings? eg things that would generally be blurred/missing/etc on OSM for various reasons.
- maxerickson 8y ago
  If it is visible in the imagery they analyzed it wouldn't be intentionally left out of OSM without some superseding information that it didn't exist anymore.
  - justinclift 8y ago
    Makes sense, thanks. :)
rmc 8y ago
Microsoft has allowed the OSM community to trace from Bing aerial imagery for almost 10 years now. That's been a massive benefit to OSM, way more than this dump of buildings. They should be congratulated and thanked for that contribution.
- Krasnol 8y ago
  Why do you assume anyone using this data is not thankful?
  - Jonnax 8y ago
    Because consumers of the data may not know.
    - Krasnol 8y ago
      I'm sure consumers doesn't know my input there too but I don't care much. I mean...what do you expect? A banner? Money? Those who entered the data from their browser and directly used the bing data do know and I'm sure they appreciate it.
  - ddingus 8y ago
    There is no need for that assumption. This is just positive reinforcement. Maybe we see more of a good thing.
- chris_wot 8y ago
  Now this is the sort of way of handling competition I can deal with. Find the data they have locked up and that you don't compete on, but that you need to produce and collect anyway and then release it for anyone to use.

405 points by seshagiric 8y ago

I don't understand why this is needed. Here in Finland the government just gave this info out for free to anyone (CC 4.0). They have all the information anyway because they have need for it due to planning/building permits, defense, etc. The data is very accurate (for example I can zoom into my parents cottage and see the out house on it). Later on they used laser scanning from planes to scan the whole country for very accurate topological map too. The moment this information was released for free both OSM and Google Maps quality jumped a notch or two. Before this google maps only had roads but now it has small foot paths etc too. https://www.maanmittauslaitos.fi/en/e-services/open-data-file-download-service https://www.maanmittauslaitos.fi/en/e-services/open-data-fil...

rcdmd 8y ago
The USA is about 30x Finland's size by landarea and building data is primarily stored at the county or equivalent level of government (3000 of those). Imagine munging data from 3000 different counties, each with their own standards and systems of recording.
- endorphone 8y ago
  It should be trivial. It should be standardized. It is absolutely bizarre that we need to rely upon commercial companies to map things like streets and buildings when this is very accurately tracked by government. In an ideal world OSM data grids could be delegated to the appropriate governments, where it could have perfect precision with changes, street closures, one ways, redirects, etc.
  - ISL 8y ago
    There's a huge difference between tracking for building permits and opening an API to the world. As an example -- in Seattle, sewer lines are still denoted with "sewer cards", digital images of pencil drawings on hundred-year-old paper. It is sufficient for sewer records, but utterly insufficient for OSM.
    - StudentStuff 8y ago
      In Washington State, nearly all of the counties provide daily or weekly data exports for lot lines, streets and more. OSM regularly imports this data, but if you go much outside Seattle and are using Google Maps, prepare for a rubbish experience, as Google hasn't bothered to source the data directly from the counties (thus there are many missing and incorrectly drawn roads in the San Juans) and they're relying on a 3rd party vendor that is running a few years behind on pulling county data. To put this in perspective, in Kitsap county alone they've renumbered nearly 300k lots over the past few years. Google Maps doesn't get those new addresses or road names until its trickled down through USPS (who they're required to notify) then eventually to their data vendor, leaving you with a spotty patchwork of places that Google users have updated to the correct addresses. Google had a good strategy with sourcing user contributions, but pulling the public domain data that is offered freely for download and is literally the canonical source isn't hard, a small team can manage it for OSM, why is Google paying a 3rd party vendor that gives them trash instead of the latest data?
      - aaronbrethorst 8y ago
        This explains why I’ve had such a miserable experience navigating the peninsula with Google turn by turn directions. Thank you for the context.
  - toast0 8y ago
    This is sometimes accurately tracked by some government, but there are thousands of governments in this country. Getting thousands of organizations to a standard is pretty hard when they don't get a concrete benefit from it. At the same time, the federal government has no business knowing the shape of my buildings, thank you very much.
  - reaperducer 8y ago
    It should be trivial. It should be standardized. Your use of the word "trivial" indicates you've never worked on complex GIS systems. Whether it should be standardized is a matter of debate. Currently, each county or state or municipality uses the software and standard that suits its needs, and more importantly -- it's budget. The GIS needs of New York City are not the same as the GIS needs of Saint George, Utah. Saint George doesn't need skyscraper functions, and New York doesn't want to measure airports on top of mesas. The real world is messy, and so is pretty much every single GIS deployment. The real world doesn't digitize well.
    - oeh34thoens 8y ago
      Likewise, New York probably doesn't have the nuclear contamination that St. George, Utah does[0]! [0] <https://en.wikipedia.org/wiki/St._George%2C_Utah#Nuclear_contamination> https://en.wikipedia.org/wiki/St._George%2C_Utah#Nuclear_con...
    - endorphone 8y ago
      "Whether it should be standardized is a matter of debate." In the same way that vaccines are `debatable'. To export an accurate, up to date model of roads (and road data) and optionally building layouts should absolutely be the norm. Nor does a standardized export format necessitate using a single uniform GIS solution, of which I've had to inter-operate with a number. The real world digitizes spectacularly well, a lot of people just make poor choices and make excuses (or worse, buy nonsensical excuses) for not helping in getting there.
      - reaperducer 8y ago
        The real world digitizes spectacularly well No, it doesn't. If it did, then the surveying industry would be out of business. Yet every time a building is permitted in the United States, surveys are done. Here's an example for a recent project surveyed in Chicago just this year: West Roosevelt Road; South Clark Street; a line beginning at a point 116 feet north of vacated West 16″‘ Street as measured along the west line of South Clark Street that is westerly 135.20 feet along the arc of a circle having a radius of 375.00 feet concave northerly and whose chord bears north 79 degrees 49 minutes 52 seconds west a distance of 135.20 feet; a line north 69 degrees 46 minutes 04 seconds west a distance of 101.85 feet; a line north 69 degrees 49 minutes 57 seconds west a distance of 26.00 feet; a line along the arc of a circle having a radius of 407.80 feet concave southerly and whose chord bears north 75 degrees 52 minutes 04 seconds west a distance of 85.51 feet a distance of westerly 85.67 feet; a line north 83 degrees 47 minutes 05 seconds west a distance of 164.45 feet; a line north 69 degrees 43 minutes 24 seconds west a distance of 25.16 feet; a line north 43 degrees 07 minutes 24 seconds west a distance of 31.91 feet to a point on the easterly dock line of the former South Branch of the Chicago River; a line south 46 degrees 47 minutes 47 seconds west along the easterly dock line of the former South Branch of the Chicago River a distance of 73.33 feet; a line south 89 degrees 54 minutes 55 seconds west a distance of 32.69 feet; a line south 49 degrees 36 minutes 35 seconds a distance of 46.38 feet; a line north 89 degrees 54 minutes 55 seconds east a distance of 296.25 feet; a line easterly along the arc of a circle having a radius of 375.00 feet concave southerly and whose chord bears south 78 degrees 32 minutes 39 seconds east a distance of 109.97 feet for a distance of 110.36 feet; a line south 69 degrees 46 minutes 04 seconds east a distance o f 136.90 feet; a line easterly along the arc of a circle having a radius of 391.00 feet concave northerly and whose chord bears south 79 degrees 33 minutes 50 seconds east a distance of 135.64 feet for a distance of 136.33 feet; South Clark Street; vacated West 16″‘ Street; a line 155.40 feet west of and parallel to South Clark Street; the north line of vacated West 16″” Street; and the South Branch of the Chicago River Digitize that. Source: https://www.chicagoarchitecture.org/2018/05/29/everything-there-is-to-know-so-far-about-the-78-in-chicagos-south-loop/ https://www.chicagoarchitecture.org/2018/05/29/everything-th...
  - rmc 8y ago
    Turning OSM into, essentially a view of local governments would remove the wiki aspect, where anyone can edit it. Additionally, there is enough problems getting the global OSM community to come up with one tagging scheme. If every country made their own stuff, it would not be one map.
    - endorphone 8y ago
      Of course it would remove the wiki aspect where a better, canonical source is available. For the same reason that my local library doesn't use a wiki library catalogue. That doesn't prevent the wiki aspect for non-canonical data. I hardly see tagging standardization as a big road block.
  - JumpCrisscross 8y ago
    > It should be standardized No thank you. New York City doesn’t need a federal bureaucracy to be checked into every time a building goes up or a pipeline gets laid. And the level of detail which is necessary for New York would be overkill for rural counties; they have better things to spend their tax dollars on.
    - endorphone 8y ago
      A building going up or a pipeline getting laid necessitates an enormous amount of government activities. The idea of having a simple standardized output would be just too onerous is absolute nonsense. Further, saying that the detail necessary for NYC is overkill for a rural county is just bunk. Yes, of course NYC would have more detail. Do you think a rural county is going to be overloaded plotting their dozen streets? The whole "better things with tax dollars" bit is really the cherry on the top.
      - JumpCrisscross 8y ago
        > * The idea of having a simple standardized output would be just too onerous is absolute nonsense* This complicated thing is already so complicated, it couldn’t possibly hurt to add even more complexity!
- tomarr 8y ago
  Where does this size argument arrive from because it comes up consistently on USA v elsewhere arguments? 95th percentile densities are no more than europe. The population of USA is more than 30x Finland and we are talking about laser scanning (by EPA or equivalent)
  - cabaalis 8y ago
    The United States is comprised of literally thousands of sovereign, independent governments. There are even independent limits on what rules these governments can impose upon each other. It is not a "subordinate" type of situation. That's why the size argument comes into play so often. It's not really a landmass or even population argument as much as is it a complexity argument.
    - mark_edward 8y ago
      People like to say that, but in practice it's not true and hasn't been for over a century.
      - dvdhnt 8y ago
        Yep. As an example, higher level governments will attach stipulations to grants or other types of funding in order to “encourage” lower governments to do as they’d like them to. Our tax system is such that this almost always works.
      - BurritoAlPastor 8y ago
        I sadly agree that federalism is de facto dead as a framework of policy and ideology. But as a legal and bureaucratic infrastructure, it has quite refused to die, for better or for worse. City, county, and state governments still do their thing, and anybody dealing with property, construction, zoning, etc in any capacity will still need to negotiate through each of those layers separately.
      - robohamburger 8y ago
        I know building permits do seem to work like this in the US. There is probably a separate db (if its even a db) for each county. It is very much a data cleaning nightmare from my limited experience.
- chiph 8y ago
  > primarily stored at the county or equivalent level That's assuming there are any records at all. In states like Idaho and Montana, once you get out of the bigger cities I very much doubt there are permits on file for most dwellings. Partly because few people care, and partly because the land owners resent any sort of "big government" telling them what they can and cannot do.
- deleted 8y ago
  [deleted]
jonknee 8y ago
Local government rules much of the US which makes national data sets difficult to assemble. There are excellent topographic resources though: https://nationalmap.gov/ https://nationalmap.gov/
golfer 8y ago
That's great for Finland! That is a real asset to your country. The rest of the world doesn't necessarily work the same way Finland does, unfortunately.
erk__ 8y ago
When the danish goverment did it some years back, they used the data to recreate the whole of denmark in minecraft [0]. [0]:https://www.bloomberg.com/news/articles/2014-05-05/to-teach-and-delight-denmark-recreates-itself-in-minecraft https://www.bloomberg.com/news/articles/2014-05-05/to-teach-...
ebikelaw 8y ago
There are Finland-sized entities within the USA that also do this. For example Los Angeles County, twice as large as Finland. When you are pondering the question of why all USA is not as integrated as Finland, ask yourself also why Finland is not tightly integrated with Croatia. It is the same reason.
- deklerk 8y ago
  > For example Los Angeles County, twice as large as Finland. Uhh. You're off by two orders of magnitude. Los Angeles county is 4751 sq mi [1]. Finland is 130,666 sq mi [2]. 1: https://en.m.wikipedia.org/wiki/Los_Angeles_County,_California https://en.m.wikipedia.org/wiki/Los_Angeles_County,_Californ... 2: https://en.m.wikipedia.org/wiki/Finland https://en.m.wikipedia.org/wiki/Finland
  - et2o 8y ago
    He or she meant by population I would assume. Approximately 5 vs. 10 million.
    - bbeonx 8y ago
      Right. If one interpretation is wildly wrong and there is another blindingly obvious interpretation that is correct and relevant... Why would we ever give someone the benefit of the doubt?
      - TheSpiceIsLife 8y ago
        Or refer to the community guideline: Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. Which can be found here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - slededit 8y ago
      Population isn't very relevant when we're speaking specifically of things that relate to land area like topological maps.
cup-of-tea 8y ago
The US government doesn't necessarily hold data with this level of detail. If they did it would be (in theory) in the public domain anyway. Remember the difference in size between the US and European countries. As for why other governments even have to "release" such information, it's historical and surprisingly hard to change. The UK government holds possibly the most detailed map in the world covering every inch of Britain. But after 2000 when GIS based products started to take off people discovered that none of this data was available to use. OSM was created specifically to give the UK some open geographical data because there simply wasn't any and campaigns to the government were taking too long (the government data still isn't fully open).
microtheo 8y ago
Take the size and complexity and federation into account... In Switzerland we have map.geo.admin.ch with a highly accurate representation of the levels and buildings. I love it
- microtheo 8y ago
  The maps are an electronic version of the one we used in the military. They look gorgeous, retro and with a clever amount of details.
junctioniv 8y ago
Having done a lot of research into this for a project in the US, the issue is not that they aren’t available but that each county distributed their GIS data in a different format. Some have online portals with the GIs data, some have only paper records, and some counties are missing information all together. There are companies here that work to consolidate that information but it hey charge exorbitant fees to access the data.
rmc 8y ago
There are numerous "open data" licences, many of which are totally incompatible with OSM. CC-BY 4.0 in this case isn't straightforwardly importable into OSM (you need to get them to sign a waiver). So that's one reason to generate our own data, rather than using "open data" from a government.

deleted 8y ago

[deleted]

kozikow 8y ago

If anyone is interested in that type of data, our startup (https://tensorflight.com https://tensorflight.com) extracts information like footprints and 10 other property factors for insurance (including bing imagery). More info: https://tensorflight.com/catalogue#Objects-supported-by-TensorFlight https://tensorflight.com/catalogue#Objects-supported-by-Tens... .

jollivier 8y ago
Hey ! I'll look into it with a lot of interest. Your catalog link is not very obvious in your navigation (only in text corpus). You could make it more visible in the orange bottom navbar? Cheers

kerng 8y ago

>>The CNTK toolkit developed by Microsoft is open source and available on GitHub as well. The ResNet3 model is open source and available on GitHub. Our Bing Maps computer vision team will be presenting this work at the annual International State of the Map conference in Milan, Italy Nice!

davidmurdoch 8y ago

This is pretty great! This data is usually locked up in individual scraper-unfriendly county websites, sometimes behind a paywall, sometimes only by person, or even only by mail. And the data is often limited to parcel geometry, not building footprints. The other option is to use 3rd party services to purchase or lease the data. This data can cost hundreds of thousands of dollars for the entire US dataset.

jakobegger 8y ago

When I read the part about downloading the dataset from Github, my first reaction was "Can't Microsoft afford to host downloads themselves?", and then I realized they own Github now...

jmspring 8y ago
Seriously, the two aren’t related. Microsoft has had public projects on Github for a couple of years now with more important projects being made public regularly. Downvoted because the cynicism and not paying attention.

TekMol 8y ago

I wonder if I ever heard about Microsoft Maps before. What is the state of it? Who uses it? So I tried www.bing.com/maps. The first thing that surprises me is that it tries to set 5 cookies from google.com and one from www.google.com What is the reason for Microsoft to allow Google to track everybody who uses their maps?

supuun 8y ago
I visited it's website, but didn't notice any google trackers. I'm using Privacy Badger btw. Maps app is also automatically installed on every windows 10. I think they partnered heavily with here maps in the past.
lucb1e 8y ago
> What is the state of it? Who uses it? In the Netherlands the businesses (shops, restaurants, and general businesses) are more complete than OpenStreetMap, but not as complete as Google because everyone sees Google as the de facto standard and they push owners to add their info in the de facto Google search. Bing Maps is fast and responsive (unlike Google Maps on anything other than their own, proprietary platforms such as Google Chrome, the Google Maps app, or Google Earth), and has aerial imagery and traffic info (unlike OSM). Bonus game: turn off labels on satellite view, zoom out to world view, and try to find your house or other places. Surprisingly hard!

justinclift 8y ago

Out of curiosity, I wonder if this data dump includes "sensitive" buildings? eg things that would generally be blurred/missing/etc on OSM for various reasons.

maxerickson 8y ago
If it is visible in the imagery they analyzed it wouldn't be intentionally left out of OSM without some superseding information that it didn't exist anymore.
- justinclift 8y ago
  Makes sense, thanks. :)

rmc 8y ago

Microsoft has allowed the OSM community to trace from Bing aerial imagery for almost 10 years now. That's been a massive benefit to OSM, way more than this dump of buildings. They should be congratulated and thanked for that contribution.

Krasnol 8y ago
Why do you assume anyone using this data is not thankful?
- Jonnax 8y ago
  Because consumers of the data may not know.
  - Krasnol 8y ago
    I'm sure consumers doesn't know my input there too but I don't care much. I mean...what do you expect? A banner? Money? Those who entered the data from their browser and directly used the bing data do know and I'm sure they appreciate it.
- ddingus 8y ago
  There is no need for that assumption. This is just positive reinforcement. Maybe we see more of a good thing.
chris_wot 8y ago
Now this is the sort of way of handling competition I can deal with. Find the data they have locked up and that you don't compete on, but that you need to produce and collect anyway and then release it for anyone to use.