17 ms·
This post seems half right, half wrong. Wrong that OSM should be a service provider and not a database - I think it should be a database, and corporations are
by andrewljohnson 9y ago
This post seems half right, half wrong.
Wrong that OSM should be a service provider and not a database - I think it should be a database, and corporations are good for OSM. The post calls for OSM to essentially become Mapbox. It's insanely expensive to be Mapbox, and Mapzen just failed trying to be a less corporate version of Mapbox. And then the usage policy part of the article is moot or at least not that consequential if you don't believe OSM should be a service provider.
Right: Moderation tools need to be created/fixed. New mappers need to be encouraged/onboarded better. Vandalism is a problem. OSMF culture is toxic. Hidden gatekeepers are toxic. Imports need to be supported. Bots need to be enabled, and we need to consider people who contribute code/tech/imports first class OSM contributors and not just glorify the individual mapper.
Probably right: OSM needs layers, or even a more sweeping re-architecture to allow better versioning, moderation, better tooling, and easier understanding of the data.
Probably wrong: Expanding the scope of OpenStreetMap to include transient data. Too much other work to do, maybe later for this.
- emacsen 9y ago> The post calls for OSM to essentially become Mapbox The problem is that most companies using OSM are using Mapbox. This leads to a few problems, including attribution, which I decided not to touch on, but is a serious problem that Mapbox doesn't address sufficiently. If they could use OSM directly, they'd be engaged with OSM directly. They'd care about OSM. Right now OSM shoves these users away and makes them into customers of some third party (maybe Mapbox, maybe someone else) instead of caring about OSM itself. > Expanding the scope of OpenStreetMap to include transient data. Too much other work to do, maybe later for this. How about letting users experiment and seeing what cool stuff they can come up with?
- andrewljohnson 9y ago> The problem is that most companies using OSM are using Mapbox. Is that even true? And is number of companies even a good metric? Based on revenue/market cap, the bulk of companies use OSM directly. The world's biggest company (Apple) uses OSM directly. Telenav uses it directly. Don't Tesla and Uber use OSM directly too? And we use Mapbox at Gaia GPS, but we also use OSM data directly. > How about letting users experiment and seeing what cool stuff they can come up with? In my experience building a company, if you focus on too many things, you get everything wrong. This smells to me of being not on target for making OSM as strong as possible, even though it has alluring upside.
- y4mi 9y ago> The world's biggest company (Apple) uses OSM directly sigh apple zealots... its big, i give you that, but sure as hell ain't the biggest. https://en.wikipedia.org/wiki/List_of_largest_companies_by_revenue https://en.wikipedia.org/wiki/List_of_largest_companies_by_r...
- andrewljohnson 9y agoLargest by market cap: https://en.wikipedia.org/wiki/List_of_public_corporations_by_market_capitalization https://en.wikipedia.org/wiki/List_of_public_corporations_by...
- y4mi 9y agoa) biggest company does not equal company with highest market cap b) having the biggest market cap is something thats good for their shareholder, and terrible for anyone else. But yeah, do continue to praise your beloved highness.
- coldtea 9y ago>a) biggest company does not equal company with highest market cap Actually it does. Especially since you can have 1 billion in revenue by selling $10 dollar bills at $1 dollar. >b) having the biggest market cap is something thats good for their shareholder, and terrible for anyone else Whatever. >But yeah, do continue to praise your beloved highness. Irrational argument ad hominem.
- djsumdog 9y agoI mean, I realize hosting a service is expensive, but can't there be an open source set of service APIs that's self hosted? That way you can chose to get a full copy of the data (and be responsible for pulling updates) and the API, or just pay someone else for access to their hosted version of the API?
- michaelt 9y agocan't there be an open source set of service APIs that's self hosted OSM does have open source self-hosted options. Unfortunately, the developer experience is much more painful than using Google Maps. With Google Maps, if you can add Javascript to a web page you're basically good to go. Data updates, software updates, server redundancy, server support and security, all taken care of. Even if you're exclusively a front-end dev, you're making a static site, and you've never heard of Linux. Compared to that, self-hosting OSM tile generation requires a bunch of extra knowledge, equipment and maintenance.
- chuckdries 9y agoIs it something that could be containerized? Maybe not as simple as a JS embed, but deploying a self hosted docker container is far easier than installing most things I've encountered.
- petre 9y agoWith vector tiles one doesn't have to containerize anything - just shard them onto a bunch of servers and server them over https. Put an accelerator/cache in front and proxy to the desired region backend. With image tiles it's a hurdle. You have to build 18 zoom levels of tiles. The most detailed layers have an astronomic number if tiles. You also need a huge amount if storage. http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=5.538062,47.236312,15.371071,54.954 http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox... Did I mention the data being huge? What's the point of having 15Tb container, which takes forever to deploy? You can certainly containerize the services, but there's no benefit containerizing the data, since you have to replace it all at once rather than synchronizing the differences.
- mjevans 9y agoTransit data should probably be supported in a meta-sense. A machine readable bounded region with annotations about where to fetch transit data and how to parse it should likely be added. That way the OSM database links to external data sources.
- pmezard 9y agoMapzen tried to be a less corporate version of Mapbox but the quality of their data/tools was just not there. I had to implemented a specialized geocoder/administrative boundaries referential last year. Mapzen was really compelling, the documentation about how they handled hierarchical data was clear and interesting but execution was lacking. I am French, handling European data was important and I quickly hit several pain points: - Their github repository organization was a mess. A lot of small ones without a clear map of which did what. - The vagrant setup to experiment locally was unmaintained - I found a basic unicode normalization issue in the geocoder several hours after playing with it. - Manipulating their dataset via git-lfs was a massive pain - Their geocoder was nicely presented but relations to administrative levels was done with neightbourhood search to administrative area centroids, instead of (even approximated) geometries. - The data was not as comprehensive and up to date as OSM one, at least in France. For instance, France second level administrative areas (Régions) were redesigned at the beginning of 2016. They were defined in OSM but progress seemed fairly slow in Mapzen. Now, I appreciate Mapzen was a very ambitious project and handling cartographic data at this scope and scale is a difficult job, but as pure data consumer, it was not comparable to OSM. I eventually used OSM data and while their documentation/tools are quite a mess too, wiki style, it is workable. Datasets are large but there are a lot of way to retrieve them efficiently. Their non-protobuf binary format hits a sweet spot between size efficiency and ease of processing. And at least in Europe, the data is fairly good, at least for my requirements.
- WhitneyLand 9y agoYou make the point OSM should be a database and not a service, but that’s a false dichotomy as nothing inherently prevents it from being both. The main point against a service seems to be that’s it’s an expensive thing to do, however I don’t see how we can draw any conclusion from that without weighing the costs against the upsides. I don’t see how the expense argument can be made with confidence without first establishing some basic data points. For example: 1) How expensive is expensive? 2) Have the benefits and advantages been throughly explored ie what’s the value proposition? 3) Given this value proposition, what level of fund raising success would be plausible? As an aside, the ineffectiveness of leadership and how it is seemingly blocking progress is quite striking. Off the top of my head, I can’t think of any actively developed software with a non-deprecated api that hasn’t changed in so many years.
- Doctor_Fegg 9y agoOSM has gained countless new capabilities since moving to API 0.6 in 2009. World-class, super-fast routing. Beautiful vector maps. A much easier raster map API. A powerful and performant query language. To say that progress is being "blocked" is not at all accurate. Sure, these aren't provided by what's formally known as the "OSM API". They are provided by OSRM/Graphhopper, Mapbox GL/Tangram, Leaflet/OpenLayers 3, and Overpass respectively. Because that's, deliberately, how OSM works: it's the centre of an ecosystem, not a single service provider. The OSM API is for the use of map editing programs. It's an implementation detail for the authors of iD, JOSM, Vespucci and a few others. It isn't something that anyone who isn't writing an editor needs to concern themselves about; and I honestly haven't heard many comments from editor maintainers that they want the API to do things it doesn't at present (though, Bryan, Dirk, Simon, do jump in and correct me if I'm wrong).
- WhitneyLand 9y agoDo you have any connection to any company involved with OSM tech, just so I understand your perspective? Mentioning OSM has gained functionality is irrelevant, the article’s point is the API which serves a different purpose. What other companies offer is irrelevant that’s not OSM. Your other points seem to attempt to diminish the importance of the API, yet the state of modern software has become such that almost any significant software project can benefit from a rich, well implemented and maintained API. Not to mention maintaining a high quality API often in and of itself leads to improvements in the overall architecture of software, due to how it encourages careful thought about the purpose and organization of the system.
- johannes1234321 9y ago> Imports need to be supported. I don't really agree. I have seen so many wrong imports and I have seen imports demotivating individual mappers. With a good data source an import can be a good start, but it's not a silver bullet. I agree to your other items, though.
- petre 9y agoWe wanted to contribute forest data on Africa back in 2015. It wasn't possible. We'd have to add it via JOSM or the web based editor. So we gave up since it was a no go. Now we just discard the OSM forest data and add our own when updating the map data.
- alexchamberlain 9y agoAre you not violating the licence in this case? Or do you make the new dataset available under the same licence? I think the licence is one of OSM’s big problems.
- rmc 9y ago> It wasn't possible. We'd have to add it via JOSM or the web based editor. That's strange. What were you trying to do? If you convert your data to OSM's relatively simple XML format, you can open that file in JOSM, and just press the upload button. That's what most people do with imports. It's possible to communicate with the OSM API directly and make API calls, but that's essentially what JOSM does. JOSM also has a validator, which can spot some problems with your data.
- pbowyer 9y ago> Probably right: OSM needs layers, or even a more sweeping re-architecture to allow better versioning, moderation, better tooling, and easier understanding of the data. I have a hobby site about railways [0] I'm picking up again after a decade of inactivity. I wanted to display a route map of each railway, with clickable stations and the route highlighted. I can not work out how to do that. OSM has the route and station data (there and in Open Railway Map [1]). I expected to find a layer called 'Railways' with sub-layers, one for each railway, and I could export the data and highlight it on another map. Instead I have raster tiles, and no way to highlight the railway I'm talking about. I know the data must be there somewhere but how on earth do I get it? [0] https://www.narrow-gauge.co.uk https://www.narrow-gauge.co.uk [1] https://www.openrailwaymap.org/ https://www.openrailwaymap.org/
- ris 9y agoInvestigate Overpass and in particular Overpass Turbo. And really, any strict layering system in OSM would quickly just fall apart into debates of exactly what belonged in each layer. The range of detail captured by OpenStreetMap users in different areas is really far too nuanced to work crammed into coarse, crude "layer" classifications.
- mynameisvlad 9y agoHow can that be the case? Even with a range of detail, the points on a map have to represent one of a fairly limited selection of types. Even if you keep it high level like "buildings", "roads", etc. it's still a massive improvement over having no layers whatsoever.
- ris 9y ago> the points on a map have to represent one of a fairly limited selection of types Absolutely not. Not only is the selection of types huge and full of shades of grey, but an element can be tagged as things from multiple different categories at once (and this is a wonderful feature). Take a browse of the OpenStreetMap wiki sometime. OpenStreetMap data is far richer than a layer model could cope with. However, let's not confuse this with an editor having a layers-like interface feature to help the user focus on a particular subset of the data. JOSM for instance does this very successfully with the "filters" tool, which I use heavily.
- zokier 9y ago> Probably wrong: Expanding the scope of OpenStreetMap to include transient data. Too much other work to do, maybe later for this. Maybe I'm misunderstanding the whole domain, but if OSM would get layers, that would lead to better composability, then wouldn't the question of transient/frivolous data kinda become moot? The way I see it, in that scenario I could make and host my own whatever pokemon layer and OSM proper could provide the base layer, and then they could be easily merged at usage point; whereas based on the discussion now without layers you'd need to essentially make hard fork of the OSM data to make your own pokemon map?
- frenchy 9y agoPerhaps I'm misunderstanding your goal, but how could this not be achieved with a separate database, and overlay your pokemon on-top of the OSM map?
- HNSucksAss 9y agoTransient data do not belong in the database. Who's going to run around and clean all of it up, then? Temporary flight restrictions aren't added to FAA charts (unless they're bullshit "temporary" restrictions like the one around Disneyland), nor should they be.
- gg3 9y agoTest
- HNSucksAss 9y agoTotally agree. The complaints about the database nature of the project are off-base. Also, the quality of the data sucks. The author points out the lack of standardization, but also nonsensical terms (tags) are used in the database itself (road-type classifications make no sense, for example). I guess someone needs to write code to do a massive sweep through the dataset and attempt to correct some of these problems while generating layers.
- cptskippy 9y ago> Probably wrong: Expanding the scope of OpenStreetMap to include transient data. Too much other work to do, maybe later for this. Isn't all of the data in OSM transient to some degree? What's the threshold for when data is too fleeting to be worth storing? Days, months, years, decades?