6 ms·
Show HN: Simple Mbtiles Server – Self-host the entire planet of OpenStreetMaps
- archerx 2y agoI wish the readme explained what this project actually was instead of the incredibly vague and unhelpful "self host the entire planet".
- basemi 2y agoFound little more info in homepage https://github.com/markuman/sms https://github.com/markuman/sms
- fodkodrasz 2y agoStill barely more detailed. The problem with the whole OSM ecosystem is that it expects a lot of domain knowledge of technologies, formats, tools, their relations and features, limitations, that is rarely explained well, or explanaitions are often terribly outdated. Feels much like the JavaScript Frontend frameworks for an outsider, it is a lot of effort to get to an overview and understanding, despite the concepts are not new.
- davidy123 2y agoUsually this is a sign of a project that isn't ready for a lot of attention. It's very brave and a little bit stupid to offer Open Source to the world (especially without a sustainability plan), if you put a lot of effort into 'marketing' at an early stage you've probably just overpromised to people not focused on that domain, won't be able to deliver, and will burn out, and it will become yet another abandoned project. Perhaps in a few years, as the project picks up uses and contributors, it will get its own website and support network and be easy to understand and use by "anyone."
- bspammer 2y agoI can’t tell if your comment was meant to be ironic but OSM has been around for 20 years, and its data is used by billions of people.
- davidy123 2y agoThe comment was about https://github.com/markuman/sms https://github.com/markuman/sms, not OSM.
- Guvante 2y agoI don't think this project is looking for a lot of attention. It is more like a sample of cool stuff you can do.
- fodkodrasz 2y agoI agree, but still 1-2 paragraph about why this was made, what is for laymans, and what it is in terms of OSM ecosystem. Possibly my extrovert self makes it natural to write similar intros to repos I make public. (I even used to blog!)
- PontifexMinimus 2y ago> incredibly vague and unhelpful As is very common with projects on Github. They should have a paragraph at the start explaining simply what they do.
- MrGilbert 2y agoAnd a screenshot for every project with a UI. I‘d love to see that more often.
- deleted 2y ago[deleted]
- totaldude87 2y agoCopilot could easily scan and auto generate the read me.
- toolz 2y agoThey 'could' have a paragraph at the beginning to help potential users understand the purpose of the project. It might seem inconsequential, but maintainers often deal with a lot of angry entitled users and it could be helpful to stay away from language that implies that the people giving away their code for free (and for fun) have any obligation to provide anyone anything.
- squarefoot 2y agoTrue on principle, but if an explanation or a screenshot that costs 5 hours to make would prevent replying or even just reading support requests for 6 hours, I would call it a victory.
- EGreg 2y agoThat’s true, I was super confused what “self-host” entailed. They should go into detail explaining that. Leaflet? Because I understood about “the whole planet” in the context of OSM.
- Aachen 2y agoAnyone looking for this will already be in map/OpenStreetMap context. That people originating from StumbleUpon or a niche little place called ycombinator news are confused is not too surprising tbh: that very likely wasn't the audience they had in mind
- input_sh 2y agoConsidering the title starts with "Show HN" and that the HN username is very closely tied to the GitHub username, I'm gonna go out on a limb here by saying HN is indeed the target audience.
- rizky05 2y ago[dead]
- deleted 2y ago[deleted]
- 808essa 2y agoexplanation of what this is would be nice
- orblivion 2y agoOpenStreetMap is a map ecosystem based on an open data set. There are numerous redundant ways of achieving the same thing with this data, whether it's hosting the "tiles" (the thing you see on the map), search, UI, location bookmarking, navigation, etc. This is great for flexibility, but it's a difficult thing to dive into and understand. With that said, the "tile hosting" part is notoriously difficult to actually self-host. It's a lot of data, it traditionally took a lot of time to transform the raw data into the png files representing the tiles. Many if not most web apps actually call out to openstreetmap.org or similar to do the tiles for them. That's not fully self-hosting. But more recently, there have been new ways to do tiles, namely with vector formats instead of pngs. They take up less space, they're easier to share around. So this submission appears to be a simple few lines to get your fully self-hosted tile server of the whole planet up and running. You should then be able to use this in whatever web app you're making to make it fully self hosted (though if you wanted self-hosted search, that's another hard one...).
- elashri 2y agoEntire planet maps/topology I guess. But my mind had much much more ambitious first thought.
- fareesh 2y agoLooks like every single "tile" image from OpenStreetMaps (OSM) a service that hosts them so that you can pass the same OSM parameters to the URL and get maps
- hakube 2y agoIt would be great if it's an actual planet with its own ecosystem. This is just a tile map
- dsr_ 2y agoOSM is OpenStreetMaps. This appears to be a server to locally mirror the ~90GB global tile set and serve them with your choice of style to OSM client applications.
- andrewshadura 2y agoIt's not OpenStreetMaps, it's OpenStreetMap.
- Aachen 2y agoI've always wondered why people keep adding the s. Is it marketing because Google Map sounds like a spelling mistake (missing 's after Google) and we're noticing the effect of that other brand, or is there something about OpenStreetMap that makes it seem like a plural project? People don't say Linuxes, Signals, or Androids
- axelthegerman 2y agoMaybe because "Google Map" is actually called "Google Maps" (with the s)?
- NoboruWataya 2y agoI think it's most likely that most of the other commonly used alternatives (most significantly Google Maps but also Apple Maps, Bing Maps, Maps.me) use the plural.
- sbarre 2y agoSame feedback as everyone else: You fell into the trap of assuming that everyone else understands this topic/project/thing that you're deeply familiar with because you've been living it for so long. Not everyone knows what "OSM" is. Even a 2-3 sentence paragraph in your TL;DR that explains (a) what this is and (b) why people should care or want this, would be immensely helpful.
- Aachen 2y ago> Not everyone knows what "OSM" is. Obligatory: https://xkcd.com/1053/ https://xkcd.com/1053/
- dagw 2y agoOn the other hand if you don't know what "OSM" is, then you probably don't need this, and probably don't need to care.
- sbarre 2y agoFair but I was referring to the whole write-up on the Gist page not just the headline here on HN.
- jakepage91 2y agoImage, screenshot, Gifs, anything really so I can see what it looks like before allocating 90GBS would be nice.
- mobilemidget 2y ago90 is okay, when I did years ago I was on 300+ but it doesn't really state which zoom level is available.
- btzs 2y agoIs there any explanation why it has decreased in size?
- dagw 2y agoThere are two ways to store map tile data like OSM. One is as pre-rendered raster tiles (256x256 png images) and the other is as tiles of vector data that get rendered either on the fly by dedicate rendering middleware or rendered client side by the browser. Vector tiles are anywhere between 20-50% smaller that corresponding raster tiles. Raster tiles was the standard way of doing web maps up until a few years ago, when client side rendering in the browser became feasible and now most people use vector tiles.
- dagw 2y agoIt seems to be 0-14 according to the GitHub repo.
- moffkalast 2y agoYeah going to full 20 requires about 2TB iirc, OSM doesn't host beyond level 19 anymore either to reduce traffic. 14 is around 10m per pixel, best you'll be able to make out on that is like a street. Not useful enough to bother with imho, even 19 leaves a lot to be desired up close. https://wiki.openstreetmap.org/wiki/Zoom_levels https://wiki.openstreetmap.org/wiki/Zoom_levels
- trojan13 2y agoWhere is my .exe file to run the entire planet in ~30 minutes? To add some value to my joke comment: I would assume this is running "Open Street Map" locally on a webserver.
- robertlagrant 2y agoBecause various comments are upset that this isn't actually self-hosting a planet simulation[0] or that the readme isn't good enough - this is really cool. Premade tile server for OpenStreetMaps is lovely. [0] https://news.ycombinator.com/item?id=41223091 https://news.ycombinator.com/item?id=41223091
- IgorPartola 2y agoI remember when I worked on a project in 2010-2012 that relied on OSM it took days to set up a part of the system, the documentation was mostly in German or if you were lucky in an arcane bash script, and which part did what required a deep understanding of the entire thing. I don’t need this anymore but this is really nice.
- mosselman 2y agoSo I assume if I run this I can aim something like leaflet at it for the map tiles? Any information you have on how production-ready this is would be nice. Things such as performance, stability, max expected clients with a small vps or something.
- tuxone 2y agoSome things you may want to consider when hosting map tiles in production: - load balancing and caching. Some tiles are much more popular than others. - how will you update the tiles (sync from OSM). - if required, how will you manage customizations (and merges with OSM).
- leighleighleigh 2y agoA lotta negativity in here... My immediate reaction was "fuck yeah I always wanted to host OSM but I couldn't figure out how last time". thanks dude :)
- dschep 2y agowithout a means to update the map, this isn't a breakthrough. I'd rather use protomaps[0](which has regular builds, from a better-known source) with caddy. [0] https://protomaps.com/ https://protomaps.com/
- kh_hk 2y agoWhile useful, this still uses a server to process tile requests. Why not https://github.com/protomaps/PMTiles https://github.com/protomaps/PMTiles ? For the record, I am currently using maptiler over at https://citybik.es https://citybik.es to serve tiles, and I am evaluating moving to serving static pmtiles on range requests to cut the middle man
- wiredfool 2y agoPMTiles is great, but there's a risk hosting multi-gb files on metered hosting -- a client might not respect range requests and you open yourself up to a denial of wallet attack. I can't really see doing it on S3, unless it's behind a lightweight tile server. I'd do it on hetzner or linode or one of the lower cost bandwidth providers. (edit -- citybik.es is pretty cool, are you backing up to GBFS or a different set of data feeds? e.g. https://github.com/MobilityData/gbfs/blob/master/systems.csv https://github.com/MobilityData/gbfs/blob/master/systems.csv is a list of ~ 800 gbfs feeds, some dockless, some docked I've done a bit of work in this space, feel free to ping me)
- kh_hk 2y agoWill keep that in mind if I go that route. GBFS is just one format of the many supported. This is where we keep all instance data https://github.com/eskerda/pybikes/tree/master/pybikes/data https://github.com/eskerda/pybikes/tree/master/pybikes/data . This is the list of GBFS systems we have https://github.com/eskerda/pybikes/blob/master/pybikes/data/gbfs.json https://github.com/eskerda/pybikes/blob/master/pybikes/data/...
- bdon 2y agoTo mitigate the "denial of wallet" attack on S3 there are a few options: AWS Lambda: https://docs.protomaps.com/deploy/aws https://docs.protomaps.com/deploy/aws lightweight tile server: https://docs.protomaps.com/pmtiles/cli#serve https://docs.protomaps.com/pmtiles/cli#serve
- sorenjan 2y agoHosting maps of the entire planet in a single file doesn't seem practical. How do you keep it updated? PMTiles is good for small local maps, but I don't think they're meant for this scale.
- hk__2 2y agoWhy copy the README as a gist instead of posting a link to the repo? https://github.com/markuman/sms?tab=readme-ov-file https://github.com/markuman/sms?tab=readme-ov-file
- andrewshadura 2y agoIt's OpenStreetMap, without the "s" at the end, please fix the title.
- lopkeny12ko 2y ago[flagged]
- donalhunt 2y ago20 years ± a few days... https://birthday20.openstreetmap.org/ https://birthday20.openstreetmap.org/
- ranger_danger 2y agohttps://xkcd.com/1053 https://xkcd.com/1053
- xrd 2y agoDoes anyone know if there is a way to host just a few of the tiles in a static way? For example, if I wanted to build a web page which just shows a map at zoom level 6 for a lat/lng point, and then go to zoom level 13. That would require a tiny subset of tiles; is there a simple way to add the tiles plus the JS code in a static way so no external downloads were necessary?
- Raphaellll 2y agoYou can simply pre-render them and store the png tiles in a folder. The folder structure and naming has to be such that e.g. leaflet understands it. Did this for a research demo here: https://map2seq.schumann.pub/nllni/demo/ https://map2seq.schumann.pub/nllni/demo/
- orblivion 2y agoIs there actually a simple way to pre-render? AFAIK it's a lot of work and learning at least if you're coming from scratch.
- wiredfool 2y agoQGIS can make a tile pyramid of the current map/project at a specific set of bounds.
- jonah 2y agoI've used TileMill to create tiles for a specific layer of data for a specific area. (Fire evacuation zones for a county.) https://github.com/tilemill-project https://github.com/tilemill-project
- brennopost 2y agoI had success with Protomaps (https://protomaps.com/ https://protomaps.com/).
- orblivion 2y agoLook at https://protomaps.com/ https://protomaps.com/ - You can have a single file in a that represents the vector data for a region at multiple zoom levels. There used to be a really nice UI where you could just select a region. They've changed things around a bit since then so I'm not sure how it works now.
- deleted 2y ago[deleted]
- gregoryw3 2y agoTangentially related, does anyone know of an open source 3D terrain system? I’ve been thinking of making a native map and globe program similar to Google earth to embed inside a dashboard (visually similar to kerbal space program). So far I was thinking about just using ArcGIS (or something else) and taking in free Lidar data and imposing the depth to satellite or OSM tiles then rendering them as OBJs in OpenGL (or maybe a full 3d engine like unity). This unfortunately wouldn’t be an automatic process as lidar data is taxing to process in real time, especially for a SBC. I do see that there are some 3D tiles sets out there but I don’t see where I can pull them from nor how it compares to the lidar data. However, the main goal is a 3D terrain viewer in an offline native program.
- cyberax 2y ago> Tangentially related, does anyone know of an open source 3D terrain system? You can get the heightmap from NASA. It's 10x10 meter resolution, which is enough for static geography.
- simlevesque 2y agoI'd like to find a cheap way to self-host a stripped version of Nominatim planet. There's lots of data that I don't really need to deal with and I don't want to have to use 32/64gb of ram.
- Borg3 2y agoHmm, Im not really sure why MBTiles are popular? They seems to be cache unfriendly. Good old tiles via /{x}/{y}/{z}.png seems to be better. And you can slap web cache in front. Hosting those is super simple. The problem I have is how to generate them efficently w/o big resources.