5 ms·
Show HN: My demo for vector embeddings for the Earth's surface
- DerSaidin 3y agoSeems to not handle the ocean well.
- spousty 3y agoIt's due to the fact that they used satellite imagery to create the embeddings. The map is just for visualization. They probably used 5 or more bands of the satellite data which means each pixel is going to be slightly different due to things like depth, amount of silt in the water, amount of plankton.... Having worked on these types of problems before the model is doing a pretty great job matching pixels.
- ckrapu 3y agoThanks! And you are giving it too much credit here - it's just trained on one-hot encoded land cover (24 classes) from Copernicus. Using imagery directly would be # 2 on my list of to-dos after including elevation in the input data.
- spousty 3y agoOh so did you run the CNN on pre-classified data
- ckrapu 3y agoI intentionally avoided using lots of ocean areas - this way I cut down the number of required sites for inference from ~100 million (at resolution 7 in the H3 system) to around 25 million.
- deleted 3y ago[deleted]
- skygazer 3y agoThis tool looks very interesting, and seems to work well, but being utterly unfamiliar with geospatial vector embeddings, their purpose or use, I had no idea what I was looking at, or why. It seems to show areas of similarity, within a radius of a central query location, with regard to (perhaps) vegetation cover (e.g., forests, grasslands, wetlands), artificial surfaces (e.g., urban areas, roads), agricultural areas, water bodies, etc, overlayed on Google Maps, and allows exporting of the embeddings for lat/lons as cvs. It looks like land features for hexagonal grid areas have been turned into points in a 15 dimensional space, and some sort of nearest-neighbor search is done to return most similar other grid areas within the larger area. It does indeed seem accurate in my area! I'm not sure what this would be useful for, but I'm assuming urban planning, real estate, agriculture or conservation? I know I'm not the target audience, but more info or ideas would be fascinating.
- dlnovell 3y agoChris - just saw your presentation of this at PNNL, awesome seeing it pop up on HN too!
- ckrapu 3y agoCool! Glad you got to see it working and that presentation was a nice reason to make sure everything was cleaned up.
- bobeartoes 3y agoIs the presentation of this work available online? Would love to watch or read through!
- aaomidi 3y agoThis is amazing!
- 1024core 3y agoMoved the center to SF and I've been sitting, watching the spinner. Some documentation would be helpful.
- crubier 3y agoI've seen the same thing, querying SF hangs for some reason. And so does Cascais in Portugal. It works in San Mateo and Lisbon though
- ckrapu 3y agoI think I can see what's going on here - I used a shapefile with the boundaries of the world's countries (and their coastlines) which had some geometric simplification applied to it. This file was used to mask out any water (for which the embedding model won't do much), but I think that the simplification process snapped the coastline too far inland, leaving some points on land which were masked out erroneously.
- throwaway743 3y agoDude, please provide context on the site. I have no clue what I'm looking at or its purpose. Not trying to poo poo on it, just want context.
- breckenedge 3y agoIt’s highlighting similar areas to the area currently under the cursor.
- ZoomerCretin 3y agoSimilar how? Geographically? Climate? Population?
- breckenedge 3y agoDepends on the data fed to the model. Probably all of the above and many more.
- ckrapu 3y agoSorry! The presentation could be better. I'll work on the FAQ.
- ckrapu 3y agoI've had to build out some version of a geospatial vector embedding / latent variable dataset for at least 4 separate projects now. Come see the viewer I've built on top of it! The embeddings come from globally available Copernicus land cover data.
- fnordpiglet 3y agoCan you explain what I’m looking at? I don’t know how to interpret the hex tiles :-)
- tartakovsky 3y agoGreat question. A legend or brief description of the underlying logic / heuristic would be helpful.
- breckenedge 3y agoThe heuristic is likely the result of an ML algorithm, so the underlying logic may not make much sense to us.
- wyldfire 3y agoI'm pretty sure I'm not the intended audience but I also have no idea what this is used for. Surveying? Real estate tycoons? Oil & gas exploration?
- potatoman22 3y agoIt's a way to encode land to make predictions of it. E.g. is the land arable, is it rural, how similar is it to X, etc. Embeddings help encode data in formats more usable by ML models.
- wyldfire 3y agoSure, I get that part -- but then how do people use the predictions?
- watersb 3y agoVery nice!