5 ms·

Yes it's very good fun just exploring the embeddings! It's all wrapped by the geotessera Python library, so with uv and gdal installed just try this for your fa

by avsm 1y ago

Yes it's very good fun just exploring the embeddings! It's all wrapped by the geotessera Python library, so with uv and gdal installed just try this for your favourite region to get a false-colour map of the 128-dimensional embeddings: # for cambridge # https://github.com/ucam-eo/geotessera/blob/main/example/CB.geojson curl -OL https://raw.githubusercontent.com/ucam-eo/geotessera/refs/heads/main/example/CB.geojson # download the embeddings as geotiffs uvx geotessera download --region-file CB.geojson -o cb2 # do a false colour PCA down to 3 dimensions from 128 uvx geotessera visualize cb2 cb2.tif # project onto webmercator and visualise using leafletjs over openstreetmap uvx geotessera webmap cb2.tif --output cb2-map --serve Because the embeddings are precomputed, the library just has to download the tiles from our server. More at: https://anil.recoil.org/notes/geotessera-python https://anil.recoil.org/notes/geotessera-python Downstream classifiers are really fast to train (seconds for small regions). You can try out a notebook in VSCode to mess around with it graphically using https://github.com/ucam-eo/tessera-interactive-map https://github.com/ucam-eo/tessera-interactive-map The berries were a bit sour, summer is sadly over here!

throwup238 1y ago
This is all far outside of my wheel house but I'm curious if there's any way to use this for rocks and geology? Identifying dikes and veins on cliff sides from satellites would be really cool.
- tony_cannistra 1y ago
  almost definitely!
- sadiq 1y ago
  It might work. TESSERA's embeddings are at a 10 metre resolution, so it might depend on the size of the features you are looking for. If those features have distinct changes in colour or texture over time or they scatter radar in different ways compared with their surroundings then you should be able to discriminate them. The easiest way to test is to try out the interactive notebook and drop some labels in known areas.
  - throwup238 1y ago
    Is there a way to cluster the embeddings spatially or look for patterns isolated so some dimensions? (Again, way out of my wheel house) What I mean is a vein is usually a few meters wide but can be hundreds of meters long so ten meter resolution is probably not very helpful unless the embeddings can encode some sort of pattern that stretches across many cells.
    - sadiq 1y ago
      It's possible to use embeddings as input to a convolutional network and then train that using labels. We've done that for at least one of the downstream tasks in the TESSERA paper: https://arxiv.org/abs/2506.20380 https://arxiv.org/abs/2506.20380 to estimate canopy height. The downside of that approach is that you need to spend valuable labels on learning the spatial feature extraction during training. To fix that we're working on building some pre-trained spatial feature extractors that you should only need to minimally fine-tune.
- jofer 1y ago
  A major limitation is that most different rock types look essentially identical in visual+NIR spectral ranges. Things separate once you get out to SWIR bands. Sentinel2 does have some SWIR bands and it may work reasonably well with embeddings. But a lot of the signal the embeddings are going to be focused on encoding may not be the right features to distinguish rock types. Methods more focused specifically on the SWIR range are more likely to work reliably. E.g. simple band ratios of SWIR bands may give a cleaner signal than general purpose embeddings in this case. Hyperspectral in the SWIR range is what you really want for this, but that's a whole different ball game.
  - throwup238 1y ago
    > Hyperspectral in the SWIR range is what you really want for this, but that's a whole different ball game. Are there any hyperspectral surveys with UAVs etc instead of satellites?
    - jofer 1y ago
      Usually airplanes because the instruments are heavy. But yeah, that's the most common case. Hyperspectral sats are much rarer than aerial hyperspectral.
      - avsm 1y ago
        An interesting 30x30m satellite that recently launched and is giving back data last year is EnMAP https://www.enmap.org https://www.enmap.org. Hooking that up to TESSERA is on our todo list as soon as we can get our mittens on the satellite data

by avsm 1y ago

throwup238 1y ago

This is all far outside of my wheel house but I'm curious if there's any way to use this for rocks and geology? Identifying dikes and veins on cliff sides from satellites would be really cool.

tony_cannistra 1y ago
almost definitely!
sadiq 1y ago
It might work. TESSERA's embeddings are at a 10 metre resolution, so it might depend on the size of the features you are looking for. If those features have distinct changes in colour or texture over time or they scatter radar in different ways compared with their surroundings then you should be able to discriminate them. The easiest way to test is to try out the interactive notebook and drop some labels in known areas.
- throwup238 1y ago
  Is there a way to cluster the embeddings spatially or look for patterns isolated so some dimensions? (Again, way out of my wheel house) What I mean is a vein is usually a few meters wide but can be hundreds of meters long so ten meter resolution is probably not very helpful unless the embeddings can encode some sort of pattern that stretches across many cells.
  - sadiq 1y ago
    It's possible to use embeddings as input to a convolutional network and then train that using labels. We've done that for at least one of the downstream tasks in the TESSERA paper: https://arxiv.org/abs/2506.20380 https://arxiv.org/abs/2506.20380 to estimate canopy height. The downside of that approach is that you need to spend valuable labels on learning the spatial feature extraction during training. To fix that we're working on building some pre-trained spatial feature extractors that you should only need to minimally fine-tune.
jofer 1y ago
A major limitation is that most different rock types look essentially identical in visual+NIR spectral ranges. Things separate once you get out to SWIR bands. Sentinel2 does have some SWIR bands and it may work reasonably well with embeddings. But a lot of the signal the embeddings are going to be focused on encoding may not be the right features to distinguish rock types. Methods more focused specifically on the SWIR range are more likely to work reliably. E.g. simple band ratios of SWIR bands may give a cleaner signal than general purpose embeddings in this case. Hyperspectral in the SWIR range is what you really want for this, but that's a whole different ball game.
- throwup238 1y ago
  > Hyperspectral in the SWIR range is what you really want for this, but that's a whole different ball game. Are there any hyperspectral surveys with UAVs etc instead of satellites?
  - jofer 1y ago
    Usually airplanes because the instruments are heavy. But yeah, that's the most common case. Hyperspectral sats are much rarer than aerial hyperspectral.
    - avsm 1y ago
      An interesting 30x30m satellite that recently launched and is giving back data last year is EnMAP https://www.enmap.org https://www.enmap.org. Hooking that up to TESSERA is on our todo list as soon as we can get our mittens on the satellite data