Right now it seems that they've trained a true foundational model. It can recreate its training inputs, but that only matters as far as downstream performance g

Right now it seems that they've trained a true foundational model. It can recreate its training inputs, but that only matters as far as downstream performance goes. Based on a brief scanning of the repo & website: - Their training process is a self-supervised MAE to recover the original satellite images and some additional metadata. - They're trying to then use this for plastic pollution detection, deforestation, mining, etc. I'd liken this to a llama-base. It's exposed to a lot of satellite data, but to do something useful really has to be fine-tuned (or clustered in the case of unsupervised embeddings). No word yet on the performance against these specific downstream tasks.