4 ms·

Show HN: Moonshine – open-source, pretrained ML models for satellite

86 points by nharada 4y ago

Hey HackerNews, Today I'd like to share my open source project, Moonshine! Pretrained vision models are a popular way to reduce how much data you need and to speed up training, but for remote sensing (i.e. aerial, satellite) it can be a challenge to find good weights. Why use Moonshine? 1. Pretrained on multispectral data: Existing popular CV packages are nice to use but are trained on ImageNet or similar, meaning that not only is the domain of the pretraining different, but you may be restricted to only 3 channels. For remote sensing platforms with multispectral data, this might be a non-starter. Moonshine includes models specifically trained for your remote sensing problems. 2. Focus on usability: There are academic releases of specialized models that do support remote sensing data, but often they are difficult to use. They might be hard to install, and often are supported by a grad student who is more focused on publication than software. A core tenant of Moonshine is that it should be easy to use. I've been working on this project for nearly a year now and I'm really excited to show it off and most importantly get feedback! I have big plans for what to build in the future, but this set of features was the smallest one I could think of that would provide some value. Docs: https://moonshineai.readthedocs.io/en/latest/index.html https://moonshineai.readthedocs.io/en/latest/index.html Github: https://github.com/moonshinelabs-ai/moonshine https://github.com/moonshinelabs-ai/moonshine

zecken 4y ago
Wow this seems really useful! Is it bad the first thing I think when I see this is how a nefarious government can use it with bad intentions?
mistrial9 4y ago
How can this get to World Wildlife Fund and other non-specialists doing environmental work ? add new publicity (check!) .. other ways?
- nharada 4y ago
  That's a great question -- I'd really love to get this in the hands of people doing good and necessary work on shoe-string budgets. Non-profits, university researchers, climate modeling, etc. If you're in one of those areas, I have a Slack[1] that you can join to get help or ask questions. You can also just email me if you'd like at nate[at]moonshinelabs.ai and I'd love to talk! [1] https://join.slack.com/t/moonshinecommunity/shared_invite/zt-1rg1vnvmt-pleUR7TducaDiAhcmnqAQQ https://join.slack.com/t/moonshinecommunity/shared_invite/zt...
tectonic 4y ago
You should definitely submit this to https://github.com/orbitalindex/awesome-space#earth https://github.com/orbitalindex/awesome-space#earth!
- nharada 4y ago
  Good idea! I made a pull request. Really cool list of resources!
ravery 4y ago
It'd be great to have a page comparing this to other geoML libs. TorchGeo provides pretrained model weights for Sentinel-2: https://torchgeo.readthedocs.io/en/stable/tutorials/pretrained_weights.html https://torchgeo.readthedocs.io/en/stable/tutorials/pretrain... there's an existing comparison here: https://github.com/weiji14/zen3geo/discussions/70 https://github.com/weiji14/zen3geo/discussions/70
- nharada 4y ago
  This is a good point, I may add some info to the docs to make it easier to compare. At a high level, this library is very lightweight and doesn't contain nearly the bells and whistles that TorchGeo contains. The plans for the future are to create models that are larger, more general, and more broadly useful (for example apply to many datasets). I do have plans to create a Sentinel model as well!
johnea 4y ago
Actully, it's not for satellites, it's for satellite data. I'm at the JPL Flight S/W workshop in Pasadena right now. There are a lot of talks about ML actually IN satellites. But this S/W is strictly ground side... I'm sure it's a valuable tool for data analysis, but a rewording of the topic would be more accurate...
- mturmon 4y ago
  Yeah, it looks like it was "satellite data" and got cut off. OTOH, there's no end to the qualifiers. Some people might be disappointed that it's for Earth remote sensing data (as opposed to Mars). Some might be bothered that it's for multispectral or RGB Earth remote sensing data (as opposed to imaging spectrometry, TIR, or radar). And so on.
yeldarb 4y ago
This is really neat. Is the idea that people will finetune from these pre-trained weights? Would be really curious to learn the delta in model performance on downstream tasks relative to training from randomly initialized weights or generic weights like COCO. We released a benchmark dataset of real-world tasks shared by users recently that has several aerial detection datasets you might be able to use to measure this: https://www.rf100.org https://www.rf100.org (And if you need help this sounds like exactly the type of thing we’d be interested in collaborating on!) Edit: we also have a bunch more aerial datasets listed here that could be useful for this: https://universe.roboflow.com/browse/aerial https://universe.roboflow.com/browse/aerial
dalekchef 4y ago
In general is it okay to use datasets that have NON-Commercial clauses in them to promote a commercial business? I guess because you are not selling the models it is okay? I am not a lawyer, just curious from an ML research perspective. From: https://github.com/fMoW/dataset/blob/master/LICENSE https://github.com/fMoW/dataset/blob/master/LICENSE NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.
- nharada 4y ago
  Also not a lawyer, but in this particular case the project is open source and non-commercial as a project. My understanding in this area in general is that it's still being sorted out, but with the legal happenings in generative modeling there may be some precedents set.
rapiz 4y ago
Can someone explain what are these models for? For a person who knows nothing about geo data, it looks like something that helps to process the data that satellites capture.
- DeathArrow 4y ago
  That was my thought, too.
- randomluck040 4y ago
  When using a huge pre-trained model from the internet that has been trained on a wide variety of images, it might not include a specific set of data. In this case that’s satellite remote sensing data (imagine images from a satellite). Moonshine seems to provide a pre trained weights and models that have seen remote sensing data during their training and therefore solve domain-specific tasks better than a model with weights that have been trained on a more general set of data.

86 points by nharada 4y ago

zecken 4y ago

Wow this seems really useful! Is it bad the first thing I think when I see this is how a nefarious government can use it with bad intentions?

mistrial9 4y ago

How can this get to World Wildlife Fund and other non-specialists doing environmental work ? add new publicity (check!) .. other ways?

nharada 4y ago
That's a great question -- I'd really love to get this in the hands of people doing good and necessary work on shoe-string budgets. Non-profits, university researchers, climate modeling, etc. If you're in one of those areas, I have a Slack[1] that you can join to get help or ask questions. You can also just email me if you'd like at nate[at]moonshinelabs.ai and I'd love to talk! [1] https://join.slack.com/t/moonshinecommunity/shared_invite/zt-1rg1vnvmt-pleUR7TducaDiAhcmnqAQQ https://join.slack.com/t/moonshinecommunity/shared_invite/zt...

tectonic 4y ago

You should definitely submit this to https://github.com/orbitalindex/awesome-space#earth https://github.com/orbitalindex/awesome-space#earth!

nharada 4y ago
Good idea! I made a pull request. Really cool list of resources!

ravery 4y ago

It'd be great to have a page comparing this to other geoML libs. TorchGeo provides pretrained model weights for Sentinel-2: https://torchgeo.readthedocs.io/en/stable/tutorials/pretrained_weights.html https://torchgeo.readthedocs.io/en/stable/tutorials/pretrain... there's an existing comparison here: https://github.com/weiji14/zen3geo/discussions/70 https://github.com/weiji14/zen3geo/discussions/70

nharada 4y ago
This is a good point, I may add some info to the docs to make it easier to compare. At a high level, this library is very lightweight and doesn't contain nearly the bells and whistles that TorchGeo contains. The plans for the future are to create models that are larger, more general, and more broadly useful (for example apply to many datasets). I do have plans to create a Sentinel model as well!

johnea 4y ago

Actully, it's not for satellites, it's for satellite data. I'm at the JPL Flight S/W workshop in Pasadena right now. There are a lot of talks about ML actually IN satellites. But this S/W is strictly ground side... I'm sure it's a valuable tool for data analysis, but a rewording of the topic would be more accurate...

mturmon 4y ago
Yeah, it looks like it was "satellite data" and got cut off. OTOH, there's no end to the qualifiers. Some people might be disappointed that it's for Earth remote sensing data (as opposed to Mars). Some might be bothered that it's for multispectral or RGB Earth remote sensing data (as opposed to imaging spectrometry, TIR, or radar). And so on.

yeldarb 4y ago

This is really neat. Is the idea that people will finetune from these pre-trained weights? Would be really curious to learn the delta in model performance on downstream tasks relative to training from randomly initialized weights or generic weights like COCO. We released a benchmark dataset of real-world tasks shared by users recently that has several aerial detection datasets you might be able to use to measure this: https://www.rf100.org https://www.rf100.org (And if you need help this sounds like exactly the type of thing we’d be interested in collaborating on!) Edit: we also have a bunch more aerial datasets listed here that could be useful for this: https://universe.roboflow.com/browse/aerial https://universe.roboflow.com/browse/aerial

dalekchef 4y ago

In general is it okay to use datasets that have NON-Commercial clauses in them to promote a commercial business? I guess because you are not selling the models it is okay? I am not a lawyer, just curious from an ML research perspective. From: https://github.com/fMoW/dataset/blob/master/LICENSE https://github.com/fMoW/dataset/blob/master/LICENSE NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.

nharada 4y ago
Also not a lawyer, but in this particular case the project is open source and non-commercial as a project. My understanding in this area in general is that it's still being sorted out, but with the legal happenings in generative modeling there may be some precedents set.

rapiz 4y ago

Can someone explain what are these models for? For a person who knows nothing about geo data, it looks like something that helps to process the data that satellites capture.

DeathArrow 4y ago
That was my thought, too.
randomluck040 4y ago
When using a huge pre-trained model from the internet that has been trained on a wide variety of images, it might not include a specific set of data. In this case that’s satellite remote sensing data (imagine images from a satellite). Moonshine seems to provide a pre trained weights and models that have seen remote sensing data during their training and therefore solve domain-specific tasks better than a model with weights that have been trained on a more general set of data.