6 ms·

They say it uses a deep neural network but we don't know this is a convnet or another type of NN. Anyone has more technical infos about it?

gwern 7y ago
This is semantic segmentation, so it has to be a convnet. I've never heard of trying to do semantic segmentation any other way, and a prototype of another approach wouldn't go into production at scale. OP links to https://ai.facebook.com/blog/mapping-roads-through-deep-learning-and-weakly-supervised-training https://ai.facebook.com/blog/mapping-roads-through-deep-lear... which says that it's D-LinkNet specifically: http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w4/Zhou_D-LinkNet_LinkNet_With_CVPR_2018_paper.pdf http://openaccess.thecvf.com/content_cvpr_2018_workshops/pap... (more or less a Unet).
- joshvm 7y ago
  I don't disagree, but I'd be careful with that statement. A huge amount of landcover segmentation in remote sensing still relies on simple models - either linear regression (thresholds) or classical machine learning like random forests or SVMs. For a lot of cases, these techniques will get you 90% of the way and it's very rare to have ground truth data that is accurate enough that you can measure the difference with any real degree of confidence. A big problem in the field is the lack of good (public) ground truth. There's so little hand labelled data to work with that without humans in the loop it's extremely difficult to validate the results meaningfully (unless you have an army of staff to do it). With something like roads you could also have heuristics about what a road looks like and where it goes (e.g. it's a continuous thin line), which can help condition things. I've seen a lot of papers which are applying deep learning for semantic segmentation for satellite mapping, but they evaluate on very limited datasets, they attempt to regress to simpler models without realising it (e.g. trying to predict a linear model), or they leak train and test data and report amazing results because they randomly split data from the same region. I'm not saying that convnets aren't better than simpler models, but particularly for satellite imaging I'd take them with a pinch of salt and see what the improvement from a baseline method is. If you look at a random sampling of papers from the DeepGlobe competition, almost none of them provide the results from e.g. a cheap linear SVM. Fun side note - several existing "famous" datasets generalise poorly to the developing world because most of the imagery is from the developed world (and even more specifically the West) and infrastructure looks totally different.
  - junipertea 7y ago
    How do you give a class to each pixel in an image using a linear classifier in a way that uses the surrounding pixels as context/input? I'm genuinely curious! You are right about the data, it's expensive to make and startups based on satelite imagery tend to keep them as it's their main advantage.
    - joshvm 7y ago
      In a most cases, you can reshape to a 1xN^2 vector for a NxN region. This was how object detection worked long before convolutional inputs were popular. Have a look at mnist classification using a linear SVM, for example.
      - junipertea 7y ago
        Classification makes sense, because you do a linear (or kernel) combination of the input and squash it using sigmoid to get a probability of a class. For segmentation you output a pixel mask so you would have a NxNx3 vector to predict 1 class for 1 pixel and then you would have to do it for all pixels so you'd have to encode the position as well. Alternatively, if you take a unique weight for each position you end up with get single FC layer with NxNx3 inputs and NxN outputs (N^4 parameters). I guess for me it's hard to imagine doing segmentation "back then" and I find it very fascinating.
li4ick 7y ago
This [1] is one of the best in current research. It tries to actually follow the road. So it naturally builds the road map. [1] https://www.csail.mit.edu/news/mitqcri-system-uses-machine-learning-build-road-maps https://www.csail.mit.edu/news/mitqcri-system-uses-machine-l...

gwern 7y ago

This is semantic segmentation, so it has to be a convnet. I've never heard of trying to do semantic segmentation any other way, and a prototype of another approach wouldn't go into production at scale. OP links to https://ai.facebook.com/blog/mapping-roads-through-deep-learning-and-weakly-supervised-training https://ai.facebook.com/blog/mapping-roads-through-deep-lear... which says that it's D-LinkNet specifically: http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w4/Zhou_D-LinkNet_LinkNet_With_CVPR_2018_paper.pdf http://openaccess.thecvf.com/content_cvpr_2018_workshops/pap... (more or less a Unet).

joshvm 7y ago
I don't disagree, but I'd be careful with that statement. A huge amount of landcover segmentation in remote sensing still relies on simple models - either linear regression (thresholds) or classical machine learning like random forests or SVMs. For a lot of cases, these techniques will get you 90% of the way and it's very rare to have ground truth data that is accurate enough that you can measure the difference with any real degree of confidence. A big problem in the field is the lack of good (public) ground truth. There's so little hand labelled data to work with that without humans in the loop it's extremely difficult to validate the results meaningfully (unless you have an army of staff to do it). With something like roads you could also have heuristics about what a road looks like and where it goes (e.g. it's a continuous thin line), which can help condition things. I've seen a lot of papers which are applying deep learning for semantic segmentation for satellite mapping, but they evaluate on very limited datasets, they attempt to regress to simpler models without realising it (e.g. trying to predict a linear model), or they leak train and test data and report amazing results because they randomly split data from the same region. I'm not saying that convnets aren't better than simpler models, but particularly for satellite imaging I'd take them with a pinch of salt and see what the improvement from a baseline method is. If you look at a random sampling of papers from the DeepGlobe competition, almost none of them provide the results from e.g. a cheap linear SVM. Fun side note - several existing "famous" datasets generalise poorly to the developing world because most of the imagery is from the developed world (and even more specifically the West) and infrastructure looks totally different.
- junipertea 7y ago
  How do you give a class to each pixel in an image using a linear classifier in a way that uses the surrounding pixels as context/input? I'm genuinely curious! You are right about the data, it's expensive to make and startups based on satelite imagery tend to keep them as it's their main advantage.
  - joshvm 7y ago
    In a most cases, you can reshape to a 1xN^2 vector for a NxN region. This was how object detection worked long before convolutional inputs were popular. Have a look at mnist classification using a linear SVM, for example.
    - junipertea 7y ago
      Classification makes sense, because you do a linear (or kernel) combination of the input and squash it using sigmoid to get a probability of a class. For segmentation you output a pixel mask so you would have a NxNx3 vector to predict 1 class for 1 pixel and then you would have to do it for all pixels so you'd have to encode the position as well. Alternatively, if you take a unique weight for each position you end up with get single FC layer with NxNx3 inputs and NxN outputs (N^4 parameters). I guess for me it's hard to imagine doing segmentation "back then" and I find it very fascinating.

li4ick 7y ago

This [1] is one of the best in current research. It tries to actually follow the road. So it naturally builds the road map. [1] https://www.csail.mit.edu/news/mitqcri-system-uses-machine-learning-build-road-maps https://www.csail.mit.edu/news/mitqcri-system-uses-machine-l...