4 ms·
Pretty much. And also, models almost never operate on full-size image, and if you e.g. pixel bin a bayer image you will get a de-mosaiced image, so that's not v
by h4b4n3r0 8y ago
Pretty much. And also, models almost never operate on full-size image, and if you e.g. pixel bin a bayer image you will get a de-mosaiced image, so that's not very interesting. What is interesting is, it'd be pretty cool to train on linear HDR images for instance (i.e. direct 14-stop sensor output without contrast curves and color correction), or on visible spectrum + IR + UV, but I'm not aware of any large public datasets like this. The reason why this is interesting is because a lot of vision models suffer from the lack of robustness. Their performance on the train/validation datasets might be OK, but connect them to a real live video feed and they turn to shit. To some extent this can be addressed by data augmentation during training, but you have to understand that this is an imperfect process: you're operating on the data that's already pre-cooked, so to speak, for _human_ consumption, with color correction, gamma curves, etc already applied, and there's not a whole lot you can do to make things more robust to different lighting conditions, noise in the feed when camera bumps the gain in its ADCs, color drift due to e.g. sodium lamps, over/under exposure, etc. Unsurprisingly, this is especially noticeable in efficient, quantized models, because they don't really have much leeway built in to work around the deficiencies in the input data.
- ovi256 8y agoRegarding multi-spectral images, there was a recent Kaggle competition classifying ship-or-iceberg from radar multiple band data. So yeah, CNNs work well for that too. https://www.kaggle.com/c/statoil-iceberg-classifier-challenge https://www.kaggle.com/c/statoil-iceberg-classifier-challeng...