Thank you too for this information and for the resource, I'm looking at it now it seems very interesting. The breakdown you give above seems to me to be more a

Thank you too for this information and for the resource, I'm looking at it now it seems very interesting. The breakdown you give above seems to me to be more akin to something DL types tend to call 'feature engineering'. I also have a fun example, in this case it would be identifying land cover from satellite imagery. You can obviously just feed the raw reflectance values (RGB etc.) into a DL model to create a semantic segmentation of classes. However, it's been well established in literature at this point that that is not the most effective way to create a classification. This is similar to my previous comment, where there are lots of solutions that can be found through SGD based on these raw values. There's a lot of traditional satellite imagery analysis algorithms that are based on very simple 'band-ratios', i.e. NDVI (normalized difference vegetation index) is calculated by Near Infared - Red / Near Infared + Red. This index will visually highlight areas of vegetation that was extremely useful in human sight-based analysis to identify vegetative areas. Now, you'd expect a deep learning model that takes in all the bands to have this information already, it has the NIR band and the Red band. However, explicitly doing the NDVI calculation and using it as an input feature leads to increased accuracies for classification. The exact reasons for this are unknown, but I think you touched on some of this above. With machine learning, and DL even moreso, sometimes it's necessary to hand-hold the optimization to optimize for exactly what you want. It helps 'explainability' and oftentimes helps accuracy, at the cost of some preprocessing.