My mental model: Neural networks are for gut decisions. This fits their application domain quite well imho. Gut decisions are quick. You look at a picture and

My mental model: Neural networks are for gut decisions. This fits their application domain quite well imho. Gut decisions are quick. You look at a picture and decide its a dog quicker than you can say it. Executing a neural net is feasible even on mobile devices on battery. Good gut decisions require experience/training. You must train extensively to be good and your training data must be good. An example from "Thinking, Fast and Slow": Experienced stock traders often claim to have a gut feeling, but it is bogus because their training data is bogus (good decisions lead to bad outcomes and vice versa). In contrast, experienced firefighters have a gut feeling if it safe to enter a burning house. This works in practice. The observe stuff about the environment reliably without being able to point them out consciously. Gut decisions are not about planning or knowledge. This requires different AI techniques than neural nets and intuitively shows their shortcomings.