I share your concerns. The post-9/11 scramble for homeland security funding in academia really killed my enthusiasm for the AI/ML stuff I was doing at the time

I share your concerns. The post-9/11 scramble for homeland security funding in academia really killed my enthusiasm for the AI/ML stuff I was doing at the time. The message that is heard when tech people sell AI/ML is 'magic cure-all' and it basically comes down to not understanding Bayes' theorem: If your algorithm has any capacity to make type one or type two errors (even if the success rates seem impressive) bad things will happen when you look for infrequent events (terrorism/crime) with low priors: you get more bogus hits than true ones. Once the priors get to 50% great, but then you have bigger issues (civil war and chaos). Law enforcement is then obliged to follow up on those bogus hits which given the size of the population can be overwhelming. And that is even before you get to ethical minefields mentioned above. Making decisions about people based on models that the implementer might not even fully understand is unnerving. It makes it easy to slip in social/socio-economic assumptions about people for those who would like to try. Even if the implementer acts in good-faith, it can easily happen inadvertently. Given how little is known about how to deal with social problems in general, trying to paper over that ignorance with AI/ML makes everything worse. It is operationalizing ignorance with high-efficiency.