The new thing is what can be done with the data. Rather than simply deciding how to market your service to the customer, you can use machine learning systems t

The new thing is what can be done with the data. Rather than simply deciding how to market your service to the customer, you can use machine learning systems to conjecture the likelihood of that person being, say, a criminal, or an adulterer, or a political dissident. For each category of 'bad person' you can imagine, you can gin up some (almost completely nonsensical, but that doesn't matter) analysis profile and assign a probability score. You can then take the top couple of categories they score in and use that data however they please. For awhile I've wanted to take all publicly available data about politicians and train a machine learning system on it to predict which pols are cheating on their spouses or engaged in other sorts of elicit activities. I have zero confidence in the ability of such systems to predict human behavior, but that doesn't mean they can't be tremendously powerful tools of intimidation and manipulation. You'll notice after the Snowden leaks, each and every single statement the government ever made was extremely focused on the notion of human analysts reading the fulltext of peoples conversations. Not a mention was given to the automated systems profiling, categorizing, targetting, and producing "summaries" about people. Personally, those blind systems cobbled together by people with too few scruples ('just doing my job' is identical to 'just doing it for money' and those engineers deserve to have their faces spit in) and deployed by those with too much faith in oracular technology scare me far, far worse...