3 ms·

I work as a digital marketing analyst (think Google Analytics). While reading this piece, I find myself nodding and agreeing with 90% of it. As it turns out, th

by lmkg 12y ago

I work as a digital marketing analyst (think Google Analytics). While reading this piece, I find myself nodding and agreeing with 90% of it. As it turns out, the way that decision-makers incorporate data and expert opinion doesn't change when whether they're in charge of $10k ad budgets, or ten-year occupations of a foreign country. Some observations of my own, that I can see echoes of in this paper: Decision-makers have a mental model where analysts' job is to synthesize facts and provide them, and it is the job of the decision-makers themselves to put the facts together into a big picture and/or plan of action. This is generally a poor set-up. Firstly, "facts" with any certainty are thin on the ground, there's only inferences from small spots of data in between clouds of blind spots. Secondly, the analysts have more experience with the trends that they study, and often more training in how to incorporate them into a big picture. Most managers will say "I want all the data," but if you give them what they ask for, they will most often draw incorrect conclusions. You have to spoon-feed them, even though neither party particular wants that. People don't understand the limits of data, and they sure as heck don't understand uncertainty. If you make 100 predictions, and are 99% sure about all of them, then on average one will be untrue. And all you'll ever hear about was that you were 99% sure and were wrong. In my job, there's actually a consideration about not providing data that we don't totally trust (as opposed to providing with caveats), because a single data point that's wrong erodes trust in data in general. It sounds like that's not an option to the CIA: if they're tasked with a question, they must provide an answer (which is a problem itself). We've found that if we our client to use our analysis, we actually need to write it with his audience in mind. We literally write it with the design goal that the client be able to copy-paste our words or graphs into his own deliverable. Because that is literally what will happen. The more effort it is for the client to consume our work, the more likely they are to save themselves the time and ignore it. And as bad as that is, it's also probably the best way. If the client re-words things, he may misunderstand things. Encouraging plagiarism cuts down on the game of telephone.

jonnathanson 12y ago
"...the client be able to copy-paste our words or graphs into his own deliverable. Because that is literally what will happen." I've spent most of my career in marketing, and as far as I can tell, this happens everywhere. Well, almost everywhere. The one exception I've encountered was working at Apple, where people expected you to hammer out a beautiful and convincing Keynote deck on your own. Everywhere else, people more or less copypasted whatever decks, charts, graphs, and bullets their agencies had sent them. The problem with copypasting is that it can lead to a game of telephone. A sequential erosion in the quality of information. If you're not keeping track of who's been lifting and plagiarizing and pasting which information from which sources, at how many steps along the value chain, you end up with a deck full of secondhand information and hearsay. The quality of the information (or confidence in the information) fades a bit with each subsequent generation of copypasting. Sort of like lossy data encoding. At larger organizations, as you might imagine, there will be many "generations" of each presentation. This makes the lossiness issue even worse. The culprit here is always time. It's exactly as the author of this article points out. People don't set out to make shitty decks, or to copy and paste information, or to tl;dr their way through analysis. But that's just what people have to do to survive. If you're spending (conservatively) 20-30 hours a week in meetings, and many of those meetings leave you with deliverables on tight deadlines, you're going to be eking out any and all efficiencies that come your way. I've never done any copypasting, and if I'm being honest with myself, it's probably hurt me more than it's helped me.
sliverstorm 12y ago
... "facts" with any certainty are thin on the ground, there's only inferences... In support of that, lifting right from the article, Former CIA Director Michael Hayden once famously said, “If it is a fact, it ain’t intelligence.”
remarkEon 12y ago
-> ...and they sure as heck don't understand uncertainty. Man this could not be more true. You can get people to (sometimes) understand basic probability if they don't have a math background, but try something like bounded probability for explaining the likelihood of the severity of an event and they'll get lost.

by lmkg 12y ago

jonnathanson 12y ago

"...the client be able to copy-paste our words or graphs into his own deliverable. Because that is literally what will happen." I've spent most of my career in marketing, and as far as I can tell, this happens everywhere. Well, almost everywhere. The one exception I've encountered was working at Apple, where people expected you to hammer out a beautiful and convincing Keynote deck on your own. Everywhere else, people more or less copypasted whatever decks, charts, graphs, and bullets their agencies had sent them. The problem with copypasting is that it can lead to a game of telephone. A sequential erosion in the quality of information. If you're not keeping track of who's been lifting and plagiarizing and pasting which information from which sources, at how many steps along the value chain, you end up with a deck full of secondhand information and hearsay. The quality of the information (or confidence in the information) fades a bit with each subsequent generation of copypasting. Sort of like lossy data encoding. At larger organizations, as you might imagine, there will be many "generations" of each presentation. This makes the lossiness issue even worse. The culprit here is always time. It's exactly as the author of this article points out. People don't set out to make shitty decks, or to copy and paste information, or to tl;dr their way through analysis. But that's just what people have to do to survive. If you're spending (conservatively) 20-30 hours a week in meetings, and many of those meetings leave you with deliverables on tight deadlines, you're going to be eking out any and all efficiencies that come your way. I've never done any copypasting, and if I'm being honest with myself, it's probably hurt me more than it's helped me.

sliverstorm 12y ago

... "facts" with any certainty are thin on the ground, there's only inferences... In support of that, lifting right from the article, Former CIA Director Michael Hayden once famously said, “If it is a fact, it ain’t intelligence.”

remarkEon 12y ago

-> ...and they sure as heck don't understand uncertainty. Man this could not be more true. You can get people to (sometimes) understand basic probability if they don't have a math background, but try something like bounded probability for explaining the likelihood of the severity of an event and they'll get lost.