6 ms·
Have them read Chapter 5 of this book. https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-inte
by davemel37 9y ago
Have them read Chapter 5 of this book. https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/PsychofIntelNew.pdf https://www.cia.gov/library/center-for-the-study-of-intellig...
"Using experts in a variety of fields as test subjects, experimental psychologists
have examined the relationship between the amount of information
available to the experts, the accuracy of judgments they make
based on this information, and the experts’ confidence in the accuracy of
these judgments."
"Key findings from this research are:
• Once an experienced analyst has the minimum information necessary
to make an informed judgment, obtaining additional information
generally does not improve the accuracy of his or her
estimates. Additional information does, however, lead the analyst
to become more confident in the judgment, to the point of overconfidence.
• Experienced analysts have an imperfect understanding of what
information they actually use in making judgments. They are
unaware of the extent to which their judgments are determined
by a few dominant factors, rather than by the systematic integration
of all available information. Analysts actually use much less
of the available information than they think they do."
- 0xCMP 9y agoBut, the main issue is "once an experienced analyst has the minimum information necessary to make an informed judgement" is followed by "they are unaware of the extent to which their judgements are determined by a few dominant factors." Which implies they do not know which part of the data is most important. No? disclaimer: didn't click link/read chapter
- davemel37 9y agoThe authors ultimate solution is to use the scientific method to compare competing hypothesis...otherwise you'll be blinded by all sorts of biases. If you are running an experiment and using competing hypothesis...in theory, you can identify which metrics are best for that test... but if you had much more data, you'll probably end up using every data point past the first 5 as confirmation bias. (I'm no expert though and haven't read the book in about 3 years.)
- mynegation 9y agoDepends on the way of consuming that data. I agree it may be counter-productive to feed more data to humans, who are able to hold on only to 7-9 object in immediate memory at a time, and driven by subconscious reflexes hardwired into us by billions of evolution. AI is not going to "think" like humans. It is not going to be better or worse, just "different". And all experience so far show that the more data the better are the decisions made by AI.
- arkades 9y agoCart, horse. People are proposing smart whatever now, and handeaving its value proposition as -eventually- feeding into ML which will -eventually- have benefits to patients. I don’t think there is disagreement that data + ML is useful, but the value for increasing data because of AI assistance isn’t meaningful in the absence of that AI. Hospitals and insurers have huge data sets available: build a halfway decent AI first, and then argue for the value-add of the data.
- vasilipupkin 9y agoThose data sets are not public, hipaa and other issues make it difficult to build any sort of AI for this.
- arkades 9y agoThat is technically correct, the best kind of correct! Except any department in any hospital or insurer that welcomes you to do research can make that de-identified data available to you with a simple contract. How do you think college freshmen get in the door to work on research with physicians? No, it’s not public, but it’s trivially accessible.
- vasilipupkin 9y agoIt's not trivially accessible. It's not impossible, but far from trivial. Startups like Tempus are doing it but I doubt it's trivial for just any startup to get that data