11 ms·
Few years back I contracted for an AI startup, long story short we ran a simple test of one anotator radiologist with 15years of experience to another(similar a
by tlear 5y ago
Few years back I contracted for an AI startup, long story short we ran a simple test of one anotator radiologist with 15years of experience to another(similar amount of experience) over 50 or so CT scans. They agreed on about 60% of the time lol and I mean easily spot table size “things” that were annotated as potentially malignant nodules or “not at all because they were “scars”.
That’s when I knew we did not wtf we were doing.
- ska 5y agoReader variability is one of the many things that make this stuff a lot harder than it looks at from the outside.
- tlear 5y agoMy take on it was that we are not so far removed from witchdoctors when it comes to medicine then we would like to believe. Hard data on efficiency and performance of human doctors is very far from hard and their ability to diagnose things that are not blatantly obvious and are bout to kill you dead unless you get lucky and surgery works or chemo works(or does not and we have no clue why) is very limited. There is a huge amount of we want to believe.. and survivor bias going on in cancer treatment. Lot's of it is very harmful as well.