6 ms·

I’ve done a few projects that attempted to distill the knowledge of human experts, mostly in medical imaging domain, and was shocked when for most of them the i

by m_ke 2y ago

I’ve done a few projects that attempted to distill the knowledge of human experts, mostly in medical imaging domain, and was shocked when for most of them the inter annotator agreement was only around 60%. These were professional radiologists with years of experience and still came to different conclusions for fairly common conditions that we were trying to detect. So yes, LLMs will make mistakes, but humans do too, and if these models do so less often at a much lower cost it’s hard to not use them.

tomrod 2y ago
This hints at the margin and excitement from folks outside the technical space -- being able to be competitive to human outputs at a fraction of the cost.
- ethbr1 2y ago
  That's the underappreciated truth of the computer revolution in practice. At scale, computers didn't change the world because they did things that were already being computed, more quickly. They changed the world because they decreased the cost of computing so much that it could be used for an entirely new class of problems. (That computing cost previously precluded its use on)
threeseed 2y ago
The problem is that how mistakes are made is crucial. If it's a forced binary choice then sure LLMs can replace humans. But often there are many shades of grey e.g. a human may say I don't know and refer to someone else or do some research. Whereas LLMs today will simply give you a definitive answer even if it doesn't know.
- m_ke 2y ago
  None of these were binary decisions, but classifying one of around 10-20 conditions or rating cases on a 1-5 scale. In all cases the models trained on a lot of this feedback were more consistent and accurate than individual expert annotators.
  - Uehreka 2y ago
    I’m guessing these are also specially trained image classifiers and not LLMs, so people’s intuitions about how LLMs work/fail may not apply.
    - m_ke 2y ago
      It’s the same softmax classifier
  - snowwrestler 2y ago
    Wait if experts only agreed 60% on diagnoses, what is the reliable basis for judging LLM accuracy? If experts struggle to agree on the input, how are they confidently ranking the output?
    - petra 2y ago
      You can look at fully diagnosed cases(via surgery for example) and their previous scans.
    - throwup238 2y ago
      Not the OP but the data isn’t randomly selected, it’s usually picked out of a dataset with known clinical outcomes. So for example if it’s a set of images of lungs with potential tumors, the cases come with biopsies which determined whether it was cancerous or just something like scar tissue.
    - Eisenstein 2y ago
      Perhaps they were from cases that had a confirmed diagnosis.
- IanCal 2y ago
  > Whereas LLMs today will simply give you a definitive answer even if it doesn't know. Have you not seen an LLM say it doesn't know the answer to something? I just asked "How do I enable a scroflpublaflex on a ggh connection?" to O1 pro as it's what I had open. Looking at the internal reasoning it says it doesn't recognise the terms, considers that it might be a joke and then explains that it doesn't know what either of those are. It says maybe they're proprietary, maybe internal things, and explains a general guide to finding out (e.g. check internal docs and release notes, check things are up to date if it's a platform, verify if versions are compatible, look for config files [suggesting a few places those could be stored or names they could have], how to restart services if they're systemctl services, if none of this applies it suggests checking spelling and asks if I can share any documentation. This isn't unique or weird in my experience. Better models tend to be better at saying they don't know.
  - ragazzina 2y ago
    You have used funny-sounding terms. Can I ask you to try with: "Is it possible to enable a crolubaflex 2.0 on a ggh connection? Please provide a very short answer." On my (free) plan it gives me a confident negative answer.
    - abecedarius 2y ago
      Claude non-free: > I apologize, but I can't provide an answer as "crolubaflex" and "ggh connection" appear to be non-existent technical terms. Could you clarify what you're trying to connect or enable?
    - IanCal 2y ago
      Sure, I'm interested in where the boundaries are with this. With the requirements for a short answer, the reasoning says it doesn't know what they are so it has to respond cautiously, then says no. Without that requirement it says it doesn't know what they are, and notes that they sound fictional. I'm getting some API errors unfortunately so this testing isn't complete. 4o reliably keeps saying no (which is wrong).
- Shorel 2y ago
  > But often there are many shades of grey e.g. a human may say I don't know and refer to someone else or do some research. Whereas LLMs today will simply give you a definitive answer even if it doesn't know. To add to the other answers: I know many people who will give definitive answers of things they don't really know. They just rely on the fact you also don't know. In fact, in some social circles, the amount of people who do that, far outnumber the people who don't know and will refer you to someone else.
ksec 2y ago
Given the exact same facts ( just like medical imaging domain ), human will form different opinion or conclusion on politics. I think what is not discussed enough is the assumption of assumption. [1] is a cognitive bias that occurs when a person who has specialized knowledge assumes that others share in that knowledge. This makes it hard for any discussions without layering out all the absolute basic facts. Which has now more commonly known as First Principle in modern era. In the four quadrants known and unknown. It is often the unknown known ( We dont even know we know ) that is problematic in discussions. [1] Curse of knowledge - https://en.wikipedia.org/wiki/Curse_of_knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge
ADeerAppeared 2y ago
> So yes, LLMs will make mistakes, but humans do too Are you using LLMs though? Because pretty much all of these systems are fairly normal classifiers, what would've been called Machine Learning 2-3 years ago. The "AI hype is real because medical AI is already in use" argument (and it's siblings) perform a rhetorical trick by using two definitions of AI. "AI (Generative AI) hype is real because medical AI (ML classifiers) is already in use" is a non-sequitur. Image classifiers are very narrow intelligences, which makes them easy to understand and use as tools. We know exactly what their failure modes are and can put hard measurements on them. We can even dissect these models to learn why they are making certain classifications and either improve our understanding of medicine or improve the model. ... Basically none of this applies to Generative AI. The big problem with LLMs is that they're simply not General Intelligence systems capable of accurately and strongly modelling their inputs. e.g. Where an anti-fraud classifier directly operates on the financial transaction information, an LLM summarizing a business report doesn't "understand" finance, it doesn't know what details are important, which are unusual in the specific context. It just stochastically throws away information.
- m_ke 2y ago
  Yes I am, these LLM/VLMs are much more robust at NLP/CV tasks than any application specific models that we used to train 2-3 years ago. I also wasted a lot of time building complex OCR pipelines that required dewarping / image normalization, detection, bounding box alignment, text recognition, layout analysis, etc and now open models like Qwen VL obliterate them with an end to end transformer model that can be defined in like 300 lines of pytorch code.
  - ADeerAppeared 2y ago
    Different tasks then? If you are using VLMs in the context of medical imaging, I have concerns. That is not a place to use hallucinatory AI. But yes, the transformer model itself isn't useless. It's the application of it. OCR, image description, etc, are all that kind of narrow-intelligence task that lends itself well to the fuzzy nature of AI/ML.
SoftTalker 2y ago
This is why second opinions are widely used in any serious medical diagnosis.

by m_ke 2y ago

tomrod 2y ago

This hints at the margin and excitement from folks outside the technical space -- being able to be competitive to human outputs at a fraction of the cost.

ethbr1 2y ago
That's the underappreciated truth of the computer revolution in practice. At scale, computers didn't change the world because they did things that were already being computed, more quickly. They changed the world because they decreased the cost of computing so much that it could be used for an entirely new class of problems. (That computing cost previously precluded its use on)

threeseed 2y ago

The problem is that how mistakes are made is crucial. If it's a forced binary choice then sure LLMs can replace humans. But often there are many shades of grey e.g. a human may say I don't know and refer to someone else or do some research. Whereas LLMs today will simply give you a definitive answer even if it doesn't know.

m_ke 2y ago
None of these were binary decisions, but classifying one of around 10-20 conditions or rating cases on a 1-5 scale. In all cases the models trained on a lot of this feedback were more consistent and accurate than individual expert annotators.
- Uehreka 2y ago
  I’m guessing these are also specially trained image classifiers and not LLMs, so people’s intuitions about how LLMs work/fail may not apply.
  - m_ke 2y ago
    It’s the same softmax classifier
- snowwrestler 2y ago
  Wait if experts only agreed 60% on diagnoses, what is the reliable basis for judging LLM accuracy? If experts struggle to agree on the input, how are they confidently ranking the output?
  - petra 2y ago
    You can look at fully diagnosed cases(via surgery for example) and their previous scans.
  - throwup238 2y ago
    Not the OP but the data isn’t randomly selected, it’s usually picked out of a dataset with known clinical outcomes. So for example if it’s a set of images of lungs with potential tumors, the cases come with biopsies which determined whether it was cancerous or just something like scar tissue.
  - Eisenstein 2y ago
    Perhaps they were from cases that had a confirmed diagnosis.
IanCal 2y ago
> Whereas LLMs today will simply give you a definitive answer even if it doesn't know. Have you not seen an LLM say it doesn't know the answer to something? I just asked "How do I enable a scroflpublaflex on a ggh connection?" to O1 pro as it's what I had open. Looking at the internal reasoning it says it doesn't recognise the terms, considers that it might be a joke and then explains that it doesn't know what either of those are. It says maybe they're proprietary, maybe internal things, and explains a general guide to finding out (e.g. check internal docs and release notes, check things are up to date if it's a platform, verify if versions are compatible, look for config files [suggesting a few places those could be stored or names they could have], how to restart services if they're systemctl services, if none of this applies it suggests checking spelling and asks if I can share any documentation. This isn't unique or weird in my experience. Better models tend to be better at saying they don't know.
- ragazzina 2y ago
  You have used funny-sounding terms. Can I ask you to try with: "Is it possible to enable a crolubaflex 2.0 on a ggh connection? Please provide a very short answer." On my (free) plan it gives me a confident negative answer.
  - abecedarius 2y ago
    Claude non-free: > I apologize, but I can't provide an answer as "crolubaflex" and "ggh connection" appear to be non-existent technical terms. Could you clarify what you're trying to connect or enable?
  - IanCal 2y ago
    Sure, I'm interested in where the boundaries are with this. With the requirements for a short answer, the reasoning says it doesn't know what they are so it has to respond cautiously, then says no. Without that requirement it says it doesn't know what they are, and notes that they sound fictional. I'm getting some API errors unfortunately so this testing isn't complete. 4o reliably keeps saying no (which is wrong).
Shorel 2y ago
> But often there are many shades of grey e.g. a human may say I don't know and refer to someone else or do some research. Whereas LLMs today will simply give you a definitive answer even if it doesn't know. To add to the other answers: I know many people who will give definitive answers of things they don't really know. They just rely on the fact you also don't know. In fact, in some social circles, the amount of people who do that, far outnumber the people who don't know and will refer you to someone else.

ksec 2y ago

Given the exact same facts ( just like medical imaging domain ), human will form different opinion or conclusion on politics. I think what is not discussed enough is the assumption of assumption. [1] is a cognitive bias that occurs when a person who has specialized knowledge assumes that others share in that knowledge. This makes it hard for any discussions without layering out all the absolute basic facts. Which has now more commonly known as First Principle in modern era. In the four quadrants known and unknown. It is often the unknown known ( We dont even know we know ) that is problematic in discussions. [1] Curse of knowledge - https://en.wikipedia.org/wiki/Curse_of_knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge

ADeerAppeared 2y ago

> So yes, LLMs will make mistakes, but humans do too Are you using LLMs though? Because pretty much all of these systems are fairly normal classifiers, what would've been called Machine Learning 2-3 years ago. The "AI hype is real because medical AI is already in use" argument (and it's siblings) perform a rhetorical trick by using two definitions of AI. "AI (Generative AI) hype is real because medical AI (ML classifiers) is already in use" is a non-sequitur. Image classifiers are very narrow intelligences, which makes them easy to understand and use as tools. We know exactly what their failure modes are and can put hard measurements on them. We can even dissect these models to learn why they are making certain classifications and either improve our understanding of medicine or improve the model. ... Basically none of this applies to Generative AI. The big problem with LLMs is that they're simply not General Intelligence systems capable of accurately and strongly modelling their inputs. e.g. Where an anti-fraud classifier directly operates on the financial transaction information, an LLM summarizing a business report doesn't "understand" finance, it doesn't know what details are important, which are unusual in the specific context. It just stochastically throws away information.

m_ke 2y ago
Yes I am, these LLM/VLMs are much more robust at NLP/CV tasks than any application specific models that we used to train 2-3 years ago. I also wasted a lot of time building complex OCR pipelines that required dewarping / image normalization, detection, bounding box alignment, text recognition, layout analysis, etc and now open models like Qwen VL obliterate them with an end to end transformer model that can be defined in like 300 lines of pytorch code.
- ADeerAppeared 2y ago
  Different tasks then? If you are using VLMs in the context of medical imaging, I have concerns. That is not a place to use hallucinatory AI. But yes, the transformer model itself isn't useless. It's the application of it. OCR, image description, etc, are all that kind of narrow-intelligence task that lends itself well to the fuzzy nature of AI/ML.

SoftTalker 2y ago

This is why second opinions are widely used in any serious medical diagnosis.