6 ms·

One anecdote is not data. You need controlled trials that include both correct and incorrect diagnoses not just a headline

corethree 3y ago
Wrong. One anecdote is one data point. If one person witnesses a murder that is a data point. The court doesn't just dismiss the data point because they need a sample size of 1000 witnesses for any statement to be meaningful. This article here says something meaningful about AI and it also says something damning about doctors. Also there have been studies: https://www.sciencedirect.com/science/article/abs/pii/S019606442300642X?dgcid=author#fig1 https://www.sciencedirect.com/science/article/abs/pii/S01960...
- rvz 3y ago
  Wrong again. This is a case of reliability and just because there is one anecdote of the 'correct' diagnosis, it hardly means it is reliable overall. In fact you wouldn't use these AI systems and trust their judgement at face value without the involvement of human doctors because by default, you don't trust their outputs. This is on top of the fact that these AI systems are unexplainable black boxes which cannot transparently reason and explain their own decisions. It also appears that this 'research' study is quite frankly a surface level one with an extremely low sample size of 30 rather than the 1,000 sample size you suggested in your example. So I can only dismiss that study since it is not sufficient enough to result in conclusive evidence of reliability.
  - corethree 3y ago
    You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. 30 witnesses to a murder is highly reliable. Any judge or court who denies that is biased. 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability.
    - rvz 3y ago
      > You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? It is weak evidence and can be dismissed. That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. > 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability. So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases?
      - corethree 3y ago
        >Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. >That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? Bro, just use chatGPT a couple of times. You'll find it's reliability is roughly above 50% for queries and questions no other machine on the face of the earth can answer. It beats humans consistently at speed and in many instances where creativity is required. >So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Right now the machine is better only in certain instances. The delta from zero to now signifies another delta in the impending future.

corethree 3y ago

Wrong. One anecdote is one data point. If one person witnesses a murder that is a data point. The court doesn't just dismiss the data point because they need a sample size of 1000 witnesses for any statement to be meaningful. This article here says something meaningful about AI and it also says something damning about doctors. Also there have been studies: https://www.sciencedirect.com/science/article/abs/pii/S019606442300642X?dgcid=author#fig1 https://www.sciencedirect.com/science/article/abs/pii/S01960...

rvz 3y ago
Wrong again. This is a case of reliability and just because there is one anecdote of the 'correct' diagnosis, it hardly means it is reliable overall. In fact you wouldn't use these AI systems and trust their judgement at face value without the involvement of human doctors because by default, you don't trust their outputs. This is on top of the fact that these AI systems are unexplainable black boxes which cannot transparently reason and explain their own decisions. It also appears that this 'research' study is quite frankly a surface level one with an extremely low sample size of 30 rather than the 1,000 sample size you suggested in your example. So I can only dismiss that study since it is not sufficient enough to result in conclusive evidence of reliability.
- corethree 3y ago
  You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. 30 witnesses to a murder is highly reliable. Any judge or court who denies that is biased. 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability.
  - rvz 3y ago
    > You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? It is weak evidence and can be dismissed. That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. > 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability. So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases?
    - corethree 3y ago
      >Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. >That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? Bro, just use chatGPT a couple of times. You'll find it's reliability is roughly above 50% for queries and questions no other machine on the face of the earth can answer. It beats humans consistently at speed and in many instances where creativity is required. >So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Right now the machine is better only in certain instances. The delta from zero to now signifies another delta in the impending future.