4 ms·

You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. 30 witnesses to a murder is highly reliable. Any j

You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. 30 witnesses to a murder is highly reliable. Any judge or court who denies that is biased. 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability.

rvz 3y ago
> You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? It is weak evidence and can be dismissed. That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. > 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability. So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases?
- corethree 3y ago
  >Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. >That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? Bro, just use chatGPT a couple of times. You'll find it's reliability is roughly above 50% for queries and questions no other machine on the face of the earth can answer. It beats humans consistently at speed and in many instances where creativity is required. >So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Right now the machine is better only in certain instances. The delta from zero to now signifies another delta in the impending future.
  - rvz 3y ago
    > Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. That is not how this works. A sample size of 1 or 30 in ONE study in the medical industry is hardly convincing to physicians and other medical professions to conclusively prove that an experiment works reliably for others. Thus, you have been comparing apples to oranges all this time which research in the clinical practice does not apply to whatever you are trying to compare with in the legal practice. You need multiple, peer-reviewed and reproducible studies which verifies the claims set in the research paper and a larger sample size is favoured over an extremely small one. Not just 'past cases' or 'case studies'. That only shows that someone has done it within those disclosed limitations and in this case, it does NOT conclusively answer the reliability question. Henceforth, it scratches the surface and can be dismissed as insufficient evidence that does not prove reliability. > This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? As I have already explained you have just confused yourself with a nonsensical comparison right from the start. Had you actually read the paper you pasted instead of the title, you would have realize the non-determinstic nature of LLMs would mean that it is a limitation that would require further studies and methodologies (including larger more sample sizes) conclusively prove that ChatGPT is far more reliable than human doctors - even enough to fully replace them. So framing my point to believe as if I am attempting to prove a tautology isn't what this is. LLMs are non-deterministic black-box models that requires rigorous evaluations and experiments and suggesting that one surface-level study with a limited sample size as proof that LLMs like ChatGPT are more reliable than doctors is beyond ludicrous, especially using that shallow research paper as an example. > No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Again. One example isn't sufficient evidence to result into jumping to such wild conclusions of 'eating the entire medical diagnosis industry alive'. You have already admitted, it is not even a reliable or transparent enough to be used for medical advice especially with my question being unanswered. I'll be more specific: For every case it gives the correct output on a diagnosis, how many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? ONE case study which scratches the surface with a product that is an opaque non-deterministic tool, is something that physicians and clinicians are extremely skeptical on as fully replacement which not only it cannot fully replace them, it will always require a human doctor to check its diagnosis regardless, especially when something goes wrong.
- rowanG077 3y ago
  > So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? You couldn't find a more clear case of straw man. No one said ChatGPT is more reliable then human doctors. No one said anything about incorrect responses. The fact of the matter is that ChatGPT has diagnosed a disease which 17(!!!) doctors have missed. Even if the incorrect responses is 999/1000 cases it still worth it to include ChatGPT in this process since it's so cheap.
  - rvz 3y ago
    > You couldn't find a more clear case of straw man. No one said ChatGPT is more reliable then human doctors. No one said anything about incorrect responses. Both the shallow research paper and that anecdote point is all focused on the basis of reliability. Henceforth, it doesn't discredit my point and question. So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? > The fact of the matter is that ChatGPT has diagnosed a disease which 17(!!!) doctors have missed. Even if the incorrect responses is 999/1000 cases it still worth it to include ChatGPT in this process since it's so cheap. So obviously as expected you still need human doctors regardless, as LLMs are opaque black-boxes which have a lack of transparent reasoning with unpredictable outputs. Thus how does that mean it can be used for medical advice or not? Overall, this is a clear case of reliability to trust the output of an LLM and the entire point of the contrary is that for every 'correct' diagnosis made by the LLM, there are incorrect and vacuous responses which on top of its non-deterministic outputs comes with its lack of transparent explainability other than repeating on what it has been trained on to convince lesser expert users.
    - rowanG077 3y ago
      > So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? No one is saying chatGPT is more reliable then Doctors. Please keep this discussion grounded in reality. > So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? It doesn't. It shows chatGPT beating 17 doctors in this case. That has NOTHING to do with reliability and that assumption is a big leap of logic you, and only you, are making. > So obviously as expected you still need human doctors regardless, as LLMs are opaque black-boxes which have a lack of transparent reasoning with unpredictable outputs. Thus how does that mean it can be used for medical advice or not? Yes, of course. That doesn't mean chatGPT couldn't be incorporated into doctors work flow and provide tangible value. No one is saying chatGPT should replace doctors. Stop arguing against made up air castles. You are only fooling yourself.

rvz 3y ago

> You are dismissing the study not because you need 1000 samples. You are dismissing it because you're biased. Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? It is weak evidence and can be dismissed. That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. > 17 doctors who couldn't diagnose a simple issue for 10 years is a demonstration of total unreliability. So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases?

corethree 3y ago
>Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. >That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? Bro, just use chatGPT a couple of times. You'll find it's reliability is roughly above 50% for queries and questions no other machine on the face of the earth can answer. It beats humans consistently at speed and in many instances where creativity is required. >So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Right now the machine is better only in certain instances. The delta from zero to now signifies another delta in the impending future.
- rvz 3y ago
  > Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. That is not how this works. A sample size of 1 or 30 in ONE study in the medical industry is hardly convincing to physicians and other medical professions to conclusively prove that an experiment works reliably for others. Thus, you have been comparing apples to oranges all this time which research in the clinical practice does not apply to whatever you are trying to compare with in the legal practice. You need multiple, peer-reviewed and reproducible studies which verifies the claims set in the research paper and a larger sample size is favoured over an extremely small one. Not just 'past cases' or 'case studies'. That only shows that someone has done it within those disclosed limitations and in this case, it does NOT conclusively answer the reliability question. Henceforth, it scratches the surface and can be dismissed as insufficient evidence that does not prove reliability. > This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? As I have already explained you have just confused yourself with a nonsensical comparison right from the start. Had you actually read the paper you pasted instead of the title, you would have realize the non-determinstic nature of LLMs would mean that it is a limitation that would require further studies and methodologies (including larger more sample sizes) conclusively prove that ChatGPT is far more reliable than human doctors - even enough to fully replace them. So framing my point to believe as if I am attempting to prove a tautology isn't what this is. LLMs are non-deterministic black-box models that requires rigorous evaluations and experiments and suggesting that one surface-level study with a limited sample size as proof that LLMs like ChatGPT are more reliable than doctors is beyond ludicrous, especially using that shallow research paper as an example. > No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Again. One example isn't sufficient evidence to result into jumping to such wild conclusions of 'eating the entire medical diagnosis industry alive'. You have already admitted, it is not even a reliable or transparent enough to be used for medical advice especially with my question being unanswered. I'll be more specific: For every case it gives the correct output on a diagnosis, how many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? ONE case study which scratches the surface with a product that is an opaque non-deterministic tool, is something that physicians and clinicians are extremely skeptical on as fully replacement which not only it cannot fully replace them, it will always require a human doctor to check its diagnosis regardless, especially when something goes wrong.
rowanG077 3y ago
> So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? You couldn't find a more clear case of straw man. No one said ChatGPT is more reliable then human doctors. No one said anything about incorrect responses. The fact of the matter is that ChatGPT has diagnosed a disease which 17(!!!) doctors have missed. Even if the incorrect responses is 999/1000 cases it still worth it to include ChatGPT in this process since it's so cheap.
- rvz 3y ago
  > You couldn't find a more clear case of straw man. No one said ChatGPT is more reliable then human doctors. No one said anything about incorrect responses. Both the shallow research paper and that anecdote point is all focused on the basis of reliability. Henceforth, it doesn't discredit my point and question. So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? > The fact of the matter is that ChatGPT has diagnosed a disease which 17(!!!) doctors have missed. Even if the incorrect responses is 999/1000 cases it still worth it to include ChatGPT in this process since it's so cheap. So obviously as expected you still need human doctors regardless, as LLMs are opaque black-boxes which have a lack of transparent reasoning with unpredictable outputs. Thus how does that mean it can be used for medical advice or not? Overall, this is a clear case of reliability to trust the output of an LLM and the entire point of the contrary is that for every 'correct' diagnosis made by the LLM, there are incorrect and vacuous responses which on top of its non-deterministic outputs comes with its lack of transparent explainability other than repeating on what it has been trained on to convince lesser expert users.
  - rowanG077 3y ago
    > So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? No one is saying chatGPT is more reliable then Doctors. Please keep this discussion grounded in reality. > So how does one anecdotal point show that ChatGPT is more reliable than human doctors? If that is not the point to these celebrations, mentions, etc that both of you are doing then I can assume that it is far from the case then, unless you have a direct answer to that question? It doesn't. It shows chatGPT beating 17 doctors in this case. That has NOTHING to do with reliability and that assumption is a big leap of logic you, and only you, are making. > So obviously as expected you still need human doctors regardless, as LLMs are opaque black-boxes which have a lack of transparent reasoning with unpredictable outputs. Thus how does that mean it can be used for medical advice or not? Yes, of course. That doesn't mean chatGPT couldn't be incorporated into doctors work flow and provide tangible value. No one is saying chatGPT should replace doctors. Stop arguing against made up air castles. You are only fooling yourself.