4 ms·

>Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again b

>Biased to what? You gave ONE study with a very small sample size of 30, which is a far cry of the 1,000 sample size you suggested? Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. >That hardly means that the next study which another group attempts to reproduce is conclusive 'evidence' that it is more reliable than doctors. This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? Bro, just use chatGPT a couple of times. You'll find it's reliability is roughly above 50% for queries and questions no other machine on the face of the earth can answer. It beats humans consistently at speed and in many instances where creativity is required. >So this ONE anecdote immediately means that ChatGPT is far more reliable than human doctors and can be used for medical advice? How many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Right now the machine is better only in certain instances. The delta from zero to now signifies another delta in the impending future.

rvz 3y ago
> Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. That is not how this works. A sample size of 1 or 30 in ONE study in the medical industry is hardly convincing to physicians and other medical professions to conclusively prove that an experiment works reliably for others. Thus, you have been comparing apples to oranges all this time which research in the clinical practice does not apply to whatever you are trying to compare with in the legal practice. You need multiple, peer-reviewed and reproducible studies which verifies the claims set in the research paper and a larger sample size is favoured over an extremely small one. Not just 'past cases' or 'case studies'. That only shows that someone has done it within those disclosed limitations and in this case, it does NOT conclusively answer the reliability question. Henceforth, it scratches the surface and can be dismissed as insufficient evidence that does not prove reliability. > This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? As I have already explained you have just confused yourself with a nonsensical comparison right from the start. Had you actually read the paper you pasted instead of the title, you would have realize the non-determinstic nature of LLMs would mean that it is a limitation that would require further studies and methodologies (including larger more sample sizes) conclusively prove that ChatGPT is far more reliable than human doctors - even enough to fully replace them. So framing my point to believe as if I am attempting to prove a tautology isn't what this is. LLMs are non-deterministic black-box models that requires rigorous evaluations and experiments and suggesting that one surface-level study with a limited sample size as proof that LLMs like ChatGPT are more reliable than doctors is beyond ludicrous, especially using that shallow research paper as an example. > No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Again. One example isn't sufficient evidence to result into jumping to such wild conclusions of 'eating the entire medical diagnosis industry alive'. You have already admitted, it is not even a reliable or transparent enough to be used for medical advice especially with my question being unanswered. I'll be more specific: For every case it gives the correct output on a diagnosis, how many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? ONE case study which scratches the surface with a product that is an opaque non-deterministic tool, is something that physicians and clinicians are extremely skeptical on as fully replacement which not only it cannot fully replace them, it will always require a human doctor to check its diagnosis regardless, especially when something goes wrong.
- corethree 3y ago
  Bro. You are not addressing the dichotomy. Why is it in some instances you need stats and huge sample sizes and in other instances you need only one witness to convict someone for murder? All your doing is regurgitating the same bs everyone knows about the nature of science, statistics and sample sizes that are all blindingly obvious to anyone. I am presenting to you a valid dilemma that your big brain is skipping over because of your blind loyalty to science. You do not need a scientific study to tell you the sky is blue. A court doesn't need a thousand witnesses to convict a murder. Have you ever wondered why? Do you need me to give you 1000 IQ questions to verify that you are an intelligent being? Do you need to give your mother or father those same tests to verify their intelligence? You have common sense right? You can talk to your mom and basically bypass science to verify her intelligence right? Why the hell all of a sudden do you need a rigorous scientific study to verify the intellectual abilities of chatGPT? Your so smart that you can know your mother is intelligent without assessing her with 5000 iq questions but for chatGPT you suddenly need the raw power of scientific rigor to come to any conclusion? You can't just talk to it yourself and make your own conclusion? Bro. You're irrational. chatGPT beat 17 doctors in seconds and it doesn't even phase you. But your mom who has likely never taken an IQ test doesn't need one test to verify her intelligence. Go above the level of scientific rigor. Einstein didn't need sample sizes and statistics to speculate about black holes and general relativity. The verification came later, but the math and nature of reality was formulated and judged correct through common sense I described above. This was way before anything Einstein proposed was verified by "stats". Do you not have the ability to bypass statistical data and formulate conclusions without it? Looks like no.
  - rvz 3y ago
    > Bro. You are not addressing the dichotomy. Why is it in some instances you need stats and huge sample sizes and in other instances you need only one witness to convict someone for murder? Again, you're continuing to compare apples and oranges from different professions and applying it here which it is nonsensical. The unpredictable behaviour of LLMs like ChatGPT tells us that not only it is non-deterministic but it cannot be trusted at all and in that context, it always requires humans to check it and for the case of how reliable it is it needs much more experiments and scientific methods by others to attest that. This is exactly why medical professionals in this case will laugh at your question. For clinicians to use one study as the truth and basis with a low sample of 1 or 30 to show whether a medical device is reliable, especially an AI as a medical device is beyond ridiculous. > All your doing is regurgitating the same bs everyone knows about the nature of science, statistics and sample sizes that are all blindingly obvious to anyone. So why aren't you able to understand that then? Except that your mistake was to begin by comparing the research methods in the legal profession and the medical profession and to use that in this case as a flawed analogy to show that 'reliability' is the same in both of them. Which I'm afraid you only confused yourself. > I am presenting to you a valid dilemma... Which again is irrelevant and besides the point. Everything after that is on the point of what I've already said on transparent explainability, which it is known that LLMs and AI models such as ChatGPT cannot reason or explain their decisions transparently, and thus need examination and further reproducibility by others due to their their unpredictable behaviour. Such a system used for medical advice is quite frankly unsatisfactory for physicians and other medical professionals. Just because it worked for someone does not mean it is reliable and works for everyone else. Hence me asking you for a significantly larger sample size and more clinical research papers with ChatGPT being used. > Bro. You're irrational. chatGPT beat 17 doctors in seconds and it doesn't even phase you. But your mom who has likely never taken an IQ test doesn't need one test to verify her intelligence. I'm not the one drawing wild conclusions of reliability over one study and then suggesting that we can replace all doctors with ChatGPT just because of one anecdote showing it was correct for one it is also correct for others, which given its unpredictability that in itself is beyond illogical. You clearly are doing just that. As long as it is a black-box AI model, physicians and medical professionals will always scrutinise its unpredictable nature and reliability rather than you trusting whatever diagnosis it gives as the truth. > Go above the level of scientific rigor. Einstein didn't need sample sizes and statistics to speculate about black holes and general relativity. The verification came later, but the math and nature of reality was formulated and judged correct through common sense I described above. This was way before anything Einstein proposed was verified by "stats". What does Einstein speculating his equations have to do with showing how an AI system that is non-deterministic is also reliable? There are different methods in showing this reliability as I have explained already and you bringing that up is an irrelevant distraction. > Do you not have the ability to bypass statistical data and formulate conclusions without it? Looks like no. The entire point IS conclusively showing reliability which you using ONE paper with a low sample size you used for that as the basis of that claim is laughably insufficient for clinicians to draw an overall conclusion to show how ChatGPT is more reliable than doctors to the point where it is safe for medical advice or completely replacing doctors (which isn't going to happen anyway).

rvz 3y ago

> Go read what I said again because you completely misinterpreted. I said a 1000 sample size isn't needed by a judge. A judge can convict a murderer off of 1 witness. That is not how this works. A sample size of 1 or 30 in ONE study in the medical industry is hardly convincing to physicians and other medical professions to conclusively prove that an experiment works reliably for others. Thus, you have been comparing apples to oranges all this time which research in the clinical practice does not apply to whatever you are trying to compare with in the legal practice. You need multiple, peer-reviewed and reproducible studies which verifies the claims set in the research paper and a larger sample size is favoured over an extremely small one. Not just 'past cases' or 'case studies'. That only shows that someone has done it within those disclosed limitations and in this case, it does NOT conclusively answer the reliability question. Henceforth, it scratches the surface and can be dismissed as insufficient evidence that does not prove reliability. > This is what gets me. Do some people really go through reality only deeming things that have been under the lens of a scientific study to be real? Is the sky blue? Are you alive? Do you need a scientific study to prove it to you? Are you so enamored with science that the only way you can speculate or debate a topic is if the scientific method has been applied rigorously? As I have already explained you have just confused yourself with a nonsensical comparison right from the start. Had you actually read the paper you pasted instead of the title, you would have realize the non-determinstic nature of LLMs would mean that it is a limitation that would require further studies and methodologies (including larger more sample sizes) conclusively prove that ChatGPT is far more reliable than human doctors - even enough to fully replace them. So framing my point to believe as if I am attempting to prove a tautology isn't what this is. LLMs are non-deterministic black-box models that requires rigorous evaluations and experiments and suggesting that one surface-level study with a limited sample size as proof that LLMs like ChatGPT are more reliable than doctors is beyond ludicrous, especially using that shallow research paper as an example. > No but a sample size of 30 and beating out 17 doctors spending a decade analyzing one child in mere seconds is reasonable evidence that chatGPT is the precursor to the thing that will eat the entire medical diagnosis industry alive. Again. One example isn't sufficient evidence to result into jumping to such wild conclusions of 'eating the entire medical diagnosis industry alive'. You have already admitted, it is not even a reliable or transparent enough to be used for medical advice especially with my question being unanswered. I'll be more specific: For every case it gives the correct output on a diagnosis, how many cases are there when ChatGPT gets its response incorrect out of 1,000 cases? ONE case study which scratches the surface with a product that is an opaque non-deterministic tool, is something that physicians and clinicians are extremely skeptical on as fully replacement which not only it cannot fully replace them, it will always require a human doctor to check its diagnosis regardless, especially when something goes wrong.

corethree 3y ago
Bro. You are not addressing the dichotomy. Why is it in some instances you need stats and huge sample sizes and in other instances you need only one witness to convict someone for murder? All your doing is regurgitating the same bs everyone knows about the nature of science, statistics and sample sizes that are all blindingly obvious to anyone. I am presenting to you a valid dilemma that your big brain is skipping over because of your blind loyalty to science. You do not need a scientific study to tell you the sky is blue. A court doesn't need a thousand witnesses to convict a murder. Have you ever wondered why? Do you need me to give you 1000 IQ questions to verify that you are an intelligent being? Do you need to give your mother or father those same tests to verify their intelligence? You have common sense right? You can talk to your mom and basically bypass science to verify her intelligence right? Why the hell all of a sudden do you need a rigorous scientific study to verify the intellectual abilities of chatGPT? Your so smart that you can know your mother is intelligent without assessing her with 5000 iq questions but for chatGPT you suddenly need the raw power of scientific rigor to come to any conclusion? You can't just talk to it yourself and make your own conclusion? Bro. You're irrational. chatGPT beat 17 doctors in seconds and it doesn't even phase you. But your mom who has likely never taken an IQ test doesn't need one test to verify her intelligence. Go above the level of scientific rigor. Einstein didn't need sample sizes and statistics to speculate about black holes and general relativity. The verification came later, but the math and nature of reality was formulated and judged correct through common sense I described above. This was way before anything Einstein proposed was verified by "stats". Do you not have the ability to bypass statistical data and formulate conclusions without it? Looks like no.
- rvz 3y ago
  > Bro. You are not addressing the dichotomy. Why is it in some instances you need stats and huge sample sizes and in other instances you need only one witness to convict someone for murder? Again, you're continuing to compare apples and oranges from different professions and applying it here which it is nonsensical. The unpredictable behaviour of LLMs like ChatGPT tells us that not only it is non-deterministic but it cannot be trusted at all and in that context, it always requires humans to check it and for the case of how reliable it is it needs much more experiments and scientific methods by others to attest that. This is exactly why medical professionals in this case will laugh at your question. For clinicians to use one study as the truth and basis with a low sample of 1 or 30 to show whether a medical device is reliable, especially an AI as a medical device is beyond ridiculous. > All your doing is regurgitating the same bs everyone knows about the nature of science, statistics and sample sizes that are all blindingly obvious to anyone. So why aren't you able to understand that then? Except that your mistake was to begin by comparing the research methods in the legal profession and the medical profession and to use that in this case as a flawed analogy to show that 'reliability' is the same in both of them. Which I'm afraid you only confused yourself. > I am presenting to you a valid dilemma... Which again is irrelevant and besides the point. Everything after that is on the point of what I've already said on transparent explainability, which it is known that LLMs and AI models such as ChatGPT cannot reason or explain their decisions transparently, and thus need examination and further reproducibility by others due to their their unpredictable behaviour. Such a system used for medical advice is quite frankly unsatisfactory for physicians and other medical professionals. Just because it worked for someone does not mean it is reliable and works for everyone else. Hence me asking you for a significantly larger sample size and more clinical research papers with ChatGPT being used. > Bro. You're irrational. chatGPT beat 17 doctors in seconds and it doesn't even phase you. But your mom who has likely never taken an IQ test doesn't need one test to verify her intelligence. I'm not the one drawing wild conclusions of reliability over one study and then suggesting that we can replace all doctors with ChatGPT just because of one anecdote showing it was correct for one it is also correct for others, which given its unpredictability that in itself is beyond illogical. You clearly are doing just that. As long as it is a black-box AI model, physicians and medical professionals will always scrutinise its unpredictable nature and reliability rather than you trusting whatever diagnosis it gives as the truth. > Go above the level of scientific rigor. Einstein didn't need sample sizes and statistics to speculate about black holes and general relativity. The verification came later, but the math and nature of reality was formulated and judged correct through common sense I described above. This was way before anything Einstein proposed was verified by "stats". What does Einstein speculating his equations have to do with showing how an AI system that is non-deterministic is also reliable? There are different methods in showing this reliability as I have explained already and you bringing that up is an irrelevant distraction. > Do you not have the ability to bypass statistical data and formulate conclusions without it? Looks like no. The entire point IS conclusively showing reliability which you using ONE paper with a low sample size you used for that as the basis of that claim is laughably insufficient for clinicians to draw an overall conclusion to show how ChatGPT is more reliable than doctors to the point where it is safe for medical advice or completely replacing doctors (which isn't going to happen anyway).