4 ms·

Whether you (or anyone else) actually understands something is opaque to me. For all I know, your comment was left by an AI. There is no magic wand we can wave

Whether you (or anyone else) actually understands something is opaque to me. For all I know, your comment was left by an AI. There is no magic wand we can wave to separate out the people who actually understand something (whatever that means), from the people who just appear to. The best we can do is test a human's mastery of a subject to estimate how well they actually know and understand that topic. Which is exactly what OpenAI is doing here. What I care about is the results. If the "grade" is 10%, then I don't want to rely on it, whether it's a human or an AI. If it's 95%, then I feel fine about relying on it. Especially since I suspect that very soon, most doctors would not score as well on a benchmark like this as the SOTA models.

imiric 1y ago
> What I care about is the results. If the "grade" is 10%, then I don't want to rely on it, whether it's a human or an AI. So you judge mastery of a subject by some arbitrary grade? Produced, incidentally, by the same company that trained the model. But I'm sure they wouldn't cheat... I also care about the results, but I judge it based on direct experience. IME the current SOTA models can't be relied on to accurately produce working code most of the time, yet I'm expected to trust they can produce accurate healthcare diagnosis and advice? Give me a break. AI companies have sold us a fantasy that a neat computer trick, which can still be very useful for society, will somehow lead to AGI. Now that we're a decade into this hype cycle and investors are getting antsy, they're doubling down by anthropomorphizing the technology and selling us "chain of thought" and "reasoning", as if making everyone say these things will somehow magically produce intelligence. What's really irresponsible is that the public and governments are buying into it, which is pushing the tech into industries that can actually harm people. Safety is not just about trusting some arbitrary grade.
- senordevnyc 1y ago
  IME the current SOTA models can't be relied on to accurately produce working code most of the time My experience is the opposite.
- famouswaffles 1y ago
  >Now that we're a decade into this hype cycle and investors are getting antsy, they're doubling down by anthropomorphizing the technology and selling us "chain of thought" and "reasoning", as if making everyone say these things will somehow magically produce intelligence. The Transformer, nevermind GPT-3 did not exist a decade ago. I guess LLMs aren't the only things that hallucinate and spout confident nonsense. >I also care about the results, but I judge it based on direct experience. IME the current SOTA models can't be relied on to accurately produce working code most of the time, yet I'm expected to trust they can produce accurate healthcare diagnosis and advice? Give me a break. You don't see the fallacy of forcing your experience in one domain onto a completely unrelated one regardless of any evidence to the contrary (even if you don't trust open ai, this is hardly the only paper trialing SOTA LLMS for diagnosis)? What does code have to do with diagnosis ? And while the current SOTA is by no means perfect, if you can't get them to produce working code, that's a you problem. At the very least, many users would disagree. Yeah benchmarks aren't perfect. Doesn't mean they aren't useful. Certainly a lot more useful than your approach.
  - imiric 1y ago
    > The Transformer, nevermind GPT-3 did not exist a decade ago. I'm attributing the start of the current AI hype cycle to the resurgence of CNNs using GPUs, roughly around the time of AlexNet and AlphaGo, not to LLMs and the Transformer architecture. Though if we're being really pedantic, the original Transformer paper is from 2017, so almost a decade ago. But this is beside my point. > You don't see the fallacy of forcing your experience in one domain onto a completely unrelated one The machine has no concept of a "domain". Whether it's outputting code, poetry, images, or video, it's all data generated by probabilistic pattern matching and pseudo-randomness. The structure and accuracy of the generated data is meaningful only to humans, and it's the most important factor that is measured in all these benchmarks. We might find it collectively amusing when an AI produces funny text and weird looking pictures and video. Some might find it acceptable when it produces buggy code that humans need to fix, or when it mimics an expert by confidently spouting nonsense, which is where I personally draw the line. But we should all be concerned when the same models are used in industries where human lives depend on critical thinking by experts. We have been promised fully autonomous vehicles for more than a decade now, and only in the last couple of years have some parts of that promise begun to come true in very limited scenarios. We're all understandably reluctant to give control of a speeding 2-ton object to a machine, for obvious reasons. The process has been very gradual, with a lot of oversight and regulation, as there should be. All I'm saying is that there should be an equal amount of oversight in other industries as well, particularly healthcare. Arbitrary benchmarks don't make me trust these systems more, regardless of who produces them.

imiric 1y ago

> What I care about is the results. If the "grade" is 10%, then I don't want to rely on it, whether it's a human or an AI. So you judge mastery of a subject by some arbitrary grade? Produced, incidentally, by the same company that trained the model. But I'm sure they wouldn't cheat... I also care about the results, but I judge it based on direct experience. IME the current SOTA models can't be relied on to accurately produce working code most of the time, yet I'm expected to trust they can produce accurate healthcare diagnosis and advice? Give me a break. AI companies have sold us a fantasy that a neat computer trick, which can still be very useful for society, will somehow lead to AGI. Now that we're a decade into this hype cycle and investors are getting antsy, they're doubling down by anthropomorphizing the technology and selling us "chain of thought" and "reasoning", as if making everyone say these things will somehow magically produce intelligence. What's really irresponsible is that the public and governments are buying into it, which is pushing the tech into industries that can actually harm people. Safety is not just about trusting some arbitrary grade.

senordevnyc 1y ago
IME the current SOTA models can't be relied on to accurately produce working code most of the time My experience is the opposite.
famouswaffles 1y ago
>Now that we're a decade into this hype cycle and investors are getting antsy, they're doubling down by anthropomorphizing the technology and selling us "chain of thought" and "reasoning", as if making everyone say these things will somehow magically produce intelligence. The Transformer, nevermind GPT-3 did not exist a decade ago. I guess LLMs aren't the only things that hallucinate and spout confident nonsense. >I also care about the results, but I judge it based on direct experience. IME the current SOTA models can't be relied on to accurately produce working code most of the time, yet I'm expected to trust they can produce accurate healthcare diagnosis and advice? Give me a break. You don't see the fallacy of forcing your experience in one domain onto a completely unrelated one regardless of any evidence to the contrary (even if you don't trust open ai, this is hardly the only paper trialing SOTA LLMS for diagnosis)? What does code have to do with diagnosis ? And while the current SOTA is by no means perfect, if you can't get them to produce working code, that's a you problem. At the very least, many users would disagree. Yeah benchmarks aren't perfect. Doesn't mean they aren't useful. Certainly a lot more useful than your approach.
- imiric 1y ago
  > The Transformer, nevermind GPT-3 did not exist a decade ago. I'm attributing the start of the current AI hype cycle to the resurgence of CNNs using GPUs, roughly around the time of AlexNet and AlphaGo, not to LLMs and the Transformer architecture. Though if we're being really pedantic, the original Transformer paper is from 2017, so almost a decade ago. But this is beside my point. > You don't see the fallacy of forcing your experience in one domain onto a completely unrelated one The machine has no concept of a "domain". Whether it's outputting code, poetry, images, or video, it's all data generated by probabilistic pattern matching and pseudo-randomness. The structure and accuracy of the generated data is meaningful only to humans, and it's the most important factor that is measured in all these benchmarks. We might find it collectively amusing when an AI produces funny text and weird looking pictures and video. Some might find it acceptable when it produces buggy code that humans need to fix, or when it mimics an expert by confidently spouting nonsense, which is where I personally draw the line. But we should all be concerned when the same models are used in industries where human lives depend on critical thinking by experts. We have been promised fully autonomous vehicles for more than a decade now, and only in the last couple of years have some parts of that promise begun to come true in very limited scenarios. We're all understandably reluctant to give control of a speeding 2-ton object to a machine, for obvious reasons. The process has been very gradual, with a lot of oversight and regulation, as there should be. All I'm saying is that there should be an equal amount of oversight in other industries as well, particularly healthcare. Arbitrary benchmarks don't make me trust these systems more, regardless of who produces them.