3 ms·

I my mind, the AI only needs to be more reliable than the human expert, and doctors and lawyers and other experts are "only human". There is only so much a hum

I my mind, the AI only needs to be more reliable than the human expert, and doctors and lawyers and other experts are "only human". There is only so much a human can read and remember. There is only so much they can be convinced to change their mind in the face of new evidence. They get tired. They get cranky. They have biases, and fears. Some doctors are just terrible. I think the AI is probably more reliable than them already. I don't think we are very far away from being able to correct an AI, and for it to remember that correction forever.

dgb23 4y ago
Apparently much of the usefulness of ChatGPT comes from human guidance via reinforcement learning. The base model seems to be quite useless, but with only a little bit of human input it becomes deceptively smart.
- PeterisP 4y ago
  The key purpose for the reinforcement learning part (RLHF) is so that it would be socially acceptable to provide the model to the general public without getting into a PR nightmare like Microsoft Tay; that post-training does not make it "deceptively smart", it trades off a bit of smartness in order to ensure some alignment with certain restrictions. It decreases the performance on some tasks and e.g. the GPT-4 paper, which is very light on other details, provides some experimental evidence that this post-training significantly hurts the model's confidence calibration, which decreases its usability for tasks where you want to know how certain the model is.
recursive_loops 4y ago
I totally agree, but we're going to need to figure out better ways to verify correctness of what the AI tells us. This will be pretty hard since we can't hardly do that with human-produced stuff either. And further if the AI is trained on human-produced stuff that contains lots of errors... I don't really know how we're going to do it.
- jay_kyburz 4y ago
  If I were to hazard a guess, I would say, at first, companies are going to pay human experts to sit down with AI, ask it questions, and correct any mistakes. It might look like a conversation where a panel of human experts and the AI agree on whats true and whats not. Like scientific consensus. Eventually the AI will just run its own experiments and test its own hypothesisis.
kelnos 4y ago
I suspect an ER doctor who misdiagnosed patients 50% of the time would not be an ER doctor for very long. Also consider that, in this case, the 50% the AI got right was defined such somewhat generously. For each patient, the AI presented several possible diagnoses, only one of which was correct. If the AI were making all the decisions, would the AI also decide to treat the correct ailment in all those cases? In an emergency situation, you may not have the opportunity to go back and try other options if you're wrong the first time.
- vidarh 4y ago
  But it's also a ridiculous assessment. How much training material for ER doctors was in the training set? I wonder what my rate would be (having never done a single course in medicine)... It should be totally unsurprising that it produced bad results. That it got as much right is an indication there's a good chance you can do reasonably well by training a model with a lot of actual case studies.
  - lerchmo 4y ago
    Even providing it with a tool to search and load its context with relevant training textbooks / papers.

dgb23 4y ago

Apparently much of the usefulness of ChatGPT comes from human guidance via reinforcement learning. The base model seems to be quite useless, but with only a little bit of human input it becomes deceptively smart.

PeterisP 4y ago
The key purpose for the reinforcement learning part (RLHF) is so that it would be socially acceptable to provide the model to the general public without getting into a PR nightmare like Microsoft Tay; that post-training does not make it "deceptively smart", it trades off a bit of smartness in order to ensure some alignment with certain restrictions. It decreases the performance on some tasks and e.g. the GPT-4 paper, which is very light on other details, provides some experimental evidence that this post-training significantly hurts the model's confidence calibration, which decreases its usability for tasks where you want to know how certain the model is.

recursive_loops 4y ago

I totally agree, but we're going to need to figure out better ways to verify correctness of what the AI tells us. This will be pretty hard since we can't hardly do that with human-produced stuff either. And further if the AI is trained on human-produced stuff that contains lots of errors... I don't really know how we're going to do it.

jay_kyburz 4y ago
If I were to hazard a guess, I would say, at first, companies are going to pay human experts to sit down with AI, ask it questions, and correct any mistakes. It might look like a conversation where a panel of human experts and the AI agree on whats true and whats not. Like scientific consensus. Eventually the AI will just run its own experiments and test its own hypothesisis.

kelnos 4y ago

I suspect an ER doctor who misdiagnosed patients 50% of the time would not be an ER doctor for very long. Also consider that, in this case, the 50% the AI got right was defined such somewhat generously. For each patient, the AI presented several possible diagnoses, only one of which was correct. If the AI were making all the decisions, would the AI also decide to treat the correct ailment in all those cases? In an emergency situation, you may not have the opportunity to go back and try other options if you're wrong the first time.

vidarh 4y ago
But it's also a ridiculous assessment. How much training material for ER doctors was in the training set? I wonder what my rate would be (having never done a single course in medicine)... It should be totally unsurprising that it produced bad results. That it got as much right is an indication there's a good chance you can do reasonably well by training a model with a lot of actual case studies.
- lerchmo 4y ago
  Even providing it with a tool to search and load its context with relevant training textbooks / papers.