6 ms·

A couple things: 1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, with

A couple things: 1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? Same as a lot of vision models are trained on Imagenet and then fine tuned for the application, but you would never expect that imagenet weights would just automatically perform whatever specific task you had. 2. Where would this chatbot ever be a good idea? Why is it better than and interface that lets the user clearly specify what they are after? Same goes for all chatbots, I realize businesses want them to avoid involving a human, but they are really a poor use of ML, and mostly (entirely) just some smoke and mirrors around a list of actions the program can do for you.

joe_the_user 6y ago
1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? With due respect, I don't think that's the problem. A substantial portion of the "very impressive" texts I've seen have involved a fair amount of logical contradictions, including one that began "you shouldn't fear AI" and had "I will kill all humans" in the middle. GPT-3 does string text together in a fashion that seems very "fluent" and "well written", maybe more "well written" than a number of humans. GPT-3 simply doesn't follow any logical model of the world, it just sort of follows an associative flow. Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly.
- k__ 6y ago
  I guess, it's a bit like images of people who don't exist. It looks like it makes sense, but in the end there is nothing behind it. And this gets more obvious with text.
- acdha 6y ago
  > Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly. I think it might even make it worse: if something is obviously unnatural in a context, the reader will be less inclined to trust it. If GPT-3 used more valid terms and phrasing common in the field, it might lead someone to trust it more than they should, especially if the error rates are low enough that routine sets in.
minimaxir 6y ago
> Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? That was the unexpected result of training GPT-3 (zero-shot learning). Finetuning in theory would give better results, though.
exporectomy 6y ago
Regarding 2. That's what doctor's receptionists do all day. They must be doing it for some reason. I guess old people don't like annoying web forms? I'm old and I don't. For example, I've formed a habit of opening terms and conditions links in another tab because I've experienced forms that clear your data when you click them directly then try to to "back" afterwards. But just a few weeks ago, I did that and when I returned to submit the form, it was gone with a message telling me I'd opened another tab and had better close them all and start again. Web forms are full of aggravating problems like that. Web developers have had 30 years to get this right and they still can't, so I don't have much hope for the next 30 years. On the other hand, a whole new technology seems more promising.
- otabdeveloper4 6y ago
  Web forms work spectacularly well when you stick to, well, actual web HTML form elements. The problem is frontend developers reinventing their own widgets and protocols because "moronic SPA reasons".
  - perl4ever 6y ago
    One little thing that aggravates me is that you can't type a sequence of letters and have a drop down menu go to the appropriate choice most of the time. Like, I can type "N" and it will select the first choice beginning with "N", but I can't type "New" and go to the first one beginning with "New". This functionality was standard on the Macintosh maybe 30+ years ago, but I feel like 80% of the time on the web, it isn't.
boudin 6y ago
Chatbots are used a lot already for simple tasks, like the one mentionned in this article (booking apointments online). The issue is not that the bot is not able to replace a medical specialist, but making sure the bot will not answer something totally wrong in such a context. In this example, for a medical practice wanting to use a chatbot to automate apointments, you would want to be sure that it will never answer any other questions, especially sensitive ones.
DanBC 6y ago
> . Where would this chatbot ever be a good idea? This chatbot is never a good idea, but techbros (and you can sometimes read them on HN) think medicine is simple and that we could have AI triage in front of human healthcare professionals.

joe_the_user 6y ago

1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? With due respect, I don't think that's the problem. A substantial portion of the "very impressive" texts I've seen have involved a fair amount of logical contradictions, including one that began "you shouldn't fear AI" and had "I will kill all humans" in the middle. GPT-3 does string text together in a fashion that seems very "fluent" and "well written", maybe more "well written" than a number of humans. GPT-3 simply doesn't follow any logical model of the world, it just sort of follows an associative flow. Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly.

k__ 6y ago
I guess, it's a bit like images of people who don't exist. It looks like it makes sense, but in the end there is nothing behind it. And this gets more obvious with text.
acdha 6y ago
> Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly. I think it might even make it worse: if something is obviously unnatural in a context, the reader will be less inclined to trust it. If GPT-3 used more valid terms and phrasing common in the field, it might lead someone to trust it more than they should, especially if the error rates are low enough that routine sets in.

minimaxir 6y ago

> Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? That was the unexpected result of training GPT-3 (zero-shot learning). Finetuning in theory would give better results, though.

exporectomy 6y ago

Regarding 2. That's what doctor's receptionists do all day. They must be doing it for some reason. I guess old people don't like annoying web forms? I'm old and I don't. For example, I've formed a habit of opening terms and conditions links in another tab because I've experienced forms that clear your data when you click them directly then try to to "back" afterwards. But just a few weeks ago, I did that and when I returned to submit the form, it was gone with a message telling me I'd opened another tab and had better close them all and start again. Web forms are full of aggravating problems like that. Web developers have had 30 years to get this right and they still can't, so I don't have much hope for the next 30 years. On the other hand, a whole new technology seems more promising.

otabdeveloper4 6y ago
Web forms work spectacularly well when you stick to, well, actual web HTML form elements. The problem is frontend developers reinventing their own widgets and protocols because "moronic SPA reasons".
- perl4ever 6y ago
  One little thing that aggravates me is that you can't type a sequence of letters and have a drop down menu go to the appropriate choice most of the time. Like, I can type "N" and it will select the first choice beginning with "N", but I can't type "New" and go to the first one beginning with "New". This functionality was standard on the Macintosh maybe 30+ years ago, but I feel like 80% of the time on the web, it isn't.

boudin 6y ago

Chatbots are used a lot already for simple tasks, like the one mentionned in this article (booking apointments online). The issue is not that the bot is not able to replace a medical specialist, but making sure the bot will not answer something totally wrong in such a context. In this example, for a medical practice wanting to use a chatbot to automate apointments, you would want to be sure that it will never answer any other questions, especially sensitive ones.

DanBC 6y ago

> . Where would this chatbot ever be a good idea? This chatbot is never a good idea, but techbros (and you can sometimes read them on HN) think medicine is simple and that we could have AI triage in front of human healthcare professionals.