3 ms·

Here's what I (a person with a fairly superficial understanding of how AI works, for context) can't understand - why wouldn't it have been trivial to prevent th

Here's what I (a person with a fairly superficial understanding of how AI works, for context) can't understand - why wouldn't it have been trivial to prevent the kinds of issues that arose by just training the thing to only pull information from scientific papers/literature, and when a question arose on a topic that didn't have information in those places just say "we can't find enough information about that topic"? It seems like it was good at structuring the writing, both at the article and sentence levels, and for valid prompts it produced accurate responses. But if you entered "write a wiki article about the Alien vs. Predator hypothesis," it would structure it like a wiki article about a scientific topic but just put random AVP stuff it found on the internet into that structure. Why couldn't they just explicitly define which sources are appropriate to pull information from? That seems easier to me than building the actual product they made (but again, I am a layman here).

glitchc 4y ago
Because training in this manner requires good data. And by good I mean manually curated. The ML hype train relies on "garbage in, good out", popularized by an influential paper published at the turn of the century [1]. Anyone with a modicum of experience in experimental science knows that any experiment is only as successful as its ability to collect good data that is representative of the problem, has manageable noise based on known controlled sources and has sufficient coverage of the problem domain to provide meaningful analysis. But of course, if ML admitted that this was a necessary requirement, it would become yet another optimization technique, admittedly new classes of optimization techniques and that promise that the machine learns something new would fall flat on its face. [1] https://aclanthology.org/H01-1052.pdf https://aclanthology.org/H01-1052.pdf
- awillen 4y ago
  Sure, I get that part, but in this case it seems like curation (at least to a degree that would avoid the problems that arose) is fairly trivial - just train it on high-quality scientific journals/textbooks/etc., instead of letting it peruse the whole internet (or whatever they did) for scientific data.
deltathreetwo 4y ago

glitchc 4y ago

Because training in this manner requires good data. And by good I mean manually curated. The ML hype train relies on "garbage in, good out", popularized by an influential paper published at the turn of the century [1]. Anyone with a modicum of experience in experimental science knows that any experiment is only as successful as its ability to collect good data that is representative of the problem, has manageable noise based on known controlled sources and has sufficient coverage of the problem domain to provide meaningful analysis. But of course, if ML admitted that this was a necessary requirement, it would become yet another optimization technique, admittedly new classes of optimization techniques and that promise that the machine learns something new would fall flat on its face. [1] https://aclanthology.org/H01-1052.pdf https://aclanthology.org/H01-1052.pdf

awillen 4y ago
Sure, I get that part, but in this case it seems like curation (at least to a degree that would avoid the problems that arose) is fairly trivial - just train it on high-quality scientific journals/textbooks/etc., instead of letting it peruse the whole internet (or whatever they did) for scientific data.

deltathreetwo 4y ago