4 ms·
If I ask GPT3 to name a few good books on some expert topic, it makes up book titles. What prevents it from doing the same when asked about research papers?
by amelius 4y ago
If I ask GPT3 to name a few good books on some expert topic, it makes up book titles.
What prevents it from doing the same when asked about research papers?
- ilostmyshoes 4y agoLiterally nothing, these models are not good for scientific literature We saw similar stuff with the model that Meta released and quickly took down
- KMnO4 4y agoWhy are you so cynical? You’ve posted nothing but negative comments in this thread.
- ceejayoz 4y agoThere's every reason to be cynical about AI models of this nature being used for healthcare purposes at this point in time.
- scotty79 4y agoI don't really like the word cynical. It's usually used against people who are just accurate only not optimistic enough for somebody's taste.
- wittycardio 4y ago[dead]
- ilostmyshoes 4y agoBecause I'm a scientist and I've used several of these models. I see how they perform and it isn't even close to being useful, in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out
- lostoldsession 4y agoThese models are infant technology. They are the chess playing computers of 1960. It's completely missing the forest for the trees to call it out as useless. It's not meant to be useful right now, it's exploratory tech meant to feel out the space. Wait till it gets developed to hold it to some standard. I should note though that progress has been far faster than computer chess. We went from "An AI wont beat a Go champion for at least 30 years" to "AI beats Go champion" in two years.
- ilostmyshoes 4y agoHow long after the chess computers of the 60s did it take to beat a human?
- lostoldsession 4y agoCertainly you're not implying that technological progress is linear?
- ilostmyshoes 4y agoNot linear but comparable Language and understanding abstract ideas is a bit more complicated than chess
- eatsyourtacos 4y ago>in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out I get what you mean, but that's also kind of silly. First, almost no one who is not in the field is going to be even asking it the kind of questions you are. Second, even if they do and try to use that as reasoning etc.. well, then that moron would basically find any type of source (i.e., random youtube video) to prove their "point". So I honestly think the "harmful" argument is completely out the window. We already opened that pandoras box years ago with basically every piece of human knowledge online in so many places and people making videos on almost everything with all sorts of varying views. People say this about the coding aspect as well "oh this is going to hurt the world because people will be just asking for code and it will be wrong!" Ok.. but it pretty much sounds like they were just going to do that anyway. Let's focus on the positives of this infant technology that is still insanely impressive. We are just seeing the beginning, right?
- samuell 4y agoCheck out https://elicit.org https://elicit.org which is optimized for scientific papers (using the Semantic Scholar API somehow). Perhaps it is possible to get it to make some stuff up, but it has been pretty solid in my experience so far, at least to the point where it has been seriously useful.
- ilostmyshoes 4y agoWhile it does give some accurate summaries I noticed that it uses terms incorrectly. I can see this being a decent tool for finding papers but definitely not for creating accurate summaries, especially for use without in depth review. It's basically a 50/50 shot