4 ms·

Literally nothing, these models are not good for scientific literature We saw similar stuff with the model that Meta released and quickly took down

KMnO4 4y ago
Why are you so cynical? You’ve posted nothing but negative comments in this thread.
- ceejayoz 4y ago
  There's every reason to be cynical about AI models of this nature being used for healthcare purposes at this point in time.
- scotty79 4y ago
  I don't really like the word cynical. It's usually used against people who are just accurate only not optimistic enough for somebody's taste.
- wittycardio 4y ago
  [dead]
- ilostmyshoes 4y ago
  Because I'm a scientist and I've used several of these models. I see how they perform and it isn't even close to being useful, in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out
  - lostoldsession 4y ago
    These models are infant technology. They are the chess playing computers of 1960. It's completely missing the forest for the trees to call it out as useless. It's not meant to be useful right now, it's exploratory tech meant to feel out the space. Wait till it gets developed to hold it to some standard. I should note though that progress has been far faster than computer chess. We went from "An AI wont beat a Go champion for at least 30 years" to "AI beats Go champion" in two years.
    - ilostmyshoes 4y ago
      How long after the chess computers of the 60s did it take to beat a human?
      - lostoldsession 4y ago
        Certainly you're not implying that technological progress is linear?
        ilostmyshoes 4y ago
        Not linear but comparable Language and understanding abstract ideas is a bit more complicated than chess
  - eatsyourtacos 4y ago
    >in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out I get what you mean, but that's also kind of silly. First, almost no one who is not in the field is going to be even asking it the kind of questions you are. Second, even if they do and try to use that as reasoning etc.. well, then that moron would basically find any type of source (i.e., random youtube video) to prove their "point". So I honestly think the "harmful" argument is completely out the window. We already opened that pandoras box years ago with basically every piece of human knowledge online in so many places and people making videos on almost everything with all sorts of varying views. People say this about the coding aspect as well "oh this is going to hurt the world because people will be just asking for code and it will be wrong!" Ok.. but it pretty much sounds like they were just going to do that anyway. Let's focus on the positives of this infant technology that is still insanely impressive. We are just seeing the beginning, right?
    - q845712 4y ago
      I dunno man, I'm with the poster above (do we call that the gp?): We have lots of evidence that people in general don't have great truthiness filters for things they read online, and we have a tool that can very confidently say things that seem truthy but are not. Even before the internet, just in the age of radio and early TV, Churchill said "A lie gets halfway around the world before the truth has a chance to get its pants on." Now we have a tool that isn't lying through malice, volition, or any sort of intent, and that will combine jargon and phrases that seem authoritative into its misinformation. IMO responsible engineering involves considering the ups and downs of each solution - that's where the famous HA saying "if you have two you have one; if you have one you have none" comes from, dictating that we need at least 3 instances of a server for high-availability, right? What would it take to engineer this new tech for high accuracy? Where is anything other than the hubris of "we were so busy asking if we _could_, we never stopped to ask if we _should_" ?
samuell 4y ago
Check out https://elicit.org https://elicit.org which is optimized for scientific papers (using the Semantic Scholar API somehow). Perhaps it is possible to get it to make some stuff up, but it has been pretty solid in my experience so far, at least to the point where it has been seriously useful.
- ilostmyshoes 4y ago
  While it does give some accurate summaries I noticed that it uses terms incorrectly. I can see this being a decent tool for finding papers but definitely not for creating accurate summaries, especially for use without in depth review. It's basically a 50/50 shot

KMnO4 4y ago

Why are you so cynical? You’ve posted nothing but negative comments in this thread.

ceejayoz 4y ago
There's every reason to be cynical about AI models of this nature being used for healthcare purposes at this point in time.
scotty79 4y ago
I don't really like the word cynical. It's usually used against people who are just accurate only not optimistic enough for somebody's taste.
wittycardio 4y ago
[dead]
ilostmyshoes 4y ago
Because I'm a scientist and I've used several of these models. I see how they perform and it isn't even close to being useful, in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out
- lostoldsession 4y ago
  These models are infant technology. They are the chess playing computers of 1960. It's completely missing the forest for the trees to call it out as useless. It's not meant to be useful right now, it's exploratory tech meant to feel out the space. Wait till it gets developed to hold it to some standard. I should note though that progress has been far faster than computer chess. We went from "An AI wont beat a Go champion for at least 30 years" to "AI beats Go champion" in two years.
  - ilostmyshoes 4y ago
    How long after the chess computers of the 60s did it take to beat a human?
    - lostoldsession 4y ago
      Certainly you're not implying that technological progress is linear?
      - ilostmyshoes 4y ago
        Not linear but comparable Language and understanding abstract ideas is a bit more complicated than chess
- eatsyourtacos 4y ago
  >in fact in some cases it could be harmful if someone who doesn't know the material already believes the nonsense it spits out I get what you mean, but that's also kind of silly. First, almost no one who is not in the field is going to be even asking it the kind of questions you are. Second, even if they do and try to use that as reasoning etc.. well, then that moron would basically find any type of source (i.e., random youtube video) to prove their "point". So I honestly think the "harmful" argument is completely out the window. We already opened that pandoras box years ago with basically every piece of human knowledge online in so many places and people making videos on almost everything with all sorts of varying views. People say this about the coding aspect as well "oh this is going to hurt the world because people will be just asking for code and it will be wrong!" Ok.. but it pretty much sounds like they were just going to do that anyway. Let's focus on the positives of this infant technology that is still insanely impressive. We are just seeing the beginning, right?
  - q845712 4y ago
    I dunno man, I'm with the poster above (do we call that the gp?): We have lots of evidence that people in general don't have great truthiness filters for things they read online, and we have a tool that can very confidently say things that seem truthy but are not. Even before the internet, just in the age of radio and early TV, Churchill said "A lie gets halfway around the world before the truth has a chance to get its pants on." Now we have a tool that isn't lying through malice, volition, or any sort of intent, and that will combine jargon and phrases that seem authoritative into its misinformation. IMO responsible engineering involves considering the ups and downs of each solution - that's where the famous HA saying "if you have two you have one; if you have one you have none" comes from, dictating that we need at least 3 instances of a server for high-availability, right? What would it take to engineer this new tech for high accuracy? Where is anything other than the hubris of "we were so busy asking if we _could_, we never stopped to ask if we _should_" ?

samuell 4y ago

Check out https://elicit.org https://elicit.org which is optimized for scientific papers (using the Semantic Scholar API somehow). Perhaps it is possible to get it to make some stuff up, but it has been pretty solid in my experience so far, at least to the point where it has been seriously useful.

ilostmyshoes 4y ago
While it does give some accurate summaries I noticed that it uses terms incorrectly. I can see this being a decent tool for finding papers but definitely not for creating accurate summaries, especially for use without in depth review. It's basically a 50/50 shot