3 ms·

That's what abstract are for

eghad 6y ago
Abstracts are important (and clearly key in generating these TLDRs), but when it comes to ranking and recommending other papers (not to mention noting whether a new paper has content that can actually push a field forward) an abstract just isn't enough.
justarandomq 6y ago
After some thought I agree with you that this is the wrong problem to solve. I took a narrative detour I wanted to share: Suppose we make the analogue of a scientific paper to a piece of mineral ore (in terms of their raw content, and without written symbols in them for the sake of the analogy) extracted from some mine or quarry. This ore is somehow useful to someone, even if its value is structural: the shingles on an academic roof or a heavyweight desk. What a summarizer attempts to do is use a generic refinement process that will grind up the ore and then separate the components of interest such as Iron, Uranium, or Gold. Anyone thinking that all of metallurgy reduces to simply throwing the slab into a machine and have it spew out the precious metals will find, instead, more complexity than they bargained for, and have more questions on machines or methods to resolve. Gold, Iron, Uranium, all have different extraction process. I believe this approach may give some insight in what problems to solve instead with AI: focus on those discoveries that have helped advance "metallurgy", those of discovering and understanding the structure of the mineral ore and contents (scientific papers) and their relation with current technologies at the time, not on the philosopher's stone of 'summarizing' process more akin to a hammer that makes everything seem like a nail.
- kreeben 6y ago
  >> this is the wrong problem to solve Highly intelligent human beings have a natural ability to summarize big ideas into TLDRs. Are humans basically a bunch of "summarizers"? Probably not. Is this ability to summarize or compress big ideas into smaller, more condensed pieces of information, important to the human race? Yes, I would say that they are. So to me, this is certainly one of those problems that we correctly attempt to solve.
R0b0t1 6y ago
Some abstract too big, author word smaller good
mschuetz 6y ago
Abstracts nowadays are way too large, bordering on a full blown introduction. I agree that's what they should be for, but they are not, in practice.
austinjp 6y ago
Everybody * scans papers for the piece of info they happen to be searching for, before reading it in any detail. The abstract should contain that, but might not. And nobody * reads the entire abstract anyway. A clinician scanning a medical paper is looking for patient relevance: should they use the approach described? The statistical details are too intimidating, the preamble is irrelevant, they know the scope of the problem already. This is not what "should" happen, but it is what actually happens. The gap between published findings and clinical practice is several years. The peer review and publication process are way out of touch with clinical reality. On top of this, people find articles using Google and read them on their phones. (In reality, they read summarised opinion pieces found via Google.) A systematic reviewer may read papers in full. But even they scan papers for inclusion/exclusion criteria first. The deeper the information is buried, the greater the risk of misclassification. I'm not suggesting that TLDRs will fix this, it's just another data point in why we're seeing TLDRs being created. * "Everybody" and "nobody" here excludes researchers :)
KineticLensman 6y ago
In some situations abstracts serve as bibliographic metadata rather than a summary of the content. Examples includes cases where the content is hidden behind a paywall or, in defence, when a paper's content is classified in some way but the existence of the paper itself is not. In both cases, the abstract may help you decide whether it is worth accessing the full paper, but on its own won't give you an answer. E.g "we studied X" but not "and concluded Y". Obviously abstracts can include a content summary as well as bibliographic metadata, but not all do.
capableweb 6y ago
Speaking of abstracts, did you read the abstract from the paper you're commenting about? I don't think you did, because it outlines how this approach is different, and maybe if there was something better than the abstract, you would have read it and not assume it's the same as what we're using abstracts for today. In short, here's the major differences: > SciTLDR contains both author-written and expert-derived TLDRs > CATTS improves upon strong baselines under both automated metrics and human evaluations

Abstracts are important (and clearly key in generating these TLDRs), but when it comes to ranking and recommending other papers (not to mention noting whether a new paper has content that can actually push a field forward) an abstract just isn't enough.

justarandomq 6y ago

After some thought I agree with you that this is the wrong problem to solve. I took a narrative detour I wanted to share: Suppose we make the analogue of a scientific paper to a piece of mineral ore (in terms of their raw content, and without written symbols in them for the sake of the analogy) extracted from some mine or quarry. This ore is somehow useful to someone, even if its value is structural: the shingles on an academic roof or a heavyweight desk. What a summarizer attempts to do is use a generic refinement process that will grind up the ore and then separate the components of interest such as Iron, Uranium, or Gold. Anyone thinking that all of metallurgy reduces to simply throwing the slab into a machine and have it spew out the precious metals will find, instead, more complexity than they bargained for, and have more questions on machines or methods to resolve. Gold, Iron, Uranium, all have different extraction process. I believe this approach may give some insight in what problems to solve instead with AI: focus on those discoveries that have helped advance "metallurgy", those of discovering and understanding the structure of the mineral ore and contents (scientific papers) and their relation with current technologies at the time, not on the philosopher's stone of 'summarizing' process more akin to a hammer that makes everything seem like a nail.

kreeben 6y ago
>> this is the wrong problem to solve Highly intelligent human beings have a natural ability to summarize big ideas into TLDRs. Are humans basically a bunch of "summarizers"? Probably not. Is this ability to summarize or compress big ideas into smaller, more condensed pieces of information, important to the human race? Yes, I would say that they are. So to me, this is certainly one of those problems that we correctly attempt to solve.

R0b0t1 6y ago

Some abstract too big, author word smaller good

mschuetz 6y ago

Abstracts nowadays are way too large, bordering on a full blown introduction. I agree that's what they should be for, but they are not, in practice.

austinjp 6y ago

Everybody * scans papers for the piece of info they happen to be searching for, before reading it in any detail. The abstract should contain that, but might not. And nobody * reads the entire abstract anyway. A clinician scanning a medical paper is looking for patient relevance: should they use the approach described? The statistical details are too intimidating, the preamble is irrelevant, they know the scope of the problem already. This is not what "should" happen, but it is what actually happens. The gap between published findings and clinical practice is several years. The peer review and publication process are way out of touch with clinical reality. On top of this, people find articles using Google and read them on their phones. (In reality, they read summarised opinion pieces found via Google.) A systematic reviewer may read papers in full. But even they scan papers for inclusion/exclusion criteria first. The deeper the information is buried, the greater the risk of misclassification. I'm not suggesting that TLDRs will fix this, it's just another data point in why we're seeing TLDRs being created. * "Everybody" and "nobody" here excludes researchers :)

KineticLensman 6y ago

In some situations abstracts serve as bibliographic metadata rather than a summary of the content. Examples includes cases where the content is hidden behind a paywall or, in defence, when a paper's content is classified in some way but the existence of the paper itself is not. In both cases, the abstract may help you decide whether it is worth accessing the full paper, but on its own won't give you an answer. E.g "we studied X" but not "and concluded Y". Obviously abstracts can include a content summary as well as bibliographic metadata, but not all do.

capableweb 6y ago

Speaking of abstracts, did you read the abstract from the paper you're commenting about? I don't think you did, because it outlines how this approach is different, and maybe if there was something better than the abstract, you would have read it and not assume it's the same as what we're using abstracts for today. In short, here's the major differences: > SciTLDR contains both author-written and expert-derived TLDRs > CATTS improves upon strong baselines under both automated metrics and human evaluations