Automatic document summarization is very hard without knowing what purpose the summary serves. For what/whom is the package intended? Evaluating the quality of

Automatic document summarization is very hard without knowing what purpose the summary serves. For what/whom is the package intended? Evaluating the quality of summaries is also very difficult. How would you evaluate whether a bioRxiv paper is well summarized or not? In the scientific literature you will find that one way is to let human domain experts score various dimensions of the produced summaries e.g. using a 4-point Likert scale. Alternatively, professional summarizers produce "reference summaries" that serve as a model, and automatic summaries can be evaluated automatically against these using string overlap metrics.