3 ms·
A couple thoughts: 1. Summarizing entire bodies of text down into "bite-sized" chunks isn't inherently a good thing. It seems the main use case (and at least t
by txcwpalpha 8y ago
A couple thoughts:
1. Summarizing entire bodies of text down into "bite-sized" chunks isn't inherently a good thing. It seems the main use case (and at least the one suggested in the demo) is to be used for news articles. Now, I'm totally understanding of the fact that not everyone has the time to read every news article, but as it is, only reading part of the article (or more commonly, only reading the headline) is a huge issue with current consumption of content. This attempt to further summarize articles into small, context-less bites seems to be going in the wrong direction.
2. On the demo page, there is a "Fake News Detection" feature. I threw a couple of articles at it and it left me with so many questions I don't even know where to begin. For a few articles, it just gave me a binary "Real:1 , Fake:0" output. For others, it spit out a couple of numbers for stats like "conspiracy", "irony", "bias", "pseudoscience". Why are these the attributes chosen to measure? How are they calculated? Is something like "irony" even meaningful when trying to detect fake news?
Viewing the documentation section of the site, there is a small blurb claiming that it uses "custom AI classifiers", "custom machine learning models trained on fake and biased articles", and "database of trusted and biased websites created by our experts" to calculate these numbers. AKA, there is absolutely zero meaningful explanation as to how these numbers are calculated and why they should be trusted. This entire feature is a complete black box, and for all we know, the "database of trusted websites" could be created by Russian spies trying to sow misinformation.
- lfx 8y agoSame thoughts: 1) I did try on https://www.reuters.com/article/us-southkorea-prisonstay-idUSKCN1NS0JB https://www.reuters.com/article/us-southkorea-prisonstay-idU... and from 20% and 40% summary I have no clue of the actual meaning of the article. It seems that it just gets some sentences out, but they don't really combine in summary as a whole. 2) And seems like it has some fake news element as well "fake: 0.343". Confusing. To sum up my experience: I confused, real: 0.9; fake: 0.1. Anyhow it seems that has some great potential and may be useful in some general knowledge fact summarization in the future.
- TeMPOraL 8y agoHere's a summarization I'd like (and would pay for[0]): chats. I mean IRC logs, Slack & Telegram groupchat logs, etc. Between work and local Hackerspace, people produce so much text on IM that I can't really keep up with it. I'd love a solution that I could feed such chat logs, and get back e.g. list of topics covered. -- [0] - as long as it wasn't a cloud SaaS where I have to share my data with vendor's machines.
- charris0 8y agoTotally agree, I tried to prototype this but the datasets to train ML summarizations from are pretty much all from news articles. Trying to take that model and summaries chats resulted in gibberish for me. The salient take-aways from Chats and other non-factual / loosely structured text seems so dependent on what you care about, summarisation is difficult.