20 ms·

Credder Wants to Create an Equivalent to “Rotten Tomatoes” for News

262 points by imartin2k 8y ago

zeofig 8y ago
But who freshens the freshmen?
bko 8y ago
> In order to be vetted as an individual critic, you should meet the following guidelines: Written > • Consistent output for a minimum of two years. > • Demonstrated film/TV coverage at a publication outside of a self-published website. If you are a self-publishing critic, your reviews should reflect our key values. Having journalist rate other journalists won't serve as a good filter on the actual quality of reporting. They may pick up on technical errors, but it doesn't address the large problem with modern reporting, which is inaccuracies or misstatements on technical topics. I'd rather have a review system of SME specific to the topic discussed. For instance, an article written on some AI breakthrough would be critiqued by researchers in that field. I understand that would be more difficult as it would greatly increase the number of reviewers required, but on the flip side a single reviewer would likely be more representative of her industry. I frankly don't care what other journalists think about the work of their peers since they all have the same biases
- spammersharass 8y ago
  *his or her
- AbrahamParangi 8y ago
  The interesting thing about rotten tomatoes is that it generates a very different signal than critical review on average. Your tastes (or worldview) may not match that of movie critics (or journalists) but you might find a surprising amount in common with the median of the set.
- muzani 8y ago
  With Metacritic, it's been clear that the professionals do a much worse job than the regular users. Even the Oscars have poor correlation with movie quality, in my opinion. A lot of it ends up too pretentious. What if Daily Mail suddenly has a great article like they occasionally do? Many professional critics still wouldn't give it a good review because their reputation and careers are at stake.
  - ekianjo 8y ago
    Metacritic exposes the corruption of the video games press as well as the utter lack of qualification (as in, no proper authority compared to any other gamer out there) to write about anything.
- richmarr 8y ago
  > ... it doesn't address the large problem with modern reporting, which is inaccuracies or misstatements on technical topics. If I was going to pick a journalism problem to solve, the accuracy of technical details would not be it. If I could solve one or both of (a) funding and (b) deceptive balance, then I would quite happily sell 'technical accuracy' down the river.
  - amanaplanacanal 8y ago
    Yes, funding is the biggest one. "If it bleeds it leads" is a terrible way to give your readers or viewers a real picture of what is going on in the world.
keiferski 8y ago
Taking a look at the Rotten Tomatoes “Top 100 Films of All Time” list [1], I’m not exactly sure why it’s a model to be replicated. It’s extremely obvious that “top” means “popular and/or most advertised.” [1] https://www.rottentomatoes.com/top/bestofrt/ https://www.rottentomatoes.com/top/bestofrt/
- guelo 8y ago
  What a weird list with few selections between 1970 and 2015.
  - anextomp 8y ago
    Their "adjusted score" means that films with more ratings are rated higher. Older films have far fewer ratings by critics, same with smaller films.
  - deleted 8y ago
    [deleted]
  - misnome 8y ago
    Yes, Alien being missing is an odd sign, especially as it has 99%. (IIRC it used to be the only 100% film, until some critic posted a new review saying it hadn’t aged well)
    - icebraining 8y ago
      Alien is there (position 56).
      - misnome 8y ago
        Dang it I both scanned and searched the page, and safari browser said no text results.
- rjtavares 8y ago
  RT's model prioritizes consensus in quality instead of brilliance. Using RT for a best of list will give you good safe movies instead of outstanding ones. I'd say it's a good model for news.
krapp 8y ago
What will happen here is what happens with all sites which attempt to curate news or check facts: people who disagree with the site's conclusions will write it off as propaganda. Using "professional" journalists or "domain experts" at all will be a red flag for people who believe mainstream media and science only serves political and corporate interests.
- jerf 8y ago
  I have seen many efforts at "grading" news. I have not yet seen one that isn't just a news site that grades whether or not the news is in alignment with some ideology, with the occasional fig leaf tossed out for some particularly egregious error to try to appear even-handed. The closest I've seen are some news grading sites that are fair for a couple of months, but because a pure ideology site. As near as I've been able to tell, discarding all such sites (not just the ones that disagree with your ideology, but "all sites" is a superset of that) is the rational move.
Kaveren 8y ago
I place no value in review aggregators. They don't tell you anything useful. It's much better to find your own individual sources that you can trust over time, and they can earn or lose reputation with you based on the quality of their reporting. This isn't limited to news. I definitely don't trust a consensus of journalists to tell me a study is misinterpreted.
cat199 8y ago
give the user control of the reputation ranking, and you might have something here..
- netsharc 8y ago
  But it will just leave the system vulnerable to brigading. That even happened on RT for the Captain Marvel movie because of the female heroine, many "snowflakes" got offended and gave a movie that wasn't even out yet bad ratings. Kinda funny how these snowflakes (let me assume white males) call out "SJW"s while meanwhile they themselves actually believe society is mistreating them and so in their rage they're also expressing their own version of wanting "social justice".
  - ataturk 8y ago
    I'm not sure the keyboard commandos are snowflakes or SJWs. Maybe guys with a bit too much sensitivity and time on their hands, though. But those guys do have a point--they consider themselves loyal fans who actually buy the merch and they're upset because suddenly Hollywood has no use for them. Hollywood is all up its own ass with Mary Sue "strong female lead" characters for marketing and signaling reasons. That people are calling them on their bullshit is generally a good thing. What Hollywood is really attempting to do is market to China, I think. In their messed up minds it is more important to have asses in seats and yet they don't even understand the demographic they are working for.
  - motogpjimbo 8y ago
    > That even happened on RT for the Captain Marvel movie because of the female heroine, many "snowflakes" got offended and gave a movie that wasn't even out yet bad ratings. Alternatively, they were using the "Want to See" feature on Rotten Tomatoes exactly as it was intended to be used. People didn't like aspects of the marketing campaign for Captain Marvel and made their opinions known using a tool Rotten Tomatoes provided specifically for that purpose. There are many female-led films listed on Rotten Tomatoes that have very high approval fan ratings. By dismissing the public reaction to Captain Marvel merely as trolling by snowflakes, you're actually proving the point many people on this thread are making about partisanship and tribalism in news reporting.
ai_ia 8y ago
I have had this idea for so long. A Crowd-News Watch Dog which basically works like a wikipedia + archive.org + IMDB for News Reporting. There you can have profile for journalists or any content writers. You can basically see what a journalist has wrote about in her/his entire life. Political inclinations deducted from writings by crowd. The article written can be critiqued in the platform using a set of pre-defined set of rules. If the reporter is found employing strawman or fallacies or incorrectly reporting based on botched data/ statistical mumbo-jumbo, the commentary can be added by anyone and verified by other community members. I think it can keep rampant fake news and/or politically-charged posts masquerading as neutral pieces in check.
- nukeop 8y ago
  This would very quickly devolve into either a hardcore bullying platform, or a banfest of everyone who criticizes the favorite people of the site's owner, his friends, and moderators.
  - ai_ia 8y ago
    If an argument put forward by the article doesn't abide by journalistic principles and is bad and crtiques offer why it is bad with coherent counter-argument, I believe there can be a middle ground somewhere.
lucideer 8y ago
This seems like a broader, more ambitious version of Snopes. If successful, I wonder would it replace it? Or could this be something Snopes could expand its existing community to handle?
- ceejayoz 8y ago
  Snopes is a great model for why this will fail, though. They'll be dismissed as "fake news" by one side or another on anything controversial. Hell, I've seen folks met with offended "did I ask for criticism?!" responses when they drop a "no, Bill Gates isn't going to give you a million dollars for forwarding that chain email" Snopes article in someone's Facebook comments.
deleted 8y ago
[deleted]
specialist 8y ago
"Fake news" is such an easy problem to solve: Just sign all the work. So you can verify who said what. Done. Don't see a signature, and it matters to you, just hit "next". -- Efforts to establish truth, factualness, credibility, whatever cannot succeed. Because humans don't work that way. The people who watch Alex Jones (for instance) don't care about your truth. So stop trying.
josteink 8y ago
Someone’s always negative in these threads, so this time it might as well be me. First of all: that headline really has an apples and oranges feel to it. Basically Rotten tomatoes worked because it merely curated and aggregated something which already existed in abundance: movie reviews, deriving a consensus based on that. Doing the same for news, will depend on something which has yet to exist at any meaningful scale: news reviews (and what would that even be?). So not only must it prove its value (and neutrality) as an aggregator, but it will also have to encourage the creation of something entirely new (news reviews) at scale, because otherwise it won’t have anything to curate. I’m not holding my breath.
saltvedt 8y ago
I'm solving a very related problem with https://cited.news/ https://cited.news/ It's a news aggregator based on Wikipedia references.
unityByFreedom 8y ago
Like Musk's pravda?
joeblau 8y ago
I wrote something that was trying to do this a back in January that was called desational[1]. What I did was very basic but the process I did went as follows. 1. Break up news articles into sentences 2. Pay Amazon MTurkers to tag sentences as `news` or `not-news` 3. Train a text classifier based on the results for Mturk 4. Process new articles using the trained model to identify and strip out parts pf the article that are sensational. 5. Give the article a percent score and a letter grade about how much "news" is actually in the article based on the percentage of sentences that were identified as news. I ran it for a few weeks on tech news, but then I figured that the news industry is not a business I really want to be in. The source code is all just sitting up on GitLab. [1] - https://desational.gitlab.io/desational.com/ https://desational.gitlab.io/desational.com/
- DamnInteresting 8y ago
  Very interesting idea. However, having dabbled with Bayesian classifiers myself, I suspect that your decision to break the article into sentences may be a fatal flaw...both the Mturkers and the classifier lose out on valuable context that way. I know because I took a a similar approach in one of my own projects, and the classifier could never gain enough 'confidence' to classify things, even after thousands of trainings. The approach might work pretty well if you broke it down to paragraphs, where some context is preserved, but the more context you provide, the better the classifier will perform, often for surprising reasons. For example, I built one classifier that was performing poorly until I re-trained it without stripping HTML and headers; after that it performed splendidly. Upon examination I found that tokens such as domains and IP addresses in headers and links were hugely influential (and rightly so) in the classification. Just something to consider. :) Apologies if I incorrectly assumed that your classifier is Bayesian.
  - bduerst 8y ago
    Good point, but keep in mind that many news articles also break paragraphs out into single sentences. For example: https://www.reuters.com/article/us-france-china-airbus/airbus-close-to-signing-aircraft-deal-with-china-sources-idUSKCN1R61Y0 https://www.reuters.com/article/us-france-china-airbus/airbu...
ataturk 8y ago
Didn't RT get caught ginning the numbers for that awful Ghostbusters remake? Aren't there also some other notable WTF examples? Anything electronic, especially where big money is involved, is suspect.
LifeLiverTransp 8y ago
I cant wait for the first social civil war side trying to apply pressure to that approach.
rchaud 8y ago
I get the methodology but, what is the demand for this type of product? RT makes sense because there are too many movies coming out at any one time, and you need something to build a shortlist from. But news? Are users so spoiled for choice that they need a journalist's opinion on what's worth reading? This doesn't even seem like an anti fake-news play, since the people that consume and share that stuff aren't actually interested in news, just whatever can be used as a stick to beat the political opposition with.
- beaconstudios 8y ago
  presumably it's marketed as anti fake news but the reality is it will be politically biased towards one faction. That's usually how these things play out.
  - amanaplanacanal 8y ago
    The thing is, even if it's not biased, as soon as something is published which is inconvenient for one side or the other all the partisans are going to claim it's biased. As they already do.
    - beaconstudios 8y ago
      That's where providing the evidence comes in - you'll never convince your opponents but for skeptics and critical thinkers they can determine for themselves if the evidence supports the perspective.
duwease 8y ago
I want to bring a note of positivity here -- while most comments seem to be poking holes in the concept, all also seem to acknowledge the need for such a service to be successful. This is a domain that I've also put a lot of thought into over the years, so I hope something succeeds. And some things do. Keep in mind that Wikipedia caught a lot of flak for being unreliable or impossible at first, but nowadays they have a robust editorial process for contentious subjects that I'd consider the gold standard for any such practical applications on the internet.
- gambler 8y ago
  >while most comments seem to be poking holes in the concept, all also seem to acknowledge the need for such a service to be successful. Speaking plainly: anyone who believes you can get better news through some kind of external meta-review process is a fool. Probably a fool who also thinks the review process should validate journalists that share their political biases, because that's what it's all about these days. You cannot test quality into the product. News sources are supposed to cross-validate each other in the first place. Clearly, that have stopped working some time ago. The only thing that can save news now is an overhaul of its financing and discovery models. >Wikipedia caught a lot of flak for being unreliable or impossible at first, but nowadays they have a robust editorial process for contentious subjects Hah.
josefresco 8y ago
This, and many other concepts based on "vetting" news assumes that the audience cares. A significant portion of the audience does not care, and is instead looking for news that validates their worldview, and furthers their (and their chosen political party's) goals. So basically these services will cannibalize the part of the audience that does care about truth, turning people against each other over marginally skewed articles. Meanwhile, the other site is happy to share falsehoods, because it helps them ... win.
- deleted 8y ago
  [deleted]
- sureaboutthis 8y ago
  To a point, you are correct, but most people don't "not care". Most people are unaware and only assume a knowledgeable, experienced editor vets and edits the news. Today, most news has a marketing angle applied to it to attract more viewers and visitors.
  - swebs 8y ago
    I don't think "most" people even assume that. Only 1/3 of people under 30 even trust the mass media. https://news.gallup.com/poll/243665/media-trust-continues-recover-2016-low.aspx https://news.gallup.com/poll/243665/media-trust-continues-re...
  - josefresco 8y ago
    The number of people who shared and continue to share "fake news" is a sign (to me) that most don't care about truth and are in fact complicit. These people are selfish, narcissistic, naive maybe all of the above. > Most people are unaware I just don't buy this based on the content that I see shared. When corruption (for example) is exposed it should be a slam dunk for both sides of the political debate. However, this is no longer happening, with each side digging in deeper.
- TeMPOraL 8y ago
  It's even worse. For most people, news is just a social object. Something to talk about with family, co-workers, etc. Whether or not any particular story is true doesn't matter - what matters is that everyone in their circle is aware of the described event (whether they read the same story, or a different one). Humans are pretty good at fact-checking and critical thinking when getting it wrong means some immediate loss. But most news stories are irrelevant to almost everyone's lives, and those few that are, don't have a direct and immediate impact.
vinw 8y ago
If you like this, then you might also like spidr[0] "SPIDR is an AI-powered news aggregator that uses text analysis to group similar news together. The aim is to provide fast access to relevant news from different viewpoints." [0] https://spidr.today https://spidr.today
- DonHopkins 8y ago
  I dunno -- the name "credder" sounds much more authoritative and trustworthy to me than "spidr", because they didn't leave out the penultimate "e".
  - vinw 8y ago
    But in spidr's favour I've been able to use it for quite a while whereas credder is still just an idea.
- e15ctr0n 8y ago
  Thanks for posting this. The news aggregator looks really good. I would love to know how you built it. Which text analysis AI did you use to group the similar news together?
  - vinw 8y ago
    Sorry I can't help with that, it's not my site.
rooam-dev 8y ago
Good luck to them if course, but I am not sure this will work. Couple of reasons come to my mind: 1. How will I know that news have been reviewed by qualified people? Technically this should be news agency/source's job. 2. By the time the score will depict the news' quality, the damage may be already done. 3. RT is based on people's opinions, but news should be fact checked. What I think it should do, is to score the publishers instead... some kind of publishing performance history. Although the most effective would be to have some kind of liability for spreading/posting fake news, this would make publishers to do more fact checking before fast publishing. All above is just my IMHO.
- sureaboutthis 8y ago
  Haven't read the article yet but I would hope, instead, it would rate these based on consistent quality and not per story.
- crististm 8y ago
  > 3. RT is based on people's opinions, but news should be fact checked. Except when RT doesn't like to hear people's opinion when it differs from the mainstream. Unfortunately it's easy to box it as "trolling" and hide it under the rug.
  - crististm 8y ago
    Apparently I'm getting myself in a SJW hive. Shame on me. s/hive/swarm
    - martythemaniak 8y ago
      Did you mistake Rotten Tomatoes for Russia Today?
      - fartcannon 8y ago
        I think he's referring to when they delete reviews that are affected by cultural phenomenon.
darepublic 8y ago
Rather than create an RT for news, just create a better news outlet please. A new form of news would be the more revolutionary startup imo. I think any RT style news review site will end up being regarded much as Politifact is today -- valued only by those who agree with its ratings.
- holler 8y ago
  what would you like to see in this new news outlet?
  - darepublic 8y ago
    well right now in news we get stories with the different points of view, and just the points of view talking past each other. I want news that deals in raw hard to get data, and that monitors the data continually on a day to day basis. ie. for disaster relief in a country, to report neighbourhood by neighbourhood how much is being received and by who and in what quantity. basically it would require a ton of work, and would be much easier to achieve that level of coverage in a single city at first -- but basically try to have omniscience source of knowledge, and where knowledge is being blocked or obfuscated to identify that.
mcv 8y ago
Considering how easily Rotten Tomatoes is gamed, including for political reasons, I'm not sure I want my news to be gamed the same way.
MentallyRetired 8y ago
Yeah, let's let the people decide if the news is accurate. What could go wrong?
who-knows95 8y ago
isn't this what elon musk was talking about? :https://www.cnbc.com/2018/05/25/elon-musk-may-actually-be-making-a-website-to-rate-journalists-for-credibility-and-core-truth.html https://www.cnbc.com/2018/05/25/elon-musk-may-actually-be-ma...
trophycase 8y ago
Until it is bombarded by fake reviews?
mmmchipotlemmm 8y ago
What is the evidence that people want 'trustworthy' news?
esalman 8y ago
Like academic publications, every news article should be published along with acknowledgement of funding, conflict of interest and maybe data availability statement. It is a lot easier to judge a news article when you know who own the publication and such.
CodeSheikh 8y ago
IMHO, journalism is slowly dying. These days you find more "opinion-based" or "make-it-viral" news pieces than proper investigative journalism. Services like this can help but at the same time they can create further segregation of true news vs poor news if most of the critics start growing to be biased.
- wsc981 8y ago
  I agree, I feel many times you can get more solid news from sources like Twitter. Especially how Western media covered Syria annoyed me to no end. To me the news coverage in my country seemed mainly aligned with Western political interests. But I was happy to be able to follow at least 2 journalists [0][1] that were "on the ground" in Syria for some good solid investigative reporting. --- [0]: https://twitter.com/jenanmoussa https://twitter.com/jenanmoussa [1]: https://twitter.com/HaraldDoornbos https://twitter.com/HaraldDoornbos
- amanaplanacanal 8y ago
  Journalism costs money. If it is funded by ads, then getting more eyeballs onto the ads is going to trump good journalism every time.
jmnicholson 8y ago
We're effectively creating the rotten tomatoes for science at scite.ai. We rely upon a deep learning model to classify citation statements as supporting, contradicting, or mentioning.
chungleong 8y ago
Jesus, as though there isn't enough vain people working in the news business already. Society needs fewer trophies.
bg4 8y ago
I'd love to see this in combination with Knowhere News (https://knowherenews.com/ https://knowherenews.com/).
ryanmarsh 8y ago
I thought that was what /r/worldnews was for?
overthemoon 8y ago
What value does Rotten Tomatoes provide? Is review aggregation really the model for truth we want to use? "A lot of people find this reliable/unreliable" tells you exactly that and nothing more. This is an attempt to cram the huge, complex enterprise of discourse into some kind of boolean true/false reliable/unreliable binary it cannot fit neatly into. Disinformation and tribalism are social problems that aren't waved away with technological solutions. At the end of the day, I'm not interested in startups as truth gatekeepers. It's empty when it comes to movies--it's downright disingenuous when it comes to the news. It provides the veneer of objectivity by being a technological solution, but the enterprise's foundation is still ideological. I trust them as little as any other outlet that can't honestly assess and describe their own viewpoint. At best, you're adding another layer that will itself become ideologically polarized. If this or similar platforms succeed, we'll see a proliferation of them by people who feel like it is too <ideology> or not enough <ideology>.
Mindstormy 8y ago
Isn't this kind of already a thing? http://www.emergent.info/ http://www.emergent.info/
ObscureMind 8y ago
How is that having journalists reviewing the work of other journalists won't lead to corporatism?
kosei 8y ago
Sort of reminds me of the website SpinSpotter. Great idea, but had trouble finding a real audience for its product. https://en.m.wikipedia.org/wiki/SpinSpotter https://en.m.wikipedia.org/wiki/SpinSpotter
NoblePublius 8y ago
This seems far too complicated. The UX of Dissenter (boo, hiss, I know, control yourself) is simple by comparison, doesn’t involve another destination site, and doesn’t involve a committee deciding whose reviews are more worthy of consideration than others. What Id prefer is a ranking system of journalist merit: does this story cite anonymous sources? Is this story original reporting? Does this story simply repackage someone else’s original reporting with an editorial headline? Is it news, editorial, or opinion? What corporations advertise with the parent company? These are objective measures of newsworthiness and Credder would track none of them by default.
jellicle 8y ago
And if you have questions about the credibility of Credder, you can just check the Credder ratings on Verrit, which is itself verified by Truther and Accurit, which are backed up by Credder. Solved! News trustworthiness is not a technical problem and cannot be solved by technical solutions. News: something you don't know. Verifying news: asking people to opine about the trustworthiness of something they don't know. > "Here's an opaque cardboard box. In it is a banana. Now rate my news story on its accuracy. NO, you are not allowed to open or inspect the box in any way. Also, a lot of you have strong political opinions about bananas. Ready, GO!" Garbage in, garbage out. Far too gameable, a fundamentally insoluble problem to begin with.
deleted 8y ago
[deleted]
Dirlewanger 8y ago
A RT for news isn't going to fix the broken medium that is online journalism. Basic research, fact checking, editing/proofreading, journalistic integrity -- all things we take for granted with the news have been discarded in favor of more and more clicks/profits. If anything, we need a new model for how news is manufactured/distributed in the age of the Internet.
OscarTheGrinch 8y ago
4. Will be gamed by groups of bad actors.
DonHopkins 8y ago
Dear Credder.com: thank you for using a penultimate "e" before the final "r". Such a breath of fresh air! (Was it because Creddr.com was already taken? ;)
Circuits 8y ago
I like the idea but I don't know if I agree with the proposed approach to reviews. Personally, at a minimum, I would like to see the following from every reviewer: A structured summary- background, objectives, data sources, synthesis methods used, results, limitations, conclusions and the implications of important findings drawn by the article. In more detail: An introduction that presents the problem and the issues dealt with. An assessment of the methods which describes the research and the evaluation process; specifies the number of studies references and gives a brief evaluation of their validity. A description of conclusions/results and their quality based on identifiable and preconceived academic/professional metrics. A strong summary of conclusions, limitations of those conclusions and the outcomes of the procedure used for the research. I would really love to see reviews of reviews as well as a reviewers rating system (individual reviewers should have their own ratings) but given the limited number of reviewers that maybe asking too much.
catchmeifyoucan 8y ago
How do they make money
Causality1 8y ago
Isn't this the same crap Elon Musk was talking about when the media was roasting him for unsafe conditions at Tesla and calling rescue divers pedophiles?
laythea 8y ago
In my opinion, this doesn't really offer any value as the only way to get a good understanding of the news is to read from lots of different places and come to a view. Using this strategy then consuming a truly "random" site is better than consuming pre reviewed site as there is no bias in selection or outcome. It's not like films, where each one is independent (except marvel and dc comics obviously :)
TomMckenny 8y ago
Maybe this project will help the general public start reading reality based media. But we do have this already. Done by experts. It's called the Pulitzer. The people in a democracy must have access to information. Once a public is persuaded to reject reality based media, democracy becomes irrelevant. Thus the campaign to denigrate it in the dozen nascent autocracies. Again, I hope this project helps reverse that.
silveira 8y ago
People who want falsified narratives about reality don't care about these ratings. They will even go beyond and create their own "alternative facts" seal and go with that.
- phil248 8y ago
  True, as we already know from the myriad fact-checking sites.
gabbygab 8y ago
Perhaps they should have used a better example than rotten tomatoes. RT is owned by NBCUniversal and is terribly unreliable as a movie ratings aggregator. At one point it was a very useful site, but now it is useless. Might as well ask disney, nbc or studio execs on which movies to watch. We know that RT has lied and fudged ratings for financial or other purposes. It has the same problem as NewsGuard. It is controlled by industry insiders to benefit themselves and the industry. Would anyone trust an oil company ratings company controlled by oil companies? Rather than a "rotten tomatoes" for news, I'd rather have a "meta" news or wiki site which has a running list of all the fake news that news companies have pushed out. Make it part curated and part user driven. Let users provide, comment and even contribute to it. Make it listable by news companies and journalists. So we can see which news companies and which journalists have pushed the most fake news to the american public.
tomohawk 8y ago
How would this pick up on the problem where stories such as the Mueller investigation are hyped and mischaracterized for a long period of time, and in the end Breitbart ends up being more correct on the story than most other news outlets?
chillingeffect 8y ago
It would be much more valuable to supply chains of evidence in news stories. So often, articles are treated as isolated islands with unresolved references and only loose links to previous events. News could be so much more credible if it had extensive underpinning of every exertion. For example, in Europe, news stories almost always include a map of the local area. But one could do much better with timelines showing how each article is a step along an unfolding story, empowering readers. In theory, readers can go do all the research themselves, but the cost is quite high. That's almost exactly what the WWW was designed for! It could also includes links to wiki-like pages of every individual related to the story. But obviously the big question is market. Especially these days, people don't want evidence, only emotional crack. People for whom the information is actually financial valuable pay for accurate information in their niche. So the real challenge is to motivate people to focus on reality instead of their outrage addiction. That's a cultural goal and more likely to be a non-profit enterprise. :(
gustavmarwin 8y ago
That's a step in the good direction, I think curating the news is going to be a very precious service. I'm looking forward to decentralised ways of achieving this though, such as with https://civil.co/ https://civil.co/
_bxg1 8y ago
This is just going to descend into review-bombing chaos (look at what happens to games on Steam, and then think about how much more biased people are about news sources). A much better idea would be a curated news source registry that does objective fact-checking and scores news sources based on various criteria, with the relevant research/scoring mechanisms shown openly so people can verify the results themselves. Like Snopes meets Charity Navigator (https://www.charitynavigator.org/ https://www.charitynavigator.org/).
sharkjacobs 8y ago
I hope they're paying close attention to Rotten Tomatoes' problems with politically motivated review bombing.
bargl 8y ago
I think I posted this somewhere else but here it goes. I want a system that rates journalists not news. I want to see my previous ratings of this journalist and be able to base it on that. I want the ability to rate a pieces objectivity. I want to see ratings from users who have been proven to know how to rate objectivity. I want a system that uses Machine Learning to break me "out of" my information silos by promoting the opposite opinion, and saying as much. "Because you like, Capitalism here's an argument for Socialism." I want a system that making ratings less of a popularity contest by limiting them in a meaningful way (this is pie in the sky I'll take the first two). I don't think it's possible to create the things I want, but that's what I want. Not Reddit for news. I've seen how that works.
ilaksh 8y ago
People in this thread do not seem to appreciate the real threat to society that censorship and propaganda pose. A widely used news rating site will be used to suppress political dissent and alternate views. Maybe take a look at history and the world. Do you think that Chinese authorities would like a tool installed on everyone's computer that could rate the trustworthiness of a news article or site? Of course they would love it because they could just downgrade any reporting that was critical of the party. Based on the number of young people who seem to be begging for authorities to control their information, it appears that the China model is coming to the United States.
- unityByFreedom 8y ago
  I'm with you.
dancemethis1 8y ago
So... A website clearly intended to be a parody due to its name, but still taken seriously by people that didn't get it?

262 points by imartin2k 8y ago

But who freshens the freshmen?

bko 8y ago

> In order to be vetted as an individual critic, you should meet the following guidelines: Written > • Consistent output for a minimum of two years. > • Demonstrated film/TV coverage at a publication outside of a self-published website. If you are a self-publishing critic, your reviews should reflect our key values. Having journalist rate other journalists won't serve as a good filter on the actual quality of reporting. They may pick up on technical errors, but it doesn't address the large problem with modern reporting, which is inaccuracies or misstatements on technical topics. I'd rather have a review system of SME specific to the topic discussed. For instance, an article written on some AI breakthrough would be critiqued by researchers in that field. I understand that would be more difficult as it would greatly increase the number of reviewers required, but on the flip side a single reviewer would likely be more representative of her industry. I frankly don't care what other journalists think about the work of their peers since they all have the same biases

spammersharass 8y ago
*his or her
AbrahamParangi 8y ago
The interesting thing about rotten tomatoes is that it generates a very different signal than critical review on average. Your tastes (or worldview) may not match that of movie critics (or journalists) but you might find a surprising amount in common with the median of the set.
muzani 8y ago
With Metacritic, it's been clear that the professionals do a much worse job than the regular users. Even the Oscars have poor correlation with movie quality, in my opinion. A lot of it ends up too pretentious. What if Daily Mail suddenly has a great article like they occasionally do? Many professional critics still wouldn't give it a good review because their reputation and careers are at stake.
- ekianjo 8y ago
  Metacritic exposes the corruption of the video games press as well as the utter lack of qualification (as in, no proper authority compared to any other gamer out there) to write about anything.
richmarr 8y ago
> ... it doesn't address the large problem with modern reporting, which is inaccuracies or misstatements on technical topics. If I was going to pick a journalism problem to solve, the accuracy of technical details would not be it. If I could solve one or both of (a) funding and (b) deceptive balance, then I would quite happily sell 'technical accuracy' down the river.
- amanaplanacanal 8y ago
  Yes, funding is the biggest one. "If it bleeds it leads" is a terrible way to give your readers or viewers a real picture of what is going on in the world.

keiferski 8y ago

Taking a look at the Rotten Tomatoes “Top 100 Films of All Time” list [1], I’m not exactly sure why it’s a model to be replicated. It’s extremely obvious that “top” means “popular and/or most advertised.” [1] https://www.rottentomatoes.com/top/bestofrt/ https://www.rottentomatoes.com/top/bestofrt/

guelo 8y ago
What a weird list with few selections between 1970 and 2015.
- anextomp 8y ago
  Their "adjusted score" means that films with more ratings are rated higher. Older films have far fewer ratings by critics, same with smaller films.
- deleted 8y ago
  [deleted]
- misnome 8y ago
  Yes, Alien being missing is an odd sign, especially as it has 99%. (IIRC it used to be the only 100% film, until some critic posted a new review saying it hadn’t aged well)
  - icebraining 8y ago
    Alien is there (position 56).
    - misnome 8y ago
      Dang it I both scanned and searched the page, and safari browser said no text results.
rjtavares 8y ago
RT's model prioritizes consensus in quality instead of brilliance. Using RT for a best of list will give you good safe movies instead of outstanding ones. I'd say it's a good model for news.

krapp 8y ago

What will happen here is what happens with all sites which attempt to curate news or check facts: people who disagree with the site's conclusions will write it off as propaganda. Using "professional" journalists or "domain experts" at all will be a red flag for people who believe mainstream media and science only serves political and corporate interests.

jerf 8y ago
I have seen many efforts at "grading" news. I have not yet seen one that isn't just a news site that grades whether or not the news is in alignment with some ideology, with the occasional fig leaf tossed out for some particularly egregious error to try to appear even-handed. The closest I've seen are some news grading sites that are fair for a couple of months, but because a pure ideology site. As near as I've been able to tell, discarding all such sites (not just the ones that disagree with your ideology, but "all sites" is a superset of that) is the rational move.

Kaveren 8y ago

I place no value in review aggregators. They don't tell you anything useful. It's much better to find your own individual sources that you can trust over time, and they can earn or lose reputation with you based on the quality of their reporting. This isn't limited to news. I definitely don't trust a consensus of journalists to tell me a study is misinterpreted.

cat199 8y ago

give the user control of the reputation ranking, and you might have something here..

netsharc 8y ago
But it will just leave the system vulnerable to brigading. That even happened on RT for the Captain Marvel movie because of the female heroine, many "snowflakes" got offended and gave a movie that wasn't even out yet bad ratings. Kinda funny how these snowflakes (let me assume white males) call out "SJW"s while meanwhile they themselves actually believe society is mistreating them and so in their rage they're also expressing their own version of wanting "social justice".
- ataturk 8y ago
  I'm not sure the keyboard commandos are snowflakes or SJWs. Maybe guys with a bit too much sensitivity and time on their hands, though. But those guys do have a point--they consider themselves loyal fans who actually buy the merch and they're upset because suddenly Hollywood has no use for them. Hollywood is all up its own ass with Mary Sue "strong female lead" characters for marketing and signaling reasons. That people are calling them on their bullshit is generally a good thing. What Hollywood is really attempting to do is market to China, I think. In their messed up minds it is more important to have asses in seats and yet they don't even understand the demographic they are working for.
- motogpjimbo 8y ago
  > That even happened on RT for the Captain Marvel movie because of the female heroine, many "snowflakes" got offended and gave a movie that wasn't even out yet bad ratings. Alternatively, they were using the "Want to See" feature on Rotten Tomatoes exactly as it was intended to be used. People didn't like aspects of the marketing campaign for Captain Marvel and made their opinions known using a tool Rotten Tomatoes provided specifically for that purpose. There are many female-led films listed on Rotten Tomatoes that have very high approval fan ratings. By dismissing the public reaction to Captain Marvel merely as trolling by snowflakes, you're actually proving the point many people on this thread are making about partisanship and tribalism in news reporting.

ai_ia 8y ago

I have had this idea for so long. A Crowd-News Watch Dog which basically works like a wikipedia + archive.org + IMDB for News Reporting. There you can have profile for journalists or any content writers. You can basically see what a journalist has wrote about in her/his entire life. Political inclinations deducted from writings by crowd. The article written can be critiqued in the platform using a set of pre-defined set of rules. If the reporter is found employing strawman or fallacies or incorrectly reporting based on botched data/ statistical mumbo-jumbo, the commentary can be added by anyone and verified by other community members. I think it can keep rampant fake news and/or politically-charged posts masquerading as neutral pieces in check.

nukeop 8y ago
This would very quickly devolve into either a hardcore bullying platform, or a banfest of everyone who criticizes the favorite people of the site's owner, his friends, and moderators.
- ai_ia 8y ago
  If an argument put forward by the article doesn't abide by journalistic principles and is bad and crtiques offer why it is bad with coherent counter-argument, I believe there can be a middle ground somewhere.

lucideer 8y ago

This seems like a broader, more ambitious version of Snopes. If successful, I wonder would it replace it? Or could this be something Snopes could expand its existing community to handle?

ceejayoz 8y ago
Snopes is a great model for why this will fail, though. They'll be dismissed as "fake news" by one side or another on anything controversial. Hell, I've seen folks met with offended "did I ask for criticism?!" responses when they drop a "no, Bill Gates isn't going to give you a million dollars for forwarding that chain email" Snopes article in someone's Facebook comments.

deleted 8y ago

[deleted]

specialist 8y ago

"Fake news" is such an easy problem to solve: Just sign all the work. So you can verify who said what. Done. Don't see a signature, and it matters to you, just hit "next". -- Efforts to establish truth, factualness, credibility, whatever cannot succeed. Because humans don't work that way. The people who watch Alex Jones (for instance) don't care about your truth. So stop trying.

josteink 8y ago

Someone’s always negative in these threads, so this time it might as well be me. First of all: that headline really has an apples and oranges feel to it. Basically Rotten tomatoes worked because it merely curated and aggregated something which already existed in abundance: movie reviews, deriving a consensus based on that. Doing the same for news, will depend on something which has yet to exist at any meaningful scale: news reviews (and what would that even be?). So not only must it prove its value (and neutrality) as an aggregator, but it will also have to encourage the creation of something entirely new (news reviews) at scale, because otherwise it won’t have anything to curate. I’m not holding my breath.

saltvedt 8y ago

I'm solving a very related problem with https://cited.news/ https://cited.news/ It's a news aggregator based on Wikipedia references.

unityByFreedom 8y ago

Like Musk's pravda?

joeblau 8y ago

I wrote something that was trying to do this a back in January that was called desational[1]. What I did was very basic but the process I did went as follows. 1. Break up news articles into sentences 2. Pay Amazon MTurkers to tag sentences as `news` or `not-news` 3. Train a text classifier based on the results for Mturk 4. Process new articles using the trained model to identify and strip out parts pf the article that are sensational. 5. Give the article a percent score and a letter grade about how much "news" is actually in the article based on the percentage of sentences that were identified as news. I ran it for a few weeks on tech news, but then I figured that the news industry is not a business I really want to be in. The source code is all just sitting up on GitLab. [1] - https://desational.gitlab.io/desational.com/ https://desational.gitlab.io/desational.com/

DamnInteresting 8y ago
Very interesting idea. However, having dabbled with Bayesian classifiers myself, I suspect that your decision to break the article into sentences may be a fatal flaw...both the Mturkers and the classifier lose out on valuable context that way. I know because I took a a similar approach in one of my own projects, and the classifier could never gain enough 'confidence' to classify things, even after thousands of trainings. The approach might work pretty well if you broke it down to paragraphs, where some context is preserved, but the more context you provide, the better the classifier will perform, often for surprising reasons. For example, I built one classifier that was performing poorly until I re-trained it without stripping HTML and headers; after that it performed splendidly. Upon examination I found that tokens such as domains and IP addresses in headers and links were hugely influential (and rightly so) in the classification. Just something to consider. :) Apologies if I incorrectly assumed that your classifier is Bayesian.
- bduerst 8y ago
  Good point, but keep in mind that many news articles also break paragraphs out into single sentences. For example: https://www.reuters.com/article/us-france-china-airbus/airbus-close-to-signing-aircraft-deal-with-china-sources-idUSKCN1R61Y0 https://www.reuters.com/article/us-france-china-airbus/airbu...

ataturk 8y ago

Didn't RT get caught ginning the numbers for that awful Ghostbusters remake? Aren't there also some other notable WTF examples? Anything electronic, especially where big money is involved, is suspect.

LifeLiverTransp 8y ago

I cant wait for the first social civil war side trying to apply pressure to that approach.

rchaud 8y ago

I get the methodology but, what is the demand for this type of product? RT makes sense because there are too many movies coming out at any one time, and you need something to build a shortlist from. But news? Are users so spoiled for choice that they need a journalist's opinion on what's worth reading? This doesn't even seem like an anti fake-news play, since the people that consume and share that stuff aren't actually interested in news, just whatever can be used as a stick to beat the political opposition with.

beaconstudios 8y ago
presumably it's marketed as anti fake news but the reality is it will be politically biased towards one faction. That's usually how these things play out.
- amanaplanacanal 8y ago
  The thing is, even if it's not biased, as soon as something is published which is inconvenient for one side or the other all the partisans are going to claim it's biased. As they already do.
  - beaconstudios 8y ago
    That's where providing the evidence comes in - you'll never convince your opponents but for skeptics and critical thinkers they can determine for themselves if the evidence supports the perspective.

duwease 8y ago

I want to bring a note of positivity here -- while most comments seem to be poking holes in the concept, all also seem to acknowledge the need for such a service to be successful. This is a domain that I've also put a lot of thought into over the years, so I hope something succeeds. And some things do. Keep in mind that Wikipedia caught a lot of flak for being unreliable or impossible at first, but nowadays they have a robust editorial process for contentious subjects that I'd consider the gold standard for any such practical applications on the internet.

gambler 8y ago
>while most comments seem to be poking holes in the concept, all also seem to acknowledge the need for such a service to be successful. Speaking plainly: anyone who believes you can get better news through some kind of external meta-review process is a fool. Probably a fool who also thinks the review process should validate journalists that share their political biases, because that's what it's all about these days. You cannot test quality into the product. News sources are supposed to cross-validate each other in the first place. Clearly, that have stopped working some time ago. The only thing that can save news now is an overhaul of its financing and discovery models. >Wikipedia caught a lot of flak for being unreliable or impossible at first, but nowadays they have a robust editorial process for contentious subjects Hah.

josefresco 8y ago

This, and many other concepts based on "vetting" news assumes that the audience cares. A significant portion of the audience does not care, and is instead looking for news that validates their worldview, and furthers their (and their chosen political party's) goals. So basically these services will cannibalize the part of the audience that does care about truth, turning people against each other over marginally skewed articles. Meanwhile, the other site is happy to share falsehoods, because it helps them ... win.

deleted 8y ago
[deleted]
sureaboutthis 8y ago
To a point, you are correct, but most people don't "not care". Most people are unaware and only assume a knowledgeable, experienced editor vets and edits the news. Today, most news has a marketing angle applied to it to attract more viewers and visitors.
- swebs 8y ago
  I don't think "most" people even assume that. Only 1/3 of people under 30 even trust the mass media. https://news.gallup.com/poll/243665/media-trust-continues-recover-2016-low.aspx https://news.gallup.com/poll/243665/media-trust-continues-re...
- josefresco 8y ago
  The number of people who shared and continue to share "fake news" is a sign (to me) that most don't care about truth and are in fact complicit. These people are selfish, narcissistic, naive maybe all of the above. > Most people are unaware I just don't buy this based on the content that I see shared. When corruption (for example) is exposed it should be a slam dunk for both sides of the political debate. However, this is no longer happening, with each side digging in deeper.
TeMPOraL 8y ago
It's even worse. For most people, news is just a social object. Something to talk about with family, co-workers, etc. Whether or not any particular story is true doesn't matter - what matters is that everyone in their circle is aware of the described event (whether they read the same story, or a different one). Humans are pretty good at fact-checking and critical thinking when getting it wrong means some immediate loss. But most news stories are irrelevant to almost everyone's lives, and those few that are, don't have a direct and immediate impact.

vinw 8y ago

If you like this, then you might also like spidr[0] "SPIDR is an AI-powered news aggregator that uses text analysis to group similar news together. The aim is to provide fast access to relevant news from different viewpoints." [0] https://spidr.today https://spidr.today

DonHopkins 8y ago
I dunno -- the name "credder" sounds much more authoritative and trustworthy to me than "spidr", because they didn't leave out the penultimate "e".
- vinw 8y ago
  But in spidr's favour I've been able to use it for quite a while whereas credder is still just an idea.
e15ctr0n 8y ago
Thanks for posting this. The news aggregator looks really good. I would love to know how you built it. Which text analysis AI did you use to group the similar news together?
- vinw 8y ago
  Sorry I can't help with that, it's not my site.

rooam-dev 8y ago

Good luck to them if course, but I am not sure this will work. Couple of reasons come to my mind: 1. How will I know that news have been reviewed by qualified people? Technically this should be news agency/source's job. 2. By the time the score will depict the news' quality, the damage may be already done. 3. RT is based on people's opinions, but news should be fact checked. What I think it should do, is to score the publishers instead... some kind of publishing performance history. Although the most effective would be to have some kind of liability for spreading/posting fake news, this would make publishers to do more fact checking before fast publishing. All above is just my IMHO.

sureaboutthis 8y ago
Haven't read the article yet but I would hope, instead, it would rate these based on consistent quality and not per story.
crististm 8y ago
> 3. RT is based on people's opinions, but news should be fact checked. Except when RT doesn't like to hear people's opinion when it differs from the mainstream. Unfortunately it's easy to box it as "trolling" and hide it under the rug.
- crististm 8y ago
  Apparently I'm getting myself in a SJW hive. Shame on me. s/hive/swarm
  - martythemaniak 8y ago
    Did you mistake Rotten Tomatoes for Russia Today?
    - fartcannon 8y ago
      I think he's referring to when they delete reviews that are affected by cultural phenomenon.

darepublic 8y ago

Rather than create an RT for news, just create a better news outlet please. A new form of news would be the more revolutionary startup imo. I think any RT style news review site will end up being regarded much as Politifact is today -- valued only by those who agree with its ratings.

holler 8y ago
what would you like to see in this new news outlet?
- darepublic 8y ago
  well right now in news we get stories with the different points of view, and just the points of view talking past each other. I want news that deals in raw hard to get data, and that monitors the data continually on a day to day basis. ie. for disaster relief in a country, to report neighbourhood by neighbourhood how much is being received and by who and in what quantity. basically it would require a ton of work, and would be much easier to achieve that level of coverage in a single city at first -- but basically try to have omniscience source of knowledge, and where knowledge is being blocked or obfuscated to identify that.

mcv 8y ago

Considering how easily Rotten Tomatoes is gamed, including for political reasons, I'm not sure I want my news to be gamed the same way.

MentallyRetired 8y ago

Yeah, let's let the people decide if the news is accurate. What could go wrong?

who-knows95 8y ago

isn't this what elon musk was talking about? :https://www.cnbc.com/2018/05/25/elon-musk-may-actually-be-making-a-website-to-rate-journalists-for-credibility-and-core-truth.html https://www.cnbc.com/2018/05/25/elon-musk-may-actually-be-ma...

trophycase 8y ago

Until it is bombarded by fake reviews?

mmmchipotlemmm 8y ago

What is the evidence that people want 'trustworthy' news?

esalman 8y ago

Like academic publications, every news article should be published along with acknowledgement of funding, conflict of interest and maybe data availability statement. It is a lot easier to judge a news article when you know who own the publication and such.

CodeSheikh 8y ago

IMHO, journalism is slowly dying. These days you find more "opinion-based" or "make-it-viral" news pieces than proper investigative journalism. Services like this can help but at the same time they can create further segregation of true news vs poor news if most of the critics start growing to be biased.

wsc981 8y ago
I agree, I feel many times you can get more solid news from sources like Twitter. Especially how Western media covered Syria annoyed me to no end. To me the news coverage in my country seemed mainly aligned with Western political interests. But I was happy to be able to follow at least 2 journalists [0][1] that were "on the ground" in Syria for some good solid investigative reporting. --- [0]: https://twitter.com/jenanmoussa https://twitter.com/jenanmoussa [1]: https://twitter.com/HaraldDoornbos https://twitter.com/HaraldDoornbos
amanaplanacanal 8y ago
Journalism costs money. If it is funded by ads, then getting more eyeballs onto the ads is going to trump good journalism every time.

jmnicholson 8y ago

We're effectively creating the rotten tomatoes for science at scite.ai. We rely upon a deep learning model to classify citation statements as supporting, contradicting, or mentioning.

chungleong 8y ago

Jesus, as though there isn't enough vain people working in the news business already. Society needs fewer trophies.

bg4 8y ago

I'd love to see this in combination with Knowhere News (https://knowherenews.com/ https://knowherenews.com/).

ryanmarsh 8y ago

I thought that was what /r/worldnews was for?

overthemoon 8y ago

What value does Rotten Tomatoes provide? Is review aggregation really the model for truth we want to use? "A lot of people find this reliable/unreliable" tells you exactly that and nothing more. This is an attempt to cram the huge, complex enterprise of discourse into some kind of boolean true/false reliable/unreliable binary it cannot fit neatly into. Disinformation and tribalism are social problems that aren't waved away with technological solutions. At the end of the day, I'm not interested in startups as truth gatekeepers. It's empty when it comes to movies--it's downright disingenuous when it comes to the news. It provides the veneer of objectivity by being a technological solution, but the enterprise's foundation is still ideological. I trust them as little as any other outlet that can't honestly assess and describe their own viewpoint. At best, you're adding another layer that will itself become ideologically polarized. If this or similar platforms succeed, we'll see a proliferation of them by people who feel like it is too <ideology> or not enough <ideology>.

Mindstormy 8y ago

Isn't this kind of already a thing? http://www.emergent.info/ http://www.emergent.info/

ObscureMind 8y ago

How is that having journalists reviewing the work of other journalists won't lead to corporatism?

kosei 8y ago

Sort of reminds me of the website SpinSpotter. Great idea, but had trouble finding a real audience for its product. https://en.m.wikipedia.org/wiki/SpinSpotter https://en.m.wikipedia.org/wiki/SpinSpotter

NoblePublius 8y ago

This seems far too complicated. The UX of Dissenter (boo, hiss, I know, control yourself) is simple by comparison, doesn’t involve another destination site, and doesn’t involve a committee deciding whose reviews are more worthy of consideration than others. What Id prefer is a ranking system of journalist merit: does this story cite anonymous sources? Is this story original reporting? Does this story simply repackage someone else’s original reporting with an editorial headline? Is it news, editorial, or opinion? What corporations advertise with the parent company? These are objective measures of newsworthiness and Credder would track none of them by default.

jellicle 8y ago

And if you have questions about the credibility of Credder, you can just check the Credder ratings on Verrit, which is itself verified by Truther and Accurit, which are backed up by Credder. Solved! News trustworthiness is not a technical problem and cannot be solved by technical solutions. News: something you don't know. Verifying news: asking people to opine about the trustworthiness of something they don't know. > "Here's an opaque cardboard box. In it is a banana. Now rate my news story on its accuracy. NO, you are not allowed to open or inspect the box in any way. Also, a lot of you have strong political opinions about bananas. Ready, GO!" Garbage in, garbage out. Far too gameable, a fundamentally insoluble problem to begin with.

deleted 8y ago

[deleted]

Dirlewanger 8y ago

A RT for news isn't going to fix the broken medium that is online journalism. Basic research, fact checking, editing/proofreading, journalistic integrity -- all things we take for granted with the news have been discarded in favor of more and more clicks/profits. If anything, we need a new model for how news is manufactured/distributed in the age of the Internet.

OscarTheGrinch 8y ago

4. Will be gamed by groups of bad actors.

DonHopkins 8y ago

Dear Credder.com: thank you for using a penultimate "e" before the final "r". Such a breath of fresh air! (Was it because Creddr.com was already taken? ;)

Circuits 8y ago

I like the idea but I don't know if I agree with the proposed approach to reviews. Personally, at a minimum, I would like to see the following from every reviewer: A structured summary- background, objectives, data sources, synthesis methods used, results, limitations, conclusions and the implications of important findings drawn by the article. In more detail: An introduction that presents the problem and the issues dealt with. An assessment of the methods which describes the research and the evaluation process; specifies the number of studies references and gives a brief evaluation of their validity. A description of conclusions/results and their quality based on identifiable and preconceived academic/professional metrics. A strong summary of conclusions, limitations of those conclusions and the outcomes of the procedure used for the research. I would really love to see reviews of reviews as well as a reviewers rating system (individual reviewers should have their own ratings) but given the limited number of reviewers that maybe asking too much.

catchmeifyoucan 8y ago

How do they make money

Causality1 8y ago

Isn't this the same crap Elon Musk was talking about when the media was roasting him for unsafe conditions at Tesla and calling rescue divers pedophiles?

laythea 8y ago

In my opinion, this doesn't really offer any value as the only way to get a good understanding of the news is to read from lots of different places and come to a view. Using this strategy then consuming a truly "random" site is better than consuming pre reviewed site as there is no bias in selection or outcome. It's not like films, where each one is independent (except marvel and dc comics obviously :)

TomMckenny 8y ago

Maybe this project will help the general public start reading reality based media. But we do have this already. Done by experts. It's called the Pulitzer. The people in a democracy must have access to information. Once a public is persuaded to reject reality based media, democracy becomes irrelevant. Thus the campaign to denigrate it in the dozen nascent autocracies. Again, I hope this project helps reverse that.

silveira 8y ago

People who want falsified narratives about reality don't care about these ratings. They will even go beyond and create their own "alternative facts" seal and go with that.

phil248 8y ago
True, as we already know from the myriad fact-checking sites.

gabbygab 8y ago

Perhaps they should have used a better example than rotten tomatoes. RT is owned by NBCUniversal and is terribly unreliable as a movie ratings aggregator. At one point it was a very useful site, but now it is useless. Might as well ask disney, nbc or studio execs on which movies to watch. We know that RT has lied and fudged ratings for financial or other purposes. It has the same problem as NewsGuard. It is controlled by industry insiders to benefit themselves and the industry. Would anyone trust an oil company ratings company controlled by oil companies? Rather than a "rotten tomatoes" for news, I'd rather have a "meta" news or wiki site which has a running list of all the fake news that news companies have pushed out. Make it part curated and part user driven. Let users provide, comment and even contribute to it. Make it listable by news companies and journalists. So we can see which news companies and which journalists have pushed the most fake news to the american public.

tomohawk 8y ago

How would this pick up on the problem where stories such as the Mueller investigation are hyped and mischaracterized for a long period of time, and in the end Breitbart ends up being more correct on the story than most other news outlets?

chillingeffect 8y ago

It would be much more valuable to supply chains of evidence in news stories. So often, articles are treated as isolated islands with unresolved references and only loose links to previous events. News could be so much more credible if it had extensive underpinning of every exertion. For example, in Europe, news stories almost always include a map of the local area. But one could do much better with timelines showing how each article is a step along an unfolding story, empowering readers. In theory, readers can go do all the research themselves, but the cost is quite high. That's almost exactly what the WWW was designed for! It could also includes links to wiki-like pages of every individual related to the story. But obviously the big question is market. Especially these days, people don't want evidence, only emotional crack. People for whom the information is actually financial valuable pay for accurate information in their niche. So the real challenge is to motivate people to focus on reality instead of their outrage addiction. That's a cultural goal and more likely to be a non-profit enterprise. :(

gustavmarwin 8y ago

That's a step in the good direction, I think curating the news is going to be a very precious service. I'm looking forward to decentralised ways of achieving this though, such as with https://civil.co/ https://civil.co/

_bxg1 8y ago

This is just going to descend into review-bombing chaos (look at what happens to games on Steam, and then think about how much more biased people are about news sources). A much better idea would be a curated news source registry that does objective fact-checking and scores news sources based on various criteria, with the relevant research/scoring mechanisms shown openly so people can verify the results themselves. Like Snopes meets Charity Navigator (https://www.charitynavigator.org/ https://www.charitynavigator.org/).

sharkjacobs 8y ago

I hope they're paying close attention to Rotten Tomatoes' problems with politically motivated review bombing.

bargl 8y ago

I think I posted this somewhere else but here it goes. I want a system that rates journalists not news. I want to see my previous ratings of this journalist and be able to base it on that. I want the ability to rate a pieces objectivity. I want to see ratings from users who have been proven to know how to rate objectivity. I want a system that uses Machine Learning to break me "out of" my information silos by promoting the opposite opinion, and saying as much. "Because you like, Capitalism here's an argument for Socialism." I want a system that making ratings less of a popularity contest by limiting them in a meaningful way (this is pie in the sky I'll take the first two). I don't think it's possible to create the things I want, but that's what I want. Not Reddit for news. I've seen how that works.

ilaksh 8y ago

People in this thread do not seem to appreciate the real threat to society that censorship and propaganda pose. A widely used news rating site will be used to suppress political dissent and alternate views. Maybe take a look at history and the world. Do you think that Chinese authorities would like a tool installed on everyone's computer that could rate the trustworthiness of a news article or site? Of course they would love it because they could just downgrade any reporting that was critical of the party. Based on the number of young people who seem to be begging for authorities to control their information, it appears that the China model is coming to the United States.

unityByFreedom 8y ago
I'm with you.

dancemethis1 8y ago

So... A website clearly intended to be a parody due to its name, but still taken seriously by people that didn't get it?