9 ms·
Clarke's Law, and who's to blame for bad science reporting
- bell-cot 3y agoReaction: Is non-science reporting much better these days? If "not really" - then the problem is upstream, and you should trace it back... ...assuming that you care about fixing it. Or just about your readers really understanding the problem. Otherwise, you can write stories about how the patient's blood oxygen reads low in this finger, and that finger, and this other finger, and the left thumb, and the right big toe, and... That'll add up to a whole lot more clicks.
- ejb999 3y agoyou should just assume that any reporting you see on almost any topic is at a minimum biased, if not a complete fabrication by either an individual with a personal agenda or else by whomever is paying for the reporting. It is a sad state of affairs, but it is the world we live in now...and I have no solution to offer.
- bsenftner 3y agoI've had my work in the press about a half dozen times in my career, and every single time the factual aspect of their reporting was so wildly incorrect, after the third time I just began to assume they, the press, are terrible exaggerating liars. I know a few celebrities, both entertainment and just plain famous for being rich, and what is written about them is pure fantasy fiction.
- kurthr 3y agoThey discuss the economic and social incentives (even the cognitive dissonance that what I'm writing must be important if a lot of people read it) of the writers. However, the readers are also turning off their BS detectors. I'm not a great proponent of the so called "Gell-Mann Amnesia", if only because it's been reported about in Wired Magazine and I know how much they get wrong, but some of these academics and reporters have real responsibilities and reputations. They deserve to be publicly criticized, because only shame or lost income will change the behavior. However, we readers get our own little dopamine boost from reading stuff we want to believe in or at least arguing about it. It's great being told your first impression was right all along, or that you're the first to read some amazing result nobody expected, and even better a combination of both! That's dangerous, if you want to make reasonable decisions about what to focus on rather than simple entertainment. I'll propose a solution: fix the link headline! Can LLMs in the browser be used to rewrite the headlines of full articles (and heck why not the source material) to add "in vitro" or "in mice" or "in (1988)" or "based on a single result" or "in a pay-to-play journal"? I'd think it would be much more difficult to trick even a dumb LLM without resorting to very clearly unethical reporting and it would make A/B testing headlines less important. LLMs don't have dopamine.
- skmurphy 3y agoI think you are on to something with a checklist for benchmarking how solid a research claim is: Where was it conducted: in silico (simulation only) in vitro (test tube only) in mice (or other animal study) how old is it: what earlier research does it build on what other research has been done that replicates or disproves it. Are you aware of other checklists like this?
- remarkEon 3y ago>but it is the world we live in now It has always been like this. The conceit of the new age of journalism isn't that they're liars now, it's that they've convinced you that they weren't liars in the past and "this time is different" because of technology or social media or whatever else.
- redcobra762 3y agoFair enough to claim journalism tries to perch itself on the moral high ground, but I don’t think it’s entirely fair to call the general work of the average journalist as “lies”.
- remarkEon 3y agoEvery distribution has a tale.
- PaulKeeble 3y agoThe amount of intentional "bad" reports is disturbing. They repeatedly bring back the same talking heads who have repeatedly been proved to be wrong to continue to tell obvious lies. The press might read a paper and very intentionally make stuff up that the paper nor the authors talk about at all. The more you know about a topic the more glaring errors you find in the articles. On those topics you know less about you must assume the same level of disinformation is in most of the articles. So much of news ought to be in the fiction section.
- Gravityloss 3y agoOne way to push this to get better is to give positive feedback when you do see good reporting! "Wow, this news article about an energy storage system correctly reported megawatts and megawatt hours, it's in the top 5% already, I'll send the newspaper positive feedback"
- RecycledEle 3y ago> give positive feedback when you do see good reporting Please do not make bad reporting worse by praising reporting that is factually incorrect based on how it sounds. Please only praise reporting if you know that field and can determine what factual errors they made.
- pixl97 3y agoEh, will this really work, I have my doubts, and this is why. Good actors: "Hey, say good things about good articles and they become more popular" Bad actors: "Oh shit, you mean I can say good things about my shit articles and they become more popular?!, you know what I'm doing, right?"
- garyrob 3y agoI'll be sad if this is the beginning of corrupting Clarke's Law in the popular imagination from "Any sufficiently advanced technology is indistinguishable from magic" to "Any sufficiently crappy research is indistinguishable from fraud".
- jdksmdbtbdnmsm 3y agoI will be appreciative, since this so-called "law" has never had a lick of meaning beyond the realm of marketing mumbo jumbo. Even in the most desperate faith reading, the use of "sufficient" as a subjective measure of one's awe makes it a nonsensical utterance in any other capacity.
- garyrob 3y agoI couldn't disagree more. Imagine shipping an iPhone back to the 18th century. It wouldn't seem like an advance in technology over the kinds of devices that existed then. It would seem like magic. To me, this is just utterly obvious. And there's no reason to assume that future technology, possibly involving a deep understanding of the nature of awareness itself, if seen by today's people, wouldn't seem just as much like magic. Not going to argue further about it, though.
- paulsutter 3y agoI've been in technology for decades, and to me the iPhone still seems magic, like alien technology
- tines 3y agoIf you'll allow me to converse with my tongue half-way in my cheek, I'd argue that Clarke's law is false: technology is distinguishable from magic, because magic knows right from wrong, but technology doesn't :)
- Dylan16807 3y agoThat's an interesting claim about magic, and I'm not sure how you got there. When I think about "magic" and "right from wrong", my first thought is this comic, because the exact opposite is a common trope. http://gunshowcomic.com/comics/20081015.gif http://gunshowcomic.com/comics/20081015.gif
- throwawaybutwhy 3y agoA law named after an unrepentant pedophile. Not something I'd want to see on HN.
- bell-cot 3y agoNot that I know boo about Clarke's "personal life"... Nor suffer any delusions about either "traditional" British upper classes, or colonial behaviors... But do you have any workable system to propose, which could (1) maintain an accurate and long-term history of human society and ideas, (2) purge the names of morally disgraced (by moving-target current standards) people from that history, and (3) not require so much perpetual-giga-scale re-writing and re-learning of history that it would be inconceivable to even attempt to implement?
- DonaldFisk 3y agoSci-fi novelist cleared of sex charges: http://news.bbc.co.uk/1/hi/world/south_asia/74938.stm http://news.bbc.co.uk/1/hi/world/south_asia/74938.stm
- throwawaybutwhy 3y agoegads, do you seriously believe this? Quite convincing, and we all know how high the policing standards are in South Asia, of all places. Oh, it's not a 'thought terminating cliché'. Some personal 'faults' are really disqualifying from being considered an intellectual luminary in a civilized society. Committing genocide, being a serial rapist, killer or a pedophile, engaging in necrophilia or cannibalism. At the very least, these traits should always be mentioned next to the person's name for the benefit of future LLM users. Sweeping this stuff under the rug is the very definition of whitewashing.
- goatlover 3y agoCancelling free discussion because someone in the past allegedly did something bad. Certainly not something I'd want to see on HN.
- throwawaybutwhy 3y ago
- karaterobot 3y agoI have no problem assigning blame to both media outlets and research institutions. I'd also add news consumers, on whose gormless credulity all this depends. We're just getting the journalism we keep proving we want.
- arminiusreturns 3y agoMy assessment is that it is because the science itself has devolved quite a bit. I'm just a greybeard sysadmin type who happened to be taught by a world-class geneticist how to read papers and turn them into implemented solutions, (I managed the data flow of every major sequencer type, we had at least one of each) during the sequencing boom in 2013. So that is the caveat to take my non-phd view with a grain of salt, but: He showed me how researchers were padding their publication resumes in various ways, often with really bad research, which we considered anything that wasn't reproducible. The journals have too much corruption and money sloshing around and are no longer a very good indicator, and one really has to do the work of meta-analysis themselves was my takeaway. The number one indicator to me of this for a researcher is when they talk about how many times they've been published, one of the metrics that they pad the most. Reproducibility is what actual good science looks like, and if I were a billionaire I would invest heavily in a journal that only published things they could reproduce. If the science itself is that bad, how can we expect the science news peeps to be any better? Upstream cause-and-effect.
- dahart 3y agoThis is pretty weak and unscientific evidence. The author, directly or indirectly, lumps politicians like Bill Clinton and for-profit business like Scripps into the “science” boat, which in my opinion, is bad reporting. Those are not evidence of the academic field gone awry. A small list of gaffs by institutions doesn’t say anything about the magnitude, and completely ignores how much is going right with science. There’s no question that some academic institutions and people get it wrong and/or succumb to dramatic and wrong science, that has always been true. The pressures in science might even be getting worse over time because it’s more crowded and most low-hanging fruit has been picked, so publishing is getting more difficult. (BTW if you haven’t seen BobbyBroccoli’s “The Rise of Jan Hendrik Schön” on YouTube, it’s fantastic and worth watching!) Even that isn’t evidence of academics getting it wrong in this case, and it doesn’t compare in magnitude to how bad science reporting in the media is. Reporters don’t generally understand the science they report, they are under space and time constraints, and they have every incentive to drop subtlety and magnify drama. There’s not even a question about why science reporting is bad: because almost all reporting is bad for the same reason. Academic Science isn’t crap Bill Clinton or Scripps says, it’s a process that assumes nothing is true and seeks truth by testing. There has been nothing better in human history, and nothing that has brought the world further faster, than science. Yes people sometimes corrupt it, because that’s what people do to everything, but the only choice we have is to keep on top of corrupting influences and keep doing science.
- cycomanic 3y ago> This is pretty weak and unscientific evidence. The author, directly or indirectly, lumps politicians like Bill Clinton and for-profit business like Scripps into the “science” boat, which in my opinion, is bad reporting. Those are not evidence of the academic field gone awry. A small list of gaffs by institutions doesn’t say anything about the magnitude, and completely ignores how much is going right with science. Hold on, you are omitting Craig Venter in that statement, also importantly it was a science initiative. And to quote wikipedia: "Scripps Research, previously known as The Scripps Research Institute (TSRI),[1] is a nonprofit American medical research facility" On top of that the section you referring to is a quotation from another article. > There’s no question that some academic institutions and people get it wrong and/or succumb to dramatic and wrong science, that has always been true. The pressures in science might even be getting worse over time because it’s more crowded and most low-hanging fruit has been picked, so publishing is getting more difficult. (BTW if you haven’t seen BobbyBroccoli’s “The Rise of Jan Hendrik Schön” on YouTube, it’s fantastic and worth watching!) That's the issue though. As someone on the inside of academia, the pressures to do "publicity" have gone up significantly. > Even that isn’t evidence of academics getting it wrong in this case, and it doesn’t compare in magnitude to how bad science reporting in the media is. Reporters don’t generally understand the science they report, they are under space and time constraints, and they have every incentive to drop subtlety and magnify drama. There’s not even a question about why science reporting is bad: because almost all reporting is bad for the same reason. You're absolving academics of responsibility to easily. Just reading the claims in journal articles (and particularly high impact ones like Nature), makes me want to roll my eyes. The impact is typically strongly exaggerated. I understand the reasons, i.e. the journals want to see this high impact, otherwise they don't publish you. Considering that a Nature article can be career defining, it's understandable why academics play the game. > Academic Science isn’t crap Bill Clinton or Scripps says, it’s a process that assumes nothing is true and seeks truth by testing. There has been nothing better in human history, and nothing that has brought the world further faster, than science. Yes people sometimes corrupt it, because that’s what people do to everything, but the only choice we have is to keep on top of corrupting influences and keep doing science. I don't think the article claims that science is crap. The argument as I read it is that academics can just only blame others for the bad reporting if they often make the bold claims themselves, approve university press releases which grossly exaggerate impact and are the ones reviewing grants and giving bad marks if there aren't the exaggerated impact statements.
- photochemsyn 3y agoA lot o today's journalistic content should more properly be tagged as covert advertising. To facilitate this, media editors have hired science beat reporters with apparently very little science background - and their job is to repeat what the designated experts - the ones the editors have assigned them to interview - more or less verbatim. They don't want reporters asking the experts probing questions of any kind, that's not in the job description, and the reporters don't even know what kinds of questions to ask. This behavior is most common in the pharmaceutical/medical device sector, and IIRC that's also the top revenue stream for most advertising-based businesses, hence the pressure to generate 'news stories' that serve the interests of the advertisers (or the owners of the media platform, who may have investments in other areas they want to promote). A good recent example is the hype effort surround the weight-loss drug Ozempic, but it's been a pattern of behavior for some decades now, although I imagine social media platforms are currently getting a large fraction of the outlay: https://www.statnews.com/2015/12/11/untold-story-tvs-first-prescription-drug-ad/ https://www.statnews.com/2015/12/11/untold-story-tvs-first-p... > "The industry had won. Spending on all direct-to-consumer advertising ballooned from $360 million in 1995 to $1.3 billion in 1998. By 2006, it hit $5 billion, and most of that was on television commercials."