13 ms·
Beware the Man of Many Studies
- aaxe 3y agoUseful in this context: https://examine.com/guides/how-to-read-a-study/ https://examine.com/guides/how-to-read-a-study/
- roenxi 3y agoThe rational response to this point-counterpoint is to decentralise power & let people take their own risks as far as we are able for the truth is a rare and delicate flower which we cannot reliably identify. The only fair option is to let people make their own decisions. But for some reason the level of abstract thought required to justify that is just too high and people can't overcome their instincts. It seems nearly every time there is a crisis the body politic centralises power and, usually, gets a much worse result than just letting people do what makes sense to them. Then people just assume that because they did something and the world didn't literally end in fire that they what they did helped. Then in the good times they do their level best to run down whatever reserves are available and prepare to go in to the next crisis with nothing but elbow grease and optimism. Human nature: the biggest threat we face.
- dash2 3y agoI don't think that is really relevant. You can't decentralize the decision to (e.g.) increase or decrease criminal sentence length by ten per cent, which is why we need good economic studies to estimate its effects. In general, many policy choices cannot be decentralized away. Also, even if you decentralize them, people still want to know what their choices will do. Suppose I'm considering getting a divorce and I want to know how it will affect my children. Is everyone in this position supposed to do their own literature review? Or just "use their own good sense" and guess what the effect will be?
- golergka 3y agoYou can. Polycentric law and charter cities.
- notahacker 3y agoBoth are poor at yielding useful results for social policy. "Charter cities" and equivalents will give you all sorts of results that don't generalise (judging by the number of businesses that relocated their tax domicile to this underpopulated area, a 1% cut in corporation tax could lead to a 1000% rise in GDP per capita) and "polycentric law" won't really resolve any legal questions other than "in which circumstances can x persuade/force y to accept arbitrator z's decision"
- constantcrying 3y ago>You can't decentralize the decision to (e.g.) increase or decrease criminal sentence length by ten per cent You absolutely can. It also exists right now in the form of federalism, where people can, to some limited extent, choose their own legislation and policies.
- deleted 3y ago[deleted]
- roenxi 3y ago> You can't decentralize the decision to (e.g.) increase or decrease criminal sentence length by ten per cent. Sure we could; different laws in different locations; let people see what works. That is how the world does things now - note that different countries have different legal codes and by experiment everyone comes to a sort-of consensus on what works, with lots of variance between jurisdictians when people don't agree. That diversity of law can be re-done at any practical scale. I quite like the idea of different cities having different laws (oh, for the world to have a few more Hong Kongs in it! And note how well single-city legal entities usually do).
- pixl97 3y agoI mean, we already have that. For state crimes we have state sentencing, for federal crimes we have federal sentencing. >That diversity of law can be re-done at any practical scale. Eh..... Only at an extreme cost. I mean in a few minutes of actually thinking about the problem I hope you could see how this would totally fall apart. For example you live in BumFkt Nebraska and sell widget X. Totally legal in BumFkt. Your neighbors in Timbuktu Nebraska order X on your website and you ship to them. Oops, you've both committed felonies in Timbuktu, and now if you ever go there you'll be arrested on site and imprisoned. Typically every state constitution in the US holds itself as the highest authority in that state. Cities and make their own regulations, but the state is the final arbiter in setting the limits to their power to keep petty squabbling shit from damaging the citizens of said state. After that the federal government can step in, and quite often has as some particular states love to violate human rights dependant on the minority group you're in.
- constantcrying 3y agoYes, there is also a huge mismatch between what politicans want from "science" and what "science" can provide. Any legitimate scientific field is in a constant faction war, on various issues and making serious political decision based on that is insane. And the consequence is various highly suspect policy decision which were essentially purely based on some "scientific" subfaction gaining political power and enforcing their beliefs. The effects of these decisions are usually, likely by design, impossible to measure and subject to an enormous amount of confounders. "Success" and "failure" thus become largely up to chance and a "consensus" emerges, which is in truth largely based on coin tosses and more or less meaningless.
- cshimmin 3y agoYes! In the US particle physics community, they came up with a solution to this. Every ten years, the field as a whole gets together and sets an agenda for the field that self-prioritizes funding, before making a case to the national funding agencies and Congress. The idea is that if the high energy folks are computing with the neutrino folks, and things get nasty to the point where people on each side are saying stuff like "neutrinos suck, here's why you shouldn't fund them and fund us instead!"... All the suits are going to hear is "physics sucks here's a bunch of reasons we shouldn't fund them". The process/report is called P5 (particle physics projects prioritization panel), and it's actually happening right now to form the coming decadal report.
- walthamstow 3y agoAssume you meant competing rather than computing? Either could work!
- naasking 3y agoSounds like a great way to setup an echo chamber.
- geraldhh 3y agoconsensus chamber?
- api 3y agoI agree as long as it doesn't endanger others, but when it does you start to have a more complicated ethical problem. E.g. I'm generally in favor of at least a pressure to vaccinate for vaccines that are effective at reducing spread, but less so in cases of vaccines that are not sterilizing. In that case you're only taking a risk with your own health but are having a negligible effect on the overall spread of the disease. Things also get more complicated when dealing with minors, such as when parents refuse medical treatment for minors due to religious (or "new age" or political) reasons.
- RandomLensman 3y agoNot sure why you ignore that a lot "regulation" also came from the private sector because they didn't want to deal with stuff going wrong all the time so they self-regulated. People on their own do decide that there is stuff they don't want to leave to chance. Also, your proposition means people need to be able check out of society proper - good luck with that. Otherwise someone else will need to pick-up the tap for those "own decisions".
- biomcgary 3y agoPeople often decide they don't want to leave competition to chance. Regulatory capture is frequently used to reinforce entrenched power. Most recently, Sam Altman of OpenAI fearmongering for AI regulation, since OpenAI is currently in the lead. If he agreed to shut down OpenAI for the greater good, he'd be a bit more credible.
- anonymouskimmer 3y agoIt seems to me robotics companies are "in the lead" on AI as it pertains to (semi) autonomous decision making.
- RandomLensman 3y agoNot quite sure how that relates to private rule making, i.e., no politicians involved
- pixl97 3y ago"Private rules" = "no rules" Or I should say, the rules can only exist where natural monopolies exist, any competitive market will descend into chaos as the number of entrants in the market will increase and will quickly defect from any private rule set.
- RandomLensman 3y agoNot true at all, have a look at how people got around to regulating steam engine boilers, for example. Not natural monopolies, either.
- brookst 3y agoI don’t entirely disagree, but I think you lean too far in seeing the world as entirely populated by independent individuals whose actions don’t affect others. I am 100% fine with seat belt laws because other peoples’ fatality collisions affect me financially and in traffic jams. Society is a complex system and I don’t think “just let everyone do whatever they want” is a great answer when many of those things impact others, directly or indirectly.
- jonhohle 3y ago[flagged]
- pas 3y agowhy assign more inherent value to life than the one actually dying does?
- jonhohle 3y agoIt’s a stretch to directly couple seat belt wearing to one’s own value of their life. While (afaik) seatbelt research is pretty definitive a very closely related area, car seat research, is plagued with confirmation bias, FUD, and a lack of interest in exploring alternatives (car seat makers are the ones who fund the crash tests)[0]. It’s crazy to go to hyper modern cities that don’t put children in car seats (Seoul, for one) while in the US you’d hear shrieks of horror if your kindergartener got into a car without one. So who benefits from car seat laws? It seems like primarily car seat manufacturers. Back to seatbelts, I don’t know why someone would choose not to wear one, but if they have a reason that they value more than the safety a seatbelt provides, ultimately it affects them and their family far more than it would ever affect me. People need agency over their own lives. Not wearing a seatbelt seems like a minor inconvenience to most people, may impede others. New Hampshire doesn’t require adults to wear seatbelts and doesn’t have as many per capita motor vehicle fatalities as neighboring states. In fact, it has the second lowest rate of deaths per 100M miles traveled in the country (obviously very different from large urban areas). So an assumption about seat belt laws equaling fewer vehicle deaths may not even be a valid assumption. 0 - https://freakonomics.com/podcast/how-much-do-we-really-care-about-children-ep-447/ https://freakonomics.com/podcast/how-much-do-we-really-care-...
- nathan_compton 3y agoIf this were really the optimal solution to all problems we would already see it not just at the level of human society, but all over nature as well. And yet we don't. Group behavior is extremely powerful. It might be the case that atomized individuals will find "truth" faster, but a collective can leverage truth much more efficiently when it is found. Not only that, but nature is replete with organisms for which group behavior is clearly an advantage. I think its interesting that you say, ultimately, that the rational response can't work because of human nature. But if that is the case, then it can't be the rational response. The rational response would be the optimal response that takes human nature into account. I can't say I know what that is, but I doubt its anarchy.
- webnrrd2k 3y agoIf you really think that leting people ”take their own risks as far as we are able" is the answer then I'd love to take my own risks by ignoring all traffic laws. And if, as you claim, "the truth is a rare and delicate flower which we cannot reliably identify" then I'll even start a giant Simpsons-style tire fire in my backyard. Clearly it's unpossible to identify the hazards of such an enterprise.
- p_j_w 3y agoI wonder how GP would feel about his neighbor operating a meth lab.
- renewiltord 3y agoI think it is possible to modify his idea with the concept of externalities to make it work. Do you feel that is not the case?
- anonymouskimmer 3y agoThe problem with trying this for scientific investigations are that funding is the fundamental bottleneck. Labor is the other bottleneck, and it's one that is fundamentally at odds with the entire concept (as scientifically trained laborers can't work on their own ideas, but must work on the ideas they are employed to work on).
- mcguire 3y agoHow would you identify and impose costs for externalities without immediately being accused of "nannying" or legislating the outcome?
- wrp 3y agoTFA is a caution against accepting the results of a meta-analysis in a field where the typical research quality is low, because you can't realistically correct for all the underlying problems. I personally think you should be skeptical of meta-analyses in any field, because it is very, very hard to do one "correctly".
- haldujai 3y ago> accepting the results of a meta-analysis in a field where the typical research quality is low. I can only speak to medicine (which is filled with low quality research) but this is a non-issue in my field. Who ignores quality in a SR? Low quality research leads to a low certainty SR. See Cochrane for an example of how to do this correctly. A lot of their review use low-quality studies because that’s all that exists, it’s very much disclosed and transparent. > I personally think you should be skeptical of meta-analyses in any field, because it is very, very hard to do one "correctly". The point of PRISMA was to make it easy. In diagnostic accuracy we also have QUADAS-2 and STARD. It’s really not that hard, the point of EQUATOR network and researchers like P Bossuyt & co was to make things easier. Any medical SR would include a PRISMA statement these days. One should be skeptical of any research and critically appraise it, but systematic reviews aren’t necessarily “harder” to do than primary research studies. They also carry more weight so we have to be able to trust the methodology (hence PRISMA). Many groups do this correctly… If you mean the old school assumption that "pooling low-quality studies in a SR automatically makes it high-quality evidence" that's not really a thing anymore. The problem is a lot of primary research is at high-risk of bias. So coming up with a low-risk SR is hard, making the assessments is not. https://www.sciencedirect.com/science/article/pii/S1743919121000406 https://www.sciencedirect.com/science/article/pii/S174391912... https://www.acpjournals.org/doi/full/10.7326/0003-4819-155-8-201110180-00009 https://www.acpjournals.org/doi/full/10.7326/0003-4819-155-8... https://pubs.rsna.org/doi/full/10.1148/radiol.2015151516 https://pubs.rsna.org/doi/full/10.1148/radiol.2015151516
- deleted 3y ago[deleted]
- Gatsky 3y agoYeah, well in my field, oncology, meta-analyses are somewhat irrelevant. As you can imagine, the bar for completing a phase 3 randomised trial in oncology is pretty high. Meta-analyses are mostly there for trainees to notch up a paper. Another fine example of the author's point is the ivermectin in covid meta-analytic nonsense (which I cannot even bring myself to link), where a bunch of small rubbish trials are meta-analysed into a 'flawless' body of evidence while double blind randomised trials are impugned.
- haldujai 3y agoI don't know what part of oncology you're in but meta-analyses are still very much important.
- Gatsky 3y agoDisagree. I change practice on Monday after a single quality trial. Pick up any society guideline, only a small amount of the recommendations rely on meta-analyses. Look at immunotherapy or antibody drug conjugates, revolutionary therapies that arrived one trial at a time.
- haldujai 3y agoFor what part, cutting-edge treatments? Sure. But oncology is a very big field… For example, the majority diagnostic testing guidelines are based on meta-analyses. FYI since you asked to pick one, let's take a look at NCCN. Not sure where you're drawing your conclusion that most evidence is from phase III RCTs: "[Q] Quality and quantity of evidence refers to the number and types of clinical trials relevant to a particular intervention. To determine a score, panel members may weigh the depth of the evidence, i.e., the numbers of trials that address this issue and their design. The scale used to measure quality of evidence is: 5 (High quality): Multiple well-designed randomized trials and/or meta-analyses 4 (Good quality): One or more well-designed randomized trials 3 (Average quality): Low quality randomized trial(s) or well-designed non-randomized trial(s) 2 (Low quality): Case reports or extensive clinical experience 1 (Poor quality): Little or no evidence" "The overall quality of the clinical data and evidence that exist within the field of cancer research is highly variable, both within and across cancer types. Large, well designed, randomized controlled trials (RCTs) may provide high-quality clinical evidence in some tumor types and clinical situations. However, much of the clinical evidence available to clinicians is primarily based on data from indirect comparisons among randomized trials, phase II or non-randomized trials, or in many cases, on limited data from multiple smaller trials, retrospective studies, or clinical observations. In some clinical situations, no meaningful clinical data exist and patient care must be based upon clinical experience alone. Thus, in the field of oncology, it becomes critical and necessary (where the evidence landscape remains sparse or suboptimal) to include input from the experience and expertise of cancer specialists and other clinical experts." From an article: "We identified 1124 potential systematic reviews from our survey of the 49 NCCN guidelines for the treatment of cancer by site. Five NCCN guidelines did not cite any systematic reviews." https://www.nccn.org/guidelines/guidelines-process/development-and-update-of-guidelines https://www.nccn.org/guidelines/guidelines-process/developme... https://jamanetwork.com/journals/jamaoncology/fullarticle/2749257 https://jamanetwork.com/journals/jamaoncology/fullarticle/27... https://www.nccn.org/guidelines/guidelines-with-evidence-blocks https://www.nccn.org/guidelines/guidelines-with-evidence-blo...
- mcdonje 3y agoPetition to change the HN title to the article's subtitle.
- quickthrower2 3y agoSeconded
- agentgumshoe 3y agoI think the easiest way to correct for it all is to publish the raw data, something that is trending heavily in the wrong direction. Just publish the data, preferably for free.
- huitzitziltzin 3y ago1. You simply can’t do this in many fields, including wide swathes of health research. It’s a non-starter. 2a. Even if you could, see the OP’s point about research design. You need to understand the methods. 2b. Even if you could part (b), the raw doesn’t help you with publication bias. Your selection of raw data is going to be heavily biased in favor of the data for the papers which managed to be published. It’s not going to contain the data for the results which never saw the light of day due to failure to reach statistical significance.
- NoMoreNicksLeft 3y agoWhat prevents it from becoming the norm for the "statistically insignificant" data to be published alongside the rest? Even point 2a seems weak. That must be true of any research... someone who did not know or did not understand the methodology should fail to make sense of any data provided, and that's on them not on the research itself. Point 1 is really the one I don't understand. Is it medical/health privacy that prevents this? Surely no one expects unanonymized data to be provided. This is about teasing out bad research, not indicting anyone for the falsified sort.
- huitzitziltzin 3y ago>>What prevents it from becoming the norm for the "statistically insignificant" data to be published alongside the rest? The incentives of the researcher. You don’t want to be the guy whose lab produces ten failures and one success, even if that’s the reality of every lab. The guy who doesn’t share the null results will look better than the one who does every time. Deans and chairs only know how to count one thing in tenure decisions - lines on your CV. To put this another way, nothing stops this kind of sharing right now, and yet we have a well known file drawer problem. If the proposed solution involves “oh let’s just completely change the social norms, despite the incentives of the individuals involved,” then it’s a non starter. The kind of sharing being proposed is that kind of idea. >>Point 1 is really the one I don't understand. Is it medical/health privacy that prevents this? Surely no one expects unanonymized data to be provided. This is about teasing out bad research, not indicting anyone for the falsified sort. What you consider “anonymized” is generally not what regulatory agencies will consider anonymized and the best sources for health data at scale are always governments. (Frequently they are the only sources.) The agencies are too risk averse about the potential harms from data releases, but that’s the reality. You can’t share it. You can’t even have it on your own laptop, encrypted or not. Interesting data that lets you answer complicated questions cannot be made public. The more interesting and the more complicated, the harder the underlying data is to access and if you say you want to share it publicly they’ll laugh you out of the room. You’ll never make it through the already excessively burdensome IRB process.
- derbOac 3y agoSure, single high-quality studies can be better than a large number of low-quality studies. But a single study ultimately is just that, regardless of how well-done. There's too much that can be hidden in a single study for me to trust them completely. Even when well intended, there's all sorts of unintentional biases or design problems that lead to potential issues. I've seen too many "well-designed" studies fail on replication for me to trust them too much. For me the gold standard will always be adversarial replication. Even small studies in aggregate can tell us something about what moves effects around. Appeals to single gold standard studies tend to become appeals to or assertions of authority, which is a poor way to do rigorous science.
- HPsquared 3y agoIt all comes back to "you need to actually read the study".
- scythe 3y agoIt's nice to have a meta-analysis. I would usually prefer the meta-analysis for an effect size estimate. But I also generally live by the saying "you never see a clear picture of Bigfoot". If a phenomenon of serious interest is to be considered proven, there should be at least one conclusive high-quality study alongside any meta-analyses. As with many supposed dilemmas, it's nice to have both.
- anonymouskimmer 3y agoReplication is always very important. But for me the "gold standard study" is mechanistical, not statistical. It explains how a process putatively works, not that an effect was seen.
- ZeroGravitas 3y ago[flagged]
- mitthrowaway2 3y agoI'm not going to defend the content, but I am going to defend the format. Is the blog post overly long? Or are we just accustomed to short-and-juicy articles aimed at the declining attention spans of the twitter era? It's about 3700 words, which is about the length of a high school term paper. Substack lists it as a "17 minute read". Firefox says 21-24. Reality has a lot of detail and for something that actually aims to get technical rather than just skimming the surface, it often takes several paragraphs to analyze different facets of a complex question. In a different era, this article would have been considered an appropriate length for a magazine.
- explorer83 3y agoI think intelligent people refusing to acknowledged that science is not a field void of people with imperfect ethics, bias, financial gain incentive or a need to justify their position has immunized me from blindly trusting science. Many years ago a research professor at a highly regarded university basically described to me how funding worked at their university. Essentially the most important criteria was if their research had some promise of results aligning with who was giving the university funding. The second was how well you could navigate the politics of the university and the research funds director. This person expressed deep dissatisfaction that they had promising research in the cancer field that they could not get the lab director to fund because of the first two points.
- piuantiderp 3y agoI mean, I've seen up close talks of how to "adjust the dataset" to fit a certain, pre-ordained result for a medical research paper on children. Wouldn't discard it happens in other fields. But there is freedom in religion in most countries, one can choose what to believe in.
- intelthrow6 3y agoI’ve been reading into autism lately; and a brief thought about “intelligent people” has popped into my head. Intelligence is just a tool, that’s used to achieve the underlying organism’s goals. For example, a crow’s intelligence is used to keep out of danger and to find food. Most neurotypical people use intelligence to gain resources like money, status, and power to fulfill biological imperatives like being part of and respected by their “tribe.” Whereas some autistics will not care about this sort of thing, and be biased more towards the truth, rather than what’s socially most advantageous. For example: Stallman. He has little concern for social appearances and such, so he is free to express something close to “truth,” rather than socially-rewarding “truth.” At that point, it becomes fairly depressing how little research and “knowledge” attainment is actually geared towards “truth,” and not personal enrichment. — And a tangent. I think what separates the outcast autist and the socially well-regarded autist is confidence and self-esteem. If you act and present yourself like a low-status person, you will be treated like it, as people regard your “offness” and associate it with low-status. Whereas if you’re self-assured and act like what you’re doing is innately correct, people will pick up the body language and signs, to think that you’re actually “right” and should accept it. Sort of like those methed-up cult leaders who are utterly insane, but project unwavering self-assuredness, which people naturally gravitate to.
- Joanofarc5859 3y ago[flagged]
- poomer 3y agoGreat article. This part especially rings true to me: "If researchers have an ideological bent, a meta-analytic null may just be an expression of the typical sentiments of researchers". When I was in academia, it was increasingly the case that my peers thought of research less as a way to determine the truth, but just as a method to influence policy and public opinion. If we thought something was 80% likely to be true, there was pressure to "close ranks" and pretend as though it was 100% true, and to avoid publishing anything that contradicted it. It is also well known that papers that support certain "sides" tend to be easier to publish (and in higher ranked journals), plus can yield more media attention. See for example, this fraud in sociology - https://en.wikipedia.org/wiki/When_contact_changes_minds https://en.wikipedia.org/wiki/When_contact_changes_minds. This may be better in the natural sciences, but in social science you should not trust any paper unless you read through and fully understand the methodology. Any non experimental results has so much wiggle room in the modeling methodology that it's easy to generate any result you want. The actual percentage of papers with credible results is very low, much lower than laypeople think.
- getoffmycase 3y agoIt is not better in natural sciences. Alzheimer’s disease treatment based on the amyloid plaque hypothesis being the prime example. Basically researchers closed ranks and doled out grant money disproportionately to the amyloid hypothesis researchers.
- nathan_compton 3y agoThis is sort of like adding momentum to a gradient descent algorithm, though. It is rational congeal support around a plausible hypothesis to see if it pans out rather than pursue all hypotheses (whatever that would even mean) in a desultory fashion. In the case of amyloid plaques there may have been some academic misconduct involved as well. Sometimes I feel like non-scientists have this idea that scientists should be super-rational actors and never be bamboozled or be wrong, but frankly, that isn't realistic. Scientists are fallible, have finite time, and are subject to social trends and pressures like the rest of us. Expecting that they always get it right is unrealistic and not good for science.
- moneywoes 3y agoI was thinking of making a database of only studies that we can “trust” What does HN think? Had a tough time determining the barometer for what is trustable and what isn’t
- nathan_compton 3y agoThere is no simple way to do this. If it were easy to identify trustworthy research we wouldn't be having this discussion. From a slightly broader point of view, I'd argue that the idea that things can be easily or even meaningfully sorted into trustworthy and non-trustworthy is epistemologically facile, at the very least.
- ajhurliman 3y agoIsn’t that effectively just a publishing company?
- some_random 3y agoI'd love that, but determining trust is going to be very hard. For example, where would you put a study that is 100% factually accurate, but yields intentionally misleading statistics for political reasons?
- zachwill 3y agoExamine.com does something similar for health related studies, and I've found it valuable in the past.
- ARandumGuy 3y agoUnfortunately there's a wide range between "obviously trustworthy" and "obviously bullshit". Many bad studies require a lot of domain knowledge to understand why they're bad. Some controversial studies were widely discredited by the rest of the field, but actually turned out to be correct. Many studies are worth doing, but shouldn't be used to draw any meaningful conclusions (such as pilot studies). And finally, there are some studies that seem legitimate, but are actually just lying about their data. And that's before you get into ideological debates and political problems. We like to think scientists are bastions of rationality, but they're just as susceptible to petty infighting as any other group of people. A database of trustworthy studies would be an incredibly valuable tool. Unfortunately, it's a tool that is nigh impossible to actually create.
- photochemsyn 3y agoIf your going to try to construct a meta-analysis of many studies, the first step is going to be reading the materials and methods sections of all the included papers, the goal being to ensure that the data is actually comparable within some (small) margin of error. Disassembling the materials and methods section is the typically the most difficult part of reconstructing a paper's experimental approach as authors often are less than complete, or may refer to other (obscure) papers, or there may be concepts that are so well known in the field that they're taken for granted and not explicitly described. This entails a lot of work and some people are careful and some are sloppy. This is also true for construction of large datasets from individual scientific reports that are then used to produce analytical papers (this is not a 'meta-analysis') based on that dataset, such as in genomic studies, climate studies, etc. The solution here is again to carefully curate the databases and understand their stengths and weaknesses (quality of genomic sequencing matters, for example, and satellite data is not the same as weather ballon data in terms of climate). So if you try to do a meta-analysis of analytical studies based on databases constructed from the results of many individual scientific reports there are all kinds of pitfalls, so it's not surprising that the value of such meta-analysis can be very questionable, unless the authors are extremely careful and clear about their criteria for including or excluding papers from their study. Read enough of them, however, and it becomes easier to identify when the authors are trying to sell their predetermined conclusion via cherry-picking studies that support their position. Unclear methods sections are a good warning sign.
- fwungy 3y agoEveryone knows that 85% of statistics are made up. -Homer Simpson