6 ms·
Good discussion, but this conclusion: > I don’t know. Maybe a little. is not the right way to interpret the null result (insufficient evidence to reject null
by olooney 2mo ago
Good discussion, but this conclusion:
> I don’t know. Maybe a little.
is not the right way to interpret the null result (insufficient evidence to reject null hypothesis that it has no effect) because the prior for supplements that people are trying to sell you is so low. In the absence of clear, strong evidence, you should assume the the whole thing is just an example of motivated reasoning. People want nootropics to be real, and other people really want to sell you readily available powders by claiming they have nootropic properties. In that environment, they were always going to trying to concoct a similar narrative about some supplement, and it just happened to be creatine. Those efforts were always going to result in a handful of "positive" studies that turn out to be non-reproducible, maybe because of p-hacking, maybe because of publication bias, maybe because of outright fraud. This is what the literature always looks like for stuff that just doesn't work. If it did work - if the effect size was large enough that you could personally detect it in your own life - then the papers would be trying to put error bars around the effect size, not trying (and failing) to barely distinguish it from a placebo.
“If your experiment needs statistics, you ought to have done a better experiment.” - Ernest Rutherford
- taeric 2mo agoHeavily agreed. The supplement industry has such a low level of reliability that it has really soured me on so many practices people have. And I flat out do not understand how some of the supplement stores survive.
- PaulHoule 2mo agoHard sell. I remember the time I went to GNC and they gave me a free sample of the notorious Jack3D which had some kind of amphetamine-like stimulant in it and was associated with young soldiers in the US Army falling down dead.
- Johnny555 2mo ago>And I flat out do not understand how some of the supplement stores survive. I think mostly through placebo effect, people buy a supplement to treat condition X and they think it's getting better, so they keep buying. I mean, just look at the wide variety of homeopathic "cures" that are often so diluted that there might not be any molecules of the active ingredient left. Yet people keep buying it.
- antonvs 2mo agoThey survive because people want to believe. Look at the history: the phrase “snake oil” exists because people actually sold products by that name as a remedy. There were also tonics based on radium. “There's a sucker born every minute.”
- fragmede 2mo agoThe story of how the phrase snake oil came to be is a fascinating one. Clark Stanley, an American, ripped off a Chinese product which was actually a decent anti-inflammatory, and didn't put any snake oil in his snake oil, which is why it didn't work!
- jrm4 2mo agoOkay, but it kind of sounds like you want to nanny-state the writer's language. Their conclusion seems like an absolutely fair interpretation of what they saw, and it also strikes me as dangerous -- perhaps more dangerous -- to suggest "writer genuinely believes, or doesn't believe, a thing, but they should say it differently from their own interpretation because..people are dumb?" Something like that.
- jdiff 2mo agoOne person is not a nation.
- jrm4 2mo agoIt's a metaphor.
- jdiff 2mo agoWhat kind of metaphor? You can be a metaphorical nanny state of something smaller like a household or comment section, I suppose, but even then not with a single statement. The problem with a nanny state is the scope. And people expressing contrary opinions isn't nannying.
- Retr0id 2mo agoIf you're being academically rigorous about it, sure, but I think it's fine as an informal conclusion.
- jdiff 2mo agoEven as an informal conclusion I think it's important to know who you're up against and what their motivations are. Waving that away is how we get people like known, obvious charlatans like RFK Jr in any sort of position of power. I know this is nitpicking word choice but the informal conclusion is "Not likely, and not enough to matter for anything."
- Retr0id 2mo agoPersonally I take creatine for entirely non-cognitive reasons, so it's reassuring to know it's probably not making me any dumber.
- NopIdoN 2mo agoit might even be counteracting the negative effects of your frequent contact with airborne nitrogen
- infinite_spin 2mo agowhat negative effects? N2 is inert under physiological conditions.
- NopIdoN 2mo agonothing it's fine if you buy enough creatine. It's much easier with my proprietary blend though
- infinite_spin 2mo agoYou still haven't specified what negative side effects you're referring to. Nitrogen tends to be inert, so I'm questioning your claim that N2 causes some negative side effect, not your claim that creatine hinders this yet-unnamed side effect
- abirch 2mo agomandatory XKCD https://xkcd.com/882/ https://xkcd.com/882/ "Jellybeans cause acne"
- mrexroad 2mo agoI was visiting a friend’s lab at NASA Ames the other day. I smiled seeing this on the wall there. I sometimes daydream about what it’d be like if our primary education system devoted an hour a week to reading / discussing comics (well, XKCD specifically), or pages form What If…? Maybe it’d kill the magic of it, but I still feel plenty of seeds would get planted along the way for more critical thinking later in life.
- 0xdeadbeefbabe 2mo agoI think you are on to it. The magic would die. It's a recipe for dystopia same as the Agile Manifesto was such a recipe.
- pvaldes 2mo agoJust the green ones. If they found a link, they found a link I'm sure that the comic wants to be sarcastic but is missing the mark here. This is how science works. Trying and failing every color except green, does not mean that the green experiment has an invalid result. The author is not understanding that each experiment is independent. The correct cartoon should show researchers trying green jellybeans 100 times, with 99 negative results and one positive result, and then the press keeping only the positive (Or in this comic, claiming that all jellybeans produce acne). Mocking the scientific method, and showing scientists as lazy people that play video-games is funny, but is just a cartoon, not the reality. People should not take XKCD as god's speech without any critical thinking. This in your workplace says to me "I don't really understand statistics". There is an effect that is statistically rare just by chance, AND is consistently absent within a huge control group of "non-green jellybeans" (the author seems to think that this weakens the result, but in fact is reinforcing it). And it needs further research. Not less, not more.
- dpark 2mo ago
- switchbak 2mo agoWhat I'm getting from all this is that we have an armchair expert explaining details about something they're not an expert in, and we have you reviewing the way they present this information, saying their very weak hedge wasn't strongly negative enough. Based on priors about the industry. Basically, HN doing HN things - having a good deep think about topics outside their realm of expertise, then debating the minutiae. I'd just rather trust someone who has spent their career researching this stuff, thanks (unless that's what you do - in which case please tell us more).
- antonvs 2mo agoIt may be minutiae, but olooney’s point is valid: the evidence doesn’t seem to support saying “maybe a little”. Remove that statement from the conclusion and it’s on the right track, aside from the arbitrary IQ point calculation which is nonsensical.
- dundarious 2mo ago"HN doing HN things" over someone saying "good discussion, however, one point about the conclusion" and having that point be very rational and still open to debate. Is discussing such secondary but still highly relevant aspects verboten? Must every comment here be by a lead researcher in the specific field? Would a biomedical science researcher necessarily have any more expertise on these important secondary aspects? "Trust me, I'm a scientitian". No thanks.
- colechristensen 2mo agoThe top of HN threads is generally nitpicking minutiae which are more likely technically correct than not but also highly irrelevant. It's done by non-experts and drowns out any more valuable discussion. It'll be people arguing about exact language not being someone's preferred level, or overloading the importance and exact definition of a technical term that experts don't really care about, or blowing up some edge case or another.
- dundarious 2mo ago
- somenameforme 2mo agoI think this is quite different because (1) creatine has been proven to be completely harmless for people of all ages absent some rare thing like a kidney disorder and (2) it has a bunch of clearly measurable positive effects. So this is more like trying to determine whether ginseng is good at keeping away colds or whatever. Maybe it does (actually quickly searching this, it definitely does but let's pretend it's as folksy as it sounds for a minute) but even if it doesn't then it's harmless and has a bunch of other effects, so when one takes it - the whole possible cold prophylaxis can just be seen as an added perk.
- ffsm8 2mo agoand the thing people seem to not understand: how exactly do you _expect_ to be able to notice this change? Its something that happens after you take it for a prolonged period of time (>7days) and only has a minor/medium effect. It doesn't make you a genius. I think a better view on it is that it may mitigate some of the adhs symptoms (at least for me). I can keep on track for a significantly longer time since ive started supplementing with it at ~15g/day - and while i dont feel a difference, i can observe it when actually paying attention to what i do / how often i drift off etc.
- deleted 2mo ago[deleted]
- zardo 2mo agoIs there a particular rare kidney disorder that creatine is contraindicated for? Because kidney disease is not rare, something like 14% of US adults have kidney disease.
- adolph 2mo agoMy lay person understanding is that supplemental creatine screws with a common lab test to screen for kidney problems. Let your physician know if you are taking any supplements so they can interpret your lab test results appropriately or order different tests. https://www.kidney.org/kidney-topics/creatinine https://www.kidney.org/kidney-topics/creatinine
- artrockalter 2mo agoThe null is surely different in this case because we know that supplemented creatine goes to the brain and the brain uses it, which is definitely not a claim all supplements can make. The fact that there's a biologically plausible mechanism plus mixed statistical evidence makes "maybe" a perfectly reasonable conclusion.
- Ker102 2mo agoYou are essentially trying to say that most studies regarding supplements are not valid
- Zababa 2mo agoI think the way you interpret the null result is downstream of considering creatine as "a supplement". You can make the null result say anything by changing your prior about creatine or supplements. That's the issue with priors. You also seem to reject the possibility that some things help just a bit. The author has another article in the same vein about "things that can help maybe a bit but the evidence we have doesn't really help detecting small effects" https://dynomight.net/vitamin-d/ https://dynomight.net/vitamin-d/
- rubyfruit 2mo agoPeople have no idea how successful big pharma’s info campaign has been since what I’m increasingly calling the real Web 3.0 (post-influencer social media and arrival of big corporate narratives). According to Reddit, Epsom salt has no effect (doesn’t absorb) - you’re wasting your time by using it. They’ll cite all sorts of real, funded studies that agree and yet always the comment section is filled with anecdotal evidence of the contrary - to which a typical response is “hot water in general is soothing” (ignoring efficacy of magnesium sulfate in ice baths). I digress - creatine is a similar situation. All this confusion at best. It’s also not unlike the health effects of cannabis just prior to going effectively legal - it either causes or cures cancer, or something? We need more research! That’s the real problem! Any time you see this sort of narrative it’s corporate politics and influence at play. By virtue of how hard it is to get any information on creatine, it probably threatens some market or vertical at least (whether or not it “works”). Like A. muscaria mushroom or sassafras, you’ll have to do your own research to really know.
- nemomarx 2mo agoif there's a strong measurable effect to any of these supplements, why can't it be studied? there are companies on both sides, so someone who sells creatine could fund one if they like.
- rubyfruit 2mo agoMagnesium? Its widely known that magnesium is used by the body lol What they “doubt” is that it can be absorbed through the skin. But anyone who takes magnesium tablets can tell you the effect is the same. And it is studied btw - problem is there are countless on both sides.
- JumpCrisscross 2mo ago> Magnesium? Its widely known that magnesium is used by the body lol Who disputed this? > What they “doubt” is that it can be absorbed through the skin The science is helpful. Magnesium sulfate is absorbed less effectively than magnesium chloride. Of course, you're not eating the amount of Epsom salts one dumps in a bath. > And it is studied btw - problem is there are countless on both sides This is why meta analysis exists.