9 ms·
Bayesian analysis of ego-depletion studies favours null hypothesis
- piratebroadcast 9y agoELI5?
- gbear605 9y agoEgo depletion refers to the idea that self-control or willpower draws upon a limited pool of mental resources that can be used up [1]. Previous studies have found that it is real. This study did a complex statistical analysis and found that they didn't have enough evidence to prove that it is real. [1]: https://en.wikipedia.org/wiki/Ego_depletion https://en.wikipedia.org/wiki/Ego_depletion
- OskarS 9y agoTo give some extra context: this is another study confirming the "replication crisis"[1] in psychology and social psychology (and other sciences as well, but these are the most prominent fields) where longstanding results that were thought to be rock-solid are falling like dominoes. Basically, the science backing, like, half of all TED talks and most episodes of Radiolab is crumbling before our eyes. [1] https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- averagewall 9y agoIt seems like the least competent people to be designing and analysing psychology studies are psychology researchers. Statistics should be the number one skill required. The psychology part of these high-profile studies is usually trivial so that anyone with no psychology background could have thought of it, or at least easily understand it. I don't really see the need for a specialist psychology researcher to do this kind of work. Maybe we need a new field - "studies of interesting everyday phenomena" that's mostly about doing statistics right.
- opportune 9y agoI think you're being down voted because this might sound overly elitist, but to an extent I agree. As someone who considers himself relatively knowledgeable of statistics (coming from ML background) I have a hard time taking a lot of social science research seriously, because quite frankly, I am often left frustrated at the lack of rigor of their methodologies. Sample sizes are often way too small, and even more frequently, unacceptably biased to draw significant scientific conclusions. You also have the publication-bias problem which means that studies may be accepted as "legitimate" due to p-value hacking or because enough people were studying the problem that someone was bound to get a "statistically significant results" even when there was none.
- edilastic 9y agoThe irony is that a lot of statistics has really been developed in psychology and allied disciplines without even realizing. Meta-analysis, for example, really developed into its modern form in psychology, as a way of examining psychotherapy effects (even though the basic idea was around beforehand). Deep learning models too also have their roots in psychological models. They're called neural networks, but the models are more psychological than neurobiological per se, and have connections to other models that are firmly in the realm of psychological models.
- colemannugent 9y agoBasically, there was a theory that exercising self-control would "deplete" ones "self-control reserves". The idea was that exercising your self-control, to say avoid a junk food, would temporarily decrease your ability to resist that temptation again. This paper reports that not only were they unable to verify any of the prior research into this so called "ego depletion" effect, but they found more persuasive evidence suggesting that the theory is false.
- krashidov 9y agoThis is reassuring. Every time someone told me this I figured it had to be bullshit. I think people used it as an excuse to make bad decisions.
- pavement 9y agoGross. It sounds like a total failure to correctly model reality. Like measuring how blue the sky is, based on how much "blue" is left in the universe. The idea they attempt to quantify is improperly described in terms of qualities. e.g. Do colors even work that way? (...and keep your planck unit, M theory, quantum field eigenstate, history-of-the-big-bang, age of the universe remarks away from this hypothetical example, please. You know what I mean when I describe the sky as either blue or not blue in the colloquial sense.)
- tossaway1 9y agoWell of course it sounds ridiculous when you use an absurd analogy. A more reasonable analogy was something like muscle fatigue, or even mental fatigue, and it seems reasonable to believe that willpower can be depleted in a similar way. In fact, there were many published studies consistent with this hypothesis (as the abstract states).
- debacle 9y agoEgo depletion isn't real.
- tinymollusk 9y agoTo me, this is an interesting statement because it highlights the difference between true and useful. For several years, since I read the book Willpower (an intro piece to ego depletion), I've meditated on the idea. I've decided that even if it's not scientifically true, it can be useful to me. Some may see it as self-help tripe ("Eat your frogs in the morning!"), but it's a useful mental model for me to push myself to get stuff done sooner, rather than later, because I imagine myself not having the willpower later. Your choice of word -- real -- was probably casual, but I think a better term would be "scientifically valid". There are many things that aren't replicable that have use to many humans.
- debacle 9y agoJust because the effect isn't real doesn't mean the system can't be beneficial. There's hundreds of millions of people all over the world who benefit from a Abrahamic worldview. The word superstition has negative connotations, but perhaps wrongly so. Also, some aspects of ego depletion are grounded in science. Just because this particular model isn't valid, doesn't mean there's no valid model for ego depletion.
- crawfordcomeaux 9y agoI started using animism yesterday to motivate me to do the dishes, viewing them as the physical embodiment of souls that I, their god, allowed to get moldy/rusty through neglect. I think when we introduce choice/attention into the theory that we construct reality out of beliefs, emotions, culture (thought/behavior patterns), and intentions, it's possible to find all kinds of useful frameworks. Maybe the idea of "scientific validity" needs to be adapted for subjective experiences constructed by us.
- tinymollusk 9y agoHow do you think we could adapt that concept to subjective experiences? As I've gotten older (and learned to manage my mental state better), I've found great benefit from separating "scientifically true" and "useful to lead a good life". I suppose that makes me a hypocrite in the same way that devoutly religious scientists are. But it makes my life more fulfilled, I experience less anxiety, etc. It's an interesting ride, whatever it is.
- ameerkat 9y agoego depletion is the idea that the more you stop yourself from doing things, the harder it gets to stop yourself. It sounds like they looked at a bunch of studies on this effect and concluded that it is not likely by doing some math on those studies.
- shill 9y agoEgo-depletion is mentioned, not always by name, in hundreds productivity and focus related articles and books published in the last 5 years or so. Steve Jobs is often used as an example because he wore the same outfit to work every day, purportedly so he would not have to waste any crucial willpower early in the day on such a trivial task as choosing what to wear.
- phkahler 9y ago>> Steve Jobs is often used as an example because he wore the same outfit to work every day, purportedly so he would not have to waste any crucial willpower early in the day on such a trivial task as choosing what to wear. That was to free up time and free himself from an extra decision. Mental exhaustion is not the same as ego depletion. The later claims that "self control" is a limited resource, but picking your outfit doesn't really require that anyway.
- dbecker 9y agoIt's disappointing to see links to the abstracts on the front page of HN (given the paper itself is behind a paywall). In this case, the find some non-zero effect, but call it zero because the difference was not statistically significant. That's likely a reflection of their small sample size rather than evidence for the null hypothesis as the title suggests. But it's hard to know, or even have an intelligent discussion about it, since the paper itself is behind a paywall.
- grandmczeb 9y agoIt's on sci-hub: http://sci-hub.tw/10.1111/bjso.12236 http://sci-hub.tw/10.1111/bjso.12236
- shaki-dora 9y agoSample size is, of course, one of the factors included in any analysis that gives you a p-value, or any other measure of confidence. In this case, the abstract specifically mentions a "Bayes factor > 25". The Bayes Factor is the likelihood, i. e. the ratio of probabilities for two competing hypotheses. With no information whatsoever, it is 1. A factor of 25 means that the null hypothesis is 25 times as likely to be correct as the theory of ego depletion.
- ghkbrew 9y ago> In this case, the find some non-zero effect, but call it zero because the difference was not statistically significant. Neither the title of the paper nor the abstract say there is zero effect. They accurately and precisely state that the null hypothesis is favored and that the effect is not significantly different than zero. > That's likely a reflection of their small sample size rather than evidence for the null hypothesis as the title suggests. That doesn't seem to be the case. From the introduction: > The Hagger et al. (2010) meta-analysis reported an estimated depletion effect size of d = .62, which we used to conduct a power analysis to establish sample size (n = 33) at 0.80 power. We established n = 35 per condition for our studies, which is slightly larger than the average sample size in the published depletion literature of n = 27 (Lurquin et al., 2016), because the intent of this series of studies was to examine empirically the frequency of null findings in studies similar in size and procedures to those in the published literature. The use of a similar sample size allows for an empirical examination of how frequently the depletion effect occurs when using methods and sample size similar to those in the pre-2013 published literature. So if their samples sizes are a problem then it's a problem that the rest of the literature has too. > But it's hard to know, or even have an intelligent discussion about it, since the paper itself is behind a paywall. Agreed paywalls are unhelpful, but that doesn't mean we can't intelligently discuss what information is available. In addition, I and probably many other people on this forum do have access to many paywalled papers, so exact information is not exactly impossible to obtain.
- tw1010 9y agoIt feels like the fact that this is a bayesian-favoured study is some kind of sign that people (i.e. people that are upvoting, not necessarily the researchers) are actually taking that silly bayesian vs frequentist political war thing seriously. (Which I don't think is advisable.)
- nonbel 9y agoI agree, there is no reason to mention "bayesian" in the title other than using it as a hype term like "blockchain" or "deep learning". Also, the real difference is whether they test a "null hypothesis" vs testing an "actual hypothesis". Like almost all psych, this falls in the first category.
- KirinDave 9y agoYou say this but then you go read a paper with a cruddy t-test and then injure your eyes with the spasmodic rolling that it induces, like I did. It's kinda remarkable how many people don't actually use modern bayesian analysis techniques, which is why it's such a policy war. It's the difference between "older statistical methods uninformed by the glut of modern processing power" and "things which require a computer to do well, and preserve uncertainty."
- closed 9y agoI don't think it's an issue of old versus new methods. Daniel Lakens has an great string of blog posts / publications on how a lot of the same things can go wrong with Bayesian anaylses [1], and how p-values can be viewed as a "poor man's Bayesian updating function" [2]. [1]: http://daniellakens.blogspot.com/2017/03/no-p-values-are-not-to-blame-part-53.html http://daniellakens.blogspot.com/2017/03/no-p-values-are-not... [2]: https://daniellakens.blogspot.com/2015/11/the-relation-between-p-values-and.html https://daniellakens.blogspot.com/2015/11/the-relation-betwe...
- KirinDave 9y agoWhen I do not have a processor easily capable of bayesian mkmv sampling I'll consider using that method. But I do and I guess this means I am rich? Sure.
- andrew_ 9y agoThat's quite a title. "You know I don't speak Spanish."
- colemannugent 9y agoI always get the feeling from reading academic material on psychology that people are trying to apply the scientific method on top of unscientific principles. It feels rather like alchemists trying to explain fire via phlogiston: an attempt to quantify high-level phenomena when you don't understand any of the lower levels. All the data comes from psychology students at a college in the US who either were able to use their participation for a course requirement or given extra credit. The method by which they generated and measured depletion seems adequate, but all the data is self reported based on how the test subjects felt. The statistics gives evidence linking the amount of generated depletion and the reported depletion, but not any insight into what mechanism actually causes depletion. What use is this? Why build a map when we cannot effectively observe the territory?
- debacle 9y agoData collection in soft sciences definitely turns it into a Hardmode, but that doesn't mean all findings are bunk. https://vimeo.com/118188988 https://vimeo.com/118188988
- ajross 9y agoModern experimental science is just garbage on the whole. Psychology has the disadvantage of a very poor theoretical foundation, making a lot of its pronouncements seem hokey and fabricated. But on the whole I don't think the prevalence of poor measurement and (espeically) bad statistics is any worse there than it is in, say, biology. Unreproducible experiments are the rule in modern science, not the exception.
- opportune 9y agoDepends on the discipline. I haven't really heard of frequent non-reproducible results in physics, chemistry, and geology. It seems to be more of a medicine-biology-psychology problem. Personally I think psychology shouldn't even be a separate field from neuroscience. Psychology is, after all, merely an emergent phenomenon from neuroscience, but culture can muddy whether psychological results are truly indicative of how humans think or rather of how a culture has conditioned us to behave in a certain way. I can't remember his name (and I hope someone does and links his wikipedia article), but there was a very well-known and controversial figure within the psychology/psychiatry community who believed that psychology was only slightly better than pseudo-science, and that oftentimes it merely studies behavior and pathologizes others based on our culture rather than science. I just think we don't know enough about neuroscience to really make psychology a very legitimate field of study - it's like trying to perform chemistry without the knowledge of electrons.
- closed 9y agoI'll have to look more closely at the paper, but am very surprised by this result! I've always figured the effect sizes were quite small, and there are good researchers who don't buy its main explanations, but the idea of ego depletion shows up in a lot of areas under different names (e.g. "resource depletion in working memory"). Pretty contentious area of research, so I'm sure this will be some nice fuel for the fire..
- danschumann 9y agoAccording to marines, will power depletion is bs. This is anecdotal, sure, but I believe in momentum. If you have momentum, at some point, you don't want to fail at any will power task, because you are on a winning streak.
- dwaltrip 9y agoSo perhaps the real metric is "motivational momentum"?
- raverbashing 9y agoMore likely you will hit physical exhaustion at some point before being "will depleted"
- mirimir 9y agoIndeed. Also, if you transform what you want, you reduce need for will power.
- ORioN63 9y agoI agree with momentum. I've been using the anology for years. There are only so many points in life where it changes direction. Most of the time is doing what you did the past months/years, whatever is staying home getting drunk or working non-stop. Sure you can change direction, but it's not easy to fight inertia. You gotta apply some force to change direction!
- eeZah7Ux 9y ago
- nurettin 9y agoNot disputing the paper or the scientific method applied here, but how is "bayesian factor" different than p-values?
- robterrin 9y agoThis might answer your question: http://www.stat.columbia.edu/~gelman/research/published/avoiding.pdf http://www.stat.columbia.edu/~gelman/research/published/avoi...
- nurettin 9y agoI don't see "Bayesian factor" in there.
- Asdfbla 9y agoWell, for one thing it doesn't carry that subconscious bias of interpreting something below 0.05 as guaranteed significant as people tend to do with p-values, but is a softer metric that reflects your belief in one or the other hypothesis, given the data. I think that in itself makes it easier to evaluate and reason about. The fact that you can incorporate (and reflect on) a priori assumptions about the likelihood of one or the other model is another advantage imho.
- tmalsburg2 9y agoIn a nutshell: If the p-value is below some conventional threshold (in psychology it's usually 0.05), you can conclude that the null-hypothesis (no effect) is likely wrong and that something else must be the case. However, if p is above the threshold you can't conclude anything, because that means that there is either no effect or that the data was not sufficient to show the effect. So the p-value method (formally known as null-hypothesis significance testing, NHST) can only ever be used to reject the null hypothesis but not to support it. This is where Bayes factors are different. In that approach, you formulate not just one hypothesis but two. Then you test how well both hypotheses are supported by the data. Three conclusions are possible: Hypothesis A receives more support, the alternative hypothesis receives more support, there is not enough evidence to conclude anything with sufficient certainty. Presented this way, the Bayes factor approach appears to be strictly superior to NHST. But there are also some complications that make Bayes factors tricky (related to how you select and formulate the hypotheses) and some statisticians go as far as to say that Bayes factors are obviously non-sense (e.g. Andrew Gelman). The truth is probably that they are neither a panacea nor non-sense.
- meesterdude 9y agoI'm all for replication and ensuring scientific accuracy - but this does not change my perspective of ego-depletion. I have observed it in myself, time and time again. If not it directly, something similar enough. Even at a glucose level, various studies have shown relation between our blood sugar and ability to tolerate frustration and make decisions. So while the accuracy of ego-depletion may be in question, I can still benefit from reasoning it to be true and challenging my emotions and expectations with that. if/when a better model or perspective comes around i'll surely adopt it. But of course, if this news in some way helpful or freeing for you, by all means ride that wave. Success matters more than truth. I've observed that psychology will sometimes split hairs - arguing over whats true and whats truer. Overall this is good - but it's helpful to consider the proliferation of blatant non-truths out there and how far we've come. Sometimes "good enough" is good enough. Fill in the blanks with the facts you can find today and be willing to swap them out for better ones tomorrow.
- dwaltrip 9y ago> I have observed it in myself, time and time again. If not it directly, something similar enough. [Emphasis added by me] This is the crux. I agree with you that there are certainly some sort of effects related to will-power, self-control, focus, energy levels, and so on. This feels obvious enough, based on everyday experience. But it seems we don't yet know how to isolate, measure, or analyze these dynamics in a highly rigorous way. Of course, there is possibility that the everyday experience is actually by caused some different mechanism we don't understand yet.
- BeetleB 9y ago>Even at a glucose level, various studies have shown relation between our blood sugar and ability to tolerate frustration and make decisions. I think it is exactly these kinds of studies that are being questioned. Can they be reproduced?
- watwut 9y agoStay hungry (e.g. On low calorie diet) for few days and then watch yourself more nerveus and making dumber decisions.