Okay, I have read the fine blog post (which took a while to load). The article leads off by defining nootropics as "substances which purportedly improve mental

Okay, I have read the fine blog post (which took a while to load). The article leads off by defining nootropics as "substances which purportedly improve mental functioning with relatively few side effects when used responsibility." The blog post author mentions a few small-n, limited duration studies, and then describes his voluntary-response survey methodology: "I asked people in the two largest online nootropics communities (that I know of), Reddit’s r/nootropics and Longecity’s Brain Health forum, to take a survey describing their use of 31 different substances. I got 162 responses." "Respondents were asked to rate their subjective experiences of different nootropics on a scale of zero (completely useless, did nothing) to ten (life-changing). Comments on Reddit suggest several people misunderstood the scale, but not much I can do about that at this point." So all the data here are voluntary-response data, with the known problem of response bias[1] and other defects about which I have an entire FAQ[2] I post on HN from time to time. Worse than that, all the data are self-report data,[3] which also have plenty of problems, even when gathered in a comprehensive survey with a representative sample of the general population, because of the risks of incorrect self-reports unless checked by other data sources that are objective.[4] The blog post author then gives a lengthy description, with pretty colored charts, of his hand-waving statistical analysis of these dubious voluntary reponse, self-report data. It would not be wise (that is, it would not be a sign of improved mental functioning) to take these conclusions seriously as a description of how nootropics work, or indeed if they work at all. The author would do well to study fundamental principles of statistics by reading the online articles "Advice to Mathematics Teachers on Evaluating Introductory Statistics Textbooks"[5] and "The Introductory Statistics Course: A Ptolemaic Curriculum?"[6] to wrap his mind around how strictly necessary it is to have a valid data-gathering plan before engaging in any statistical manipulation of the data. Garbage in, garbage out. More background on experiment design to think about while evaluating the blog post kindly submitted here can be found in the essay "Warning Signs in Experimental Design and Interpretation"[7] by Peter Norvig, LISP hacker and director of research at Google, on how to interpret scientific research. What might big-time convince me to try out a nootropic substance would be an objective demonstration that someone with typical abilities (say, someone who goes into online discussion with a known propensity to believe what's on Reddit without checking it) can become a person with thoughtful insight (say, someone like Peter Norvig) at adult age simply by ingesting the nootropic. On my part, I like to improve mental functioning with relatively few side effects as much as anybody here, and one way I attempt to do that in middle age is by reading thoughtful articles by careful scholars and then discussing those among the learned participants on Hacker News. I'd appreciate comments from any of you about how one could know, objectively and verifiably, that a nootropic substance really does people good (even if they don't feel that themselves). I especially invite comments from participants here who have a professional understanding of statistical studies in clinical trials of new medicines. [1] https://en.wikipedia.org/wiki/Response_bias https://en.wikipedia.org/wiki/Response_bias [2] https://news.ycombinator.com/item?id=2322237 https://news.ycombinator.com/item?id=2322237 [3] https://en.wikipedia.org/wiki/Self-report_study https://en.wikipedia.org/wiki/Self-report_study [4] http://www.ncbi.nlm.nih.gov/pubmed/15528061 http://www.ncbi.nlm.nih.gov/pubmed/15528061 http://www.wiley.com/legacy/wileychi/barker/supp/excerpt.pdf http://www.wiley.com/legacy/wileychi/barker/supp/excerpt.pdf [5] http://statland.org/MyPapers/MAAFIXED.PDF http://statland.org/MyPapers/MAAFIXED.PDF [6] http://escholarship.org/uc/item/6hb3k0nz http://escholarship.org/uc/item/6hb3k0nz [7] http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html