6 ms·

Hey everyone, I'm the author of this guide. It's come full circle -- I posted it a week ago in a "what are you working on?" Ask HN post, someone posted it to Me

Hey everyone, I'm the author of this guide. It's come full circle -- I posted it a week ago in a "what are you working on?" Ask HN post, someone posted it to Metafilter and reddit, and it made its way to Boing Boing and Daily Kos before coming back here. I'm currently working on expanding the guide to book length, and considering options for publication (self-publishing, commercial publishers, etc.). It seems like a broad spectrum of people find it useful. I'd appreciate any suggestions from the HN crowd. (A few folks have already emailed me with tips and suggestions. Thanks!) (Also, I'm sure glad I added that email signup a couple weeks ago)

mey 13y ago
Is there any way to print the entire book as a document? Do you accept donations?
- sudont 13y ago
  http://sphinx-doc.org/latest/builders.html http://sphinx-doc.org/latest/builders.html Sphinx can output Latex for printing to PDF. We'd just need the source. BTW, excellent refresher for statistical screw-ups, I had forgotten a quarter of these (and never learned the rest.)
- capnrefsmmat 13y ago
  Not yet. I intend to get it published so you can buy a printed copy, but that hasn't happened. I don't accept donations, but sign up with your email and I'll let you know when you can pay me for a copy.
  - kisielk 13y ago
    You might want to take a look at Leanpub (https://leanpub.com https://leanpub.com) if you haven't already. Seems like it may suit you.
    - capnrefsmmat 13y ago
      Unfortunately, Leanpub exclusively uses Markdown, which doesn't support things like using a BibTeX bibliography for references. I also don't think there's a way to customize the design of the print book, apart from cover images and such. I'm writing using reStructuredText and Sphinx because they give me a bibliography, an index, full-text search, cross-referencing, custom environments (e.g. boxes for examples, tips, etc.), and all sorts of other cool features. For example, if I needed matplotlib plots, I can embed the code in my documents and Sphinx will generate them. I'd immediately put the book on Leanpub if I could get some of these features. A Leanpub using a customized Sphinx would be incredibly useful for people writing textbooks.
      - FraaJad 13y ago
        this is the exact reason i'm moving away from markdown as my principle note taking tool towards restructuredText.
edwintorok 13y ago
I like how you present various ways that you can make mistakes with statistics. One thing that is missing is like a Summary/Checklist chapter that tells you what you SHOULD do, in a few common scenarios to avoid all the mistakes presented in the previous chapter. I know its not that simple, and it depends a lot on how you are testing and what you're actually trying to achieve, but a few examples wouldn't hurt. For example: I have two sets of measurements, and I want to know: * is there a statistically significant difference between them * if yes, how much is the difference A somewhat simplistic way I'd do that is to do a two-sample t-test for question #1, and to compute statistics for the difference between the samples (mean, median, confidence intervals), but doing just that I might've already committed some of the mistakes that your site warns about, for example I completely disregarded the power of the test. FWIW I like parts of this book on statistics which focuses on statistics in the domain of computer systems / network, although it is rather too long: http://perfeval.epfl.ch/ http://perfeval.epfl.ch/
- capnrefsmmat 13y ago
  My draft (twice as long as what's online) has a feature like this at the end of each chapter. It needs more work to be useful, but it goes a long way to making the advice in the book actionable.
craigyk 13y ago
As a scientist I think you are addressing a very important problem with this book. I've taken two statistics classes, one graduate level, and even I am plagued with doubt as to wether the statistics I've used have all been applied and interpreted "correctly". That said, I think the recent spate of "a majority of science publications are wrong" stories is incredible hyperbole. Is it the raw data that is wrong (fabricated)? The main conclusions? One or two minor side points? What if the broad strokes are right but the statistics are sloppy? People also need to realize what while the Discussion and Conclusion section of publications may often read like statements of truth, they're usually just a huge lump of spinoff hypotheses in prose form. Despite my frequent frustrations with the ways science could be better, the overall arrow of progress points in the right direction. Science isn't a process where the goal is to ensure that 100% of what gets published is correct, but whereby previous assertions can be refuted and corrected. Edit: To be more specific, I think the statement in your Introduction is overly critical: "The problem isn’t fraud but poor statistical education – poor enough that some scientists conclude that most published research findings are probably false". I would change it to say: "conclude that most published research findings contain (significant) errors", or something along those lines.
- ronaldx 13y ago
  "a majority of science publications are wrong" As a scientist, I think this is probably correct. In my experience, a great majority of publications draw improper statistical conclusions, and I believe many of these are wrong in substance. "What if the broad strokes are right but the statistics are sloppy?" Publishing statements as statements of truth when they are improperly or falsely backed up would be better described as politics than science.
  - craigyk 13y ago
    I take the view that "a majority of science publications are wrong" is a purposefully misleading and sensationalistic take, even though it may be technically true. IMO, only the leading-edge of known science should factor into such studies, and I think that is probably not "mostly wrong". After all, even if there has only been 1 rock-solid publication in favor of a round Earth that is preceded by 99 publications in support of a flat-Earth, I would't call that field 99% wrong.
jrochkind1 13y ago
awesome, thanks for writing this, and for whoever posted it here.
chris_va 13y ago
Now if you could just send a copy of the book to every newspaper in the US... I would donate to that goal.
chris_va 13y ago
I've been reading this on/off for the last day. One random UX suggestion: Don't use black for your text, use something close (e.g. #333). There are a million other UX tips that you can probably get from a real expert, but the black one I noticed.
- chris_va 13y ago
  And another suggestion: http://en.wikipedia.org/wiki/Simpson's_paradox http://en.wikipedia.org/wiki/Simpson's_paradox ... might be nice to add to your list.

mey 13y ago

Is there any way to print the entire book as a document? Do you accept donations?

sudont 13y ago
http://sphinx-doc.org/latest/builders.html http://sphinx-doc.org/latest/builders.html Sphinx can output Latex for printing to PDF. We'd just need the source. BTW, excellent refresher for statistical screw-ups, I had forgotten a quarter of these (and never learned the rest.)
capnrefsmmat 13y ago
Not yet. I intend to get it published so you can buy a printed copy, but that hasn't happened. I don't accept donations, but sign up with your email and I'll let you know when you can pay me for a copy.
- kisielk 13y ago
  You might want to take a look at Leanpub (https://leanpub.com https://leanpub.com) if you haven't already. Seems like it may suit you.
  - capnrefsmmat 13y ago
    Unfortunately, Leanpub exclusively uses Markdown, which doesn't support things like using a BibTeX bibliography for references. I also don't think there's a way to customize the design of the print book, apart from cover images and such. I'm writing using reStructuredText and Sphinx because they give me a bibliography, an index, full-text search, cross-referencing, custom environments (e.g. boxes for examples, tips, etc.), and all sorts of other cool features. For example, if I needed matplotlib plots, I can embed the code in my documents and Sphinx will generate them. I'd immediately put the book on Leanpub if I could get some of these features. A Leanpub using a customized Sphinx would be incredibly useful for people writing textbooks.
    - FraaJad 13y ago
      this is the exact reason i'm moving away from markdown as my principle note taking tool towards restructuredText.

edwintorok 13y ago

I like how you present various ways that you can make mistakes with statistics. One thing that is missing is like a Summary/Checklist chapter that tells you what you SHOULD do, in a few common scenarios to avoid all the mistakes presented in the previous chapter. I know its not that simple, and it depends a lot on how you are testing and what you're actually trying to achieve, but a few examples wouldn't hurt. For example: I have two sets of measurements, and I want to know: * is there a statistically significant difference between them * if yes, how much is the difference A somewhat simplistic way I'd do that is to do a two-sample t-test for question #1, and to compute statistics for the difference between the samples (mean, median, confidence intervals), but doing just that I might've already committed some of the mistakes that your site warns about, for example I completely disregarded the power of the test. FWIW I like parts of this book on statistics which focuses on statistics in the domain of computer systems / network, although it is rather too long: http://perfeval.epfl.ch/ http://perfeval.epfl.ch/

capnrefsmmat 13y ago
My draft (twice as long as what's online) has a feature like this at the end of each chapter. It needs more work to be useful, but it goes a long way to making the advice in the book actionable.

craigyk 13y ago

As a scientist I think you are addressing a very important problem with this book. I've taken two statistics classes, one graduate level, and even I am plagued with doubt as to wether the statistics I've used have all been applied and interpreted "correctly". That said, I think the recent spate of "a majority of science publications are wrong" stories is incredible hyperbole. Is it the raw data that is wrong (fabricated)? The main conclusions? One or two minor side points? What if the broad strokes are right but the statistics are sloppy? People also need to realize what while the Discussion and Conclusion section of publications may often read like statements of truth, they're usually just a huge lump of spinoff hypotheses in prose form. Despite my frequent frustrations with the ways science could be better, the overall arrow of progress points in the right direction. Science isn't a process where the goal is to ensure that 100% of what gets published is correct, but whereby previous assertions can be refuted and corrected. Edit: To be more specific, I think the statement in your Introduction is overly critical: "The problem isn’t fraud but poor statistical education – poor enough that some scientists conclude that most published research findings are probably false". I would change it to say: "conclude that most published research findings contain (significant) errors", or something along those lines.

ronaldx 13y ago
"a majority of science publications are wrong" As a scientist, I think this is probably correct. In my experience, a great majority of publications draw improper statistical conclusions, and I believe many of these are wrong in substance. "What if the broad strokes are right but the statistics are sloppy?" Publishing statements as statements of truth when they are improperly or falsely backed up would be better described as politics than science.
- craigyk 13y ago
  I take the view that "a majority of science publications are wrong" is a purposefully misleading and sensationalistic take, even though it may be technically true. IMO, only the leading-edge of known science should factor into such studies, and I think that is probably not "mostly wrong". After all, even if there has only been 1 rock-solid publication in favor of a round Earth that is preceded by 99 publications in support of a flat-Earth, I would't call that field 99% wrong.

jrochkind1 13y ago

awesome, thanks for writing this, and for whoever posted it here.

chris_va 13y ago

Now if you could just send a copy of the book to every newspaper in the US... I would donate to that goal.

chris_va 13y ago

I've been reading this on/off for the last day. One random UX suggestion: Don't use black for your text, use something close (e.g. #333). There are a million other UX tips that you can probably get from a real expert, but the black one I noticed.

chris_va 13y ago
And another suggestion: http://en.wikipedia.org/wiki/Simpson's_paradox http://en.wikipedia.org/wiki/Simpson's_paradox ... might be nice to add to your list.