3 ms·

Hey! I'm the author and I'd be happy to answer any questions you might have. Confidence.js is based on the A/B testing code that we use at sendwithus. A/B tes

Hey! I'm the author and I'd be happy to answer any questions you might have. Confidence.js is based on the A/B testing code that we use at sendwithus. A/B testing math is hard and we've worked really hard on making this great. Let us know what you think!

sharnett 13y ago
A nit pick: I believe your 'readable' interpretation of the confidence interval isn't quite right: http://en.wikipedia.org/wiki/Confidence_interval#Meaning_and_interpretation http://en.wikipedia.org/wiki/Confidence_interval#Meaning_and...
- jessicaraygun 13y ago
  Hi sharnett, Thanks for bringing that to my attention - I agree that our explanation is a little more convoluted than it needs to be. I'll update that later today to be more "readable" ;)
mattmanser 13y ago
I think making it an object has made the code unnecessarily complicated. I think you could almost 1/2 code with a simple function. This is where I personally think javascript programming is going wrong at the moment, so much boiler plate. You're also programming a bit too defensively. Ask yourself, do you honestly need all those checks? What would actually happen if you got rid of the checks? I think your code would be clearer without them and the programmer would still realize he'd made a mistake and passed in nulls.
- jessicaraygun 13y ago
  Thanks for the feedback! Confidence.js comes from how we operate on A/B tests at sendwithus - we think of them as variants associated with results. While the code could definitely boil down to a few simple math functions, we find objects to be a good way to organize our data models.
  - hiisi 13y ago
    Also, "getRequiredSampleSize" is called twice for each variant - in the "getResult"'s loop and then inside "hasEnoughData". Same thing for "getVariant". Would you welcome a pull request? Your project is great, I'd totally use at work. Also, it was to learn the math behind that. Thanks!
    - jessicaraygun 13y ago
      Glad you like it, happy to have you contribute :)
teleclimber 13y ago
Thank-you for sharing this. Can you recommend a resource that explains the Math for people who slept through statistics class? I actually need to brush up on this topic for a new project so anything you can recommend would be greatly appreciated. Thanks!
- jessicaraygun 13y ago
  Stats class R sleepy... Real world applications like A/B testing are definitely more interesting. We did a lot of digging and there aren't a lot of resources that are easy to digest - but this one really stood out to us: http://visualwebsiteoptimizer.com/split-testing-blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/ http://visualwebsiteoptimizer.com/split-testing-blog/what-yo... I'd love to write a summary of everything I've learned - maybe a blog post in the near future!
- tagawa 13y ago
  This slide deck worked for me, with the explanations in understandable chunks: http://elem.com/~btilly/effective-ab-testing/ http://elem.com/~btilly/effective-ab-testing/

sharnett 13y ago

A nit pick: I believe your 'readable' interpretation of the confidence interval isn't quite right: http://en.wikipedia.org/wiki/Confidence_interval#Meaning_and_interpretation http://en.wikipedia.org/wiki/Confidence_interval#Meaning_and...

jessicaraygun 13y ago
Hi sharnett, Thanks for bringing that to my attention - I agree that our explanation is a little more convoluted than it needs to be. I'll update that later today to be more "readable" ;)

mattmanser 13y ago

I think making it an object has made the code unnecessarily complicated. I think you could almost 1/2 code with a simple function. This is where I personally think javascript programming is going wrong at the moment, so much boiler plate. You're also programming a bit too defensively. Ask yourself, do you honestly need all those checks? What would actually happen if you got rid of the checks? I think your code would be clearer without them and the programmer would still realize he'd made a mistake and passed in nulls.

jessicaraygun 13y ago
Thanks for the feedback! Confidence.js comes from how we operate on A/B tests at sendwithus - we think of them as variants associated with results. While the code could definitely boil down to a few simple math functions, we find objects to be a good way to organize our data models.
- hiisi 13y ago
  Also, "getRequiredSampleSize" is called twice for each variant - in the "getResult"'s loop and then inside "hasEnoughData". Same thing for "getVariant". Would you welcome a pull request? Your project is great, I'd totally use at work. Also, it was to learn the math behind that. Thanks!
  - jessicaraygun 13y ago
    Glad you like it, happy to have you contribute :)

teleclimber 13y ago

Thank-you for sharing this. Can you recommend a resource that explains the Math for people who slept through statistics class? I actually need to brush up on this topic for a new project so anything you can recommend would be greatly appreciated. Thanks!

jessicaraygun 13y ago
Stats class R sleepy... Real world applications like A/B testing are definitely more interesting. We did a lot of digging and there aren't a lot of resources that are easy to digest - but this one really stood out to us: http://visualwebsiteoptimizer.com/split-testing-blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/ http://visualwebsiteoptimizer.com/split-testing-blog/what-yo... I'd love to write a summary of everything I've learned - maybe a blog post in the near future!
tagawa 13y ago
This slide deck worked for me, with the explanations in understandable chunks: http://elem.com/~btilly/effective-ab-testing/ http://elem.com/~btilly/effective-ab-testing/