jrdorn

5 ms·

1. 1.▲
   by jrdorn 5y ago

   Most interactions between tests are really minor and indistinguishable from random noise. Those ones aren't worth worrying about. There are two ways to deal with the bigger interaction effects. First is to predict which experiments wil
2. 2.▲
   by jrdorn 5y ago

   We're big fans of PostHog ourselves. Any tool that encourages people to use data to make product decisions is a huge win in our book.
3. 3.▲
   by jrdorn 5y ago

   We are planning on an open core model similar to PostHog. So we would eventually monetize from cloud hosting and license keys for enterprise features. We wanted to use MIT to make it as easy to adopt as possible for all companies.
4. 4.▲
   by jrdorn 5y ago

   Thanks! Let us know if you run into any issues integrating and setting up your first feature.
5. 5.▲
   by jrdorn 5y ago

   Congrats on the launch! Product looks great and there's so much room for improvement when it comes to onboarding.
6. 6.▲
   by jrdorn 5y ago

   Thanks for the suggestion. We added MySQL support so we can work with Matomo now, but we're definitely lacking in documentation and tutorials. There's a similar issue with people using GA4, Snowplow, Segment, and anything else wit
7. 7.▲
   by jrdorn 5y ago

   Yeah, Statsig is building a really cool product. There's definitely a gap in GrowthBook right now, like you mentioned, between pure feature flags and experiments. We're working on some light-weight event tracking features to try
8. 8.▲
   by jrdorn 5y ago

   Tools like Flagr typically function as a microservice that handles all of the feature evaluation. That means you need a network request every time you want to get a feature's value. With GrowthBook, there's a single cacheable JSON
9. 9.▲
   by jrdorn 5y ago

   Yeah, that's our exact experience as well. I was super surprised to find out how much of a graveyard open source A/B testing was. Intuit (Wasabi), Facebook (PlanOut), and SeatGuru (Sixpack) open sourced their internal platforms ye
10. 10.▲
    by jrdorn 5y ago

    Thanks! If you can recreate the installation steps in the Dockerfile ( https://github.com/growthbook/growthbook/blob/main/Dockerfil... ), it should be possible to run it yourself on a shared host. Basicall
11. 11.▲
    by jrdorn 5y ago

    Thanks! It's been a crazy 6 months, feels like years ago
12. 12.▲
    by jrdorn 5y ago

    Ax is for automated optimization using machine learning. You define the parameters and optimization function and it decides the variations, traffic splitting, and everything else for you. Growth Book is for hypothesis testing. It let's
13. 13.▲
    by jrdorn 5y ago

    Yep, should be possible once MySQL/MariaDB is done.
14. 14.▲
    by jrdorn 5y ago

    Do you mind explaining that a little more? As it's currently designed, a company could use DBT to model their raw data into dedicated metric tables and then Growth Book sits on top of those with a really simple SQL query and some setti
15. 15.▲
    by jrdorn 5y ago

    Thanks for the comment and for your work on Alephbet! Open source A/B testing is a graveyard of abandoned projects, so it's always great to see more people actively working in this space. Georgi at Analytics Toolkit definitely kno
16. 16.▲
    by jrdorn 5y ago

    Yes. As long as the variation assignment data and success metrics are in a supported data source (SQL, GA, or Mixpanel currently), it can be queried and analyzed in Growth Book.
17. 17.▲
    by jrdorn 5y ago

    We plan to add MySQL/MariaDB support soon which should let you use Matomo data as long as you have raw SQL access. For cloud-hosted Matomo, we would have to use the reporting API, which is doable but not as good since there's no w
18. 18.▲
    by jrdorn 5y ago

    We don't have native mobile SDKs yet, but it's something we want to support in the future. Mobile is a little tricky since you either need to do a new release every time you want to start/stop a test or use remote config and
19. 19.▲
    by jrdorn 5y ago

    1. GA is very limited as a data source because of sampling and the fact that they don't expose variance. So if using GA, we only support simple binomial metrics, count data (assuming Poisson distribution), and duration data (assuming e
20. 20.▲
    by jrdorn 5y ago

    Hi, thanks for the questions! 1. We don't store any raw user data. We pull things like mean and standard deviation from data sources, run the statistics, and store the result. 2. We use a Bayesian statistics engine which is much more i
21. 21.▲
    by jrdorn 5y ago

    That's on our roadmap. We originally built the tool as a multi-tenant hosted platform so storing configs in a database made the most sense initially. For self hosting, we want to support defining db connections and metrics using yml.
22. 22.▲
    by jrdorn 5y ago

    We're definitely interested in supporting some personalization use cases in the future using contextual bandits.
23. 23.▲
    by jrdorn 5y ago

    Hi! One of the authors here. We're using MongoDB to store caches A/B test results (among other things), which are deeply nested JSON objects. MongoDB let us develop features really quickly so its been a great choice so far for us.
24. 24.▲
    by jrdorn 5y ago

    Hey HN! We noticed tons of companies build their own A/B testing platforms from scratch and thought that was kind of crazy so we made Growth Book. Let us know what you think.
25. 25.▲
    Show HN: Open-source A/B testing platform(github.com)

    11 points by jrdorn 5y ago|2 comments