6 ms·

I Replaced Baremetrics and ChartMogul with Rake

99 points by ezekg 6y ago

devmunchies 6y ago
no visualizations? you just producing raw data? why not just use SQL or excel?
- ezekg 6y ago
  Because my data is stored in Stripe. I don’t need visualizations. I print all the data in my terminal [0]. [0]: https://twitter.com/ezekgabrielse/status/1345157874356465664 https://twitter.com/ezekgabrielse/status/1345157874356465664
switchstance 6y ago
We always just used Profitwell for free.
- toomuchtodo 6y ago
  +1 for Profitwell
- burlesona 6y ago
  ProfitWell is fantastic, can’t +1 this enough
burlesona 6y ago
In case anyone is wondering: Rake is “Ruby Make,” which is just a utility for defining and running Ruby scripts.
amerine 6y ago
Thanks, Jim!
tschellenbach 6y ago
I never found the right solution (ChartMogul has issues with when they recognize revenue. It's based on payment instead of the period the invoice/subscription is for). Anyone have any tips to share about what they use?
- ezekg 6y ago
  That’s one of the many reasons I wrote this Rake task!
- cj 6y ago
  Baremetrics revenue is based on the subscription period rather than payment date. (A few years ago they made the switch to that model rather than by payment date)
- molsongolden 6y ago
  I'm assuming that when you say "instead of the period the invoice/subscription is for" you mean that the invoice isn't being split and recognized across a multi-month service period. Most products will call this Revenue Recognition. Some basic SaaS analytics/metrics products don't offer this functionality but it can be a hassle even with the ones that do. The accuracy will always depend on clean invoicing hygiene when the invoices are created. e.g. if you create a subscription in Stripe and have modeled it properly using products, prices, etc... then the metadata for revenue recognition will be available but if you generate an invoice and write "Annual subscription 1/1/21 - 12/31/21" as the line item description then every tool will recognize that amount in full on the invoice date. Revenue recognition might be a separate add-on for ChartMogul but I think they can do it. Chargebee and Recurly also do subscription management and maintain revenue recognition reports. Stripe semi-recently added revenue recognition to the billing dashboard. Some companies build a waterfall schedule in Excel then manually record revenue every month.
  - molsongolden 6y ago
    This sounds a little abrupt on a reread! Feel free to reach out (email in profile) if you'd like to quick chat about the reports you have now and the reports you'd like to see.
messutied 6y ago
Is there an open source alternative to ChartMogul? I feel like it makes sense, at least I'd like to avoid having to share financial information with a 3rd party, would much rather use some self-hosted open source solution.
- molsongolden 6y ago
  Depending on how technical you are, similar visualizations can be produced in Google Sheets/Excel or Metabase. The issues are: 1) exporting/importing the data (manual CSV, via API, sourced from an existing DB, etc...) then 2) knowing which questions to ask and how to visualize answers to those questions You could also code a custom dashboard using something like D3.js
  - michalbugno 6y ago
    A bit of self promotion, though not opensource. If you store your data in sheets, Probe Sheets plugin [1] calculates basic metrics based on some basic data: customer id, mrr, start and end time of subscription. We do not store the data you send (there is no way I can prove that to you though, but you can always obfuscate it somehow) [1] https://workspace.google.com/marketplace/app/probe/1085080411800 https://workspace.google.com/marketplace/app/probe/108508041...
chris_wot 6y ago
I keep hearing that people aren't getting everything they want out of Baremetrics. What is it that they have a problem with?
- ezekg 6y ago
  I mean, I wrote a blog post about it... :)
  - chris_wot 6y ago
    You wrote a blog post that implemented functionality that you liked and also functionality that baremetrics already implemented. You weren't hugely detailed in what in particular you found lacking that actually caused you to roll your own product.
    - ezekg 6y ago
      Yeah, you’re right. I of course wanted to save the $50/mo (or was it $99?) if I could. When I moved to having a free tier, my conversion rate metric got a little wonky. I don’t remember them handling that well. I also wanted to know my time-to-convert and they don’t show that metric, among other metrics. Also, I remember their churn metric was off sometimes. There were also some other things, like their messaging and recover features not working as you’d expect when it came to trials, which was my main need at the time. I ended up doing so much myself that they weren’t worth it to keep. Then I tried ChartMogul, had the same lack of visibility into my metrics, and ended up going all in and writing this [0]. [0]: https://twitter.com/ezekgabrielse/status/1345157874356465664 https://twitter.com/ezekgabrielse/status/1345157874356465664
- cj 6y ago
  > What is it that they have a problem with? We struggle a lot with “fake churn” in baremetrics. By that, I mean any time a customer on a credit card subscription upgrades to a “pay by invoice” tier, it shows as a churn of the credit card subscription and as new revenue for the invoice subscription. This makes churn wildly inaccurate and we spend a couple hours per month adjusting the numbers to get accurate data. FWIW profitwell suffers the same issue. We’ve looked but haven’t found a great solution for metrics when stripe isn’t the sole source of customer payments. Edit: But all things considered, we love baremetrics, just not their churn calculations (and by proxy, LTV is also broken, and any other metric that uses churn)
BummerCloud 6y ago
I'd be cautious multiplying your MRR x 12 to establish ARR. Things like seasonality or promotions could overestimate your ARR by using inflated subscription numbers. Additionally, if customers are not contractually obligated to stick around for a year, that calculation may not account for month-to-month churn.
- ezekg 6y ago
  I’m pretty comfortable with MRRx12 for annual run rate, which is pretty normal from what I’ve seen. But I do have other unmentioned revenue forecasting metrics based on current/past growth/churn rate, which I pay more attention to than my ARR. I don’t base a lot on this number either way.
- encoderer 6y ago
  ARR is always MRRx12. You're talking about a revenue model.
  - jasongill 6y ago
    I think the confusion here stems from the terms "annual run rate" and "annual recurring revenue" sharing the same acronym
    - encoderer 6y ago
      I admit it’s confusing, but those are exactly the same thing. Have the same formulas. https://www-priceintelligently-com.cdn.ampproject.org/i/s/www.priceintelligently.com/hs-fs/hubfs/Calculating%20True%20MRR.jpg?width=1511&name=Calculating%20True%20MRR.jpg https://www-priceintelligently-com.cdn.ampproject.org/i/s/ww... https://www-profitwell-com.cdn.ampproject.org/i/s/www.profitwell.com/hs-fs/hubfs/blob-256.png?width=1200&name=blob-256.png https://www-profitwell-com.cdn.ampproject.org/i/s/www.profit...
  - thih9 6y ago
    > ARR is always MRRx12. Are you saying that ARR is meaningless for heavily seasonal businesses?
encoderer 6y ago
This is how side projects start :)
- ezekg 6y ago
  Indeed. Maybe I can run with this and eventually quit my full time job. :P
skaber 6y ago
Your Ruby ActiveRecord query is so under optimized, you are loading a ton of records that instantiate objects and then apply a filter on this huge array. I'm pretty sure you could make it 100x faster.
- hiharryhere 6y ago
  I don't think those are ActiveRecord queries, they look like calls to the Stripe api via the Stripe gem. It's possible those filters like 'deleted?' are not able to be applied when querying the API.
  - ezekg 6y ago
    Correct. Those are Stripe API calls made through their client library.
- jon-wood 6y ago
  I think someone else has pointed out that they're not ActiveRecord queries, but even if they were, this is a script designed to be run manually a few times a week, which takes maybe a minute. You'd need significantly more than 100x performance increase for any time at all put into optimising queries to pay for themselves.
michalbugno 6y ago
Perhaps I am a bit biased because I create analytics software fos SaaS businesses. This is a nice writeup, could be an intro of tech people to business metrics you mention. I also code Ruby so +1 :-) I would be cautious about rolling your own implementation like this though. 1. This is not your core business. Eventually you want to start segmenting things by verticals, geography etc. You will want to visualise them. It takes time, which you want to spend building your product. 2. There are many important details in how you calculate certain metrics. Either you dive into the topic and again waste time, or you start having metrics which are custom, not really comparable with benchmarks or just plain wrong. Maybe you just spent 1hr on this and that’s ok. Just make sure you spend your time on your product domain, because that’s what’s probably gonna make you happier (unless you pivot into saas metrics world!)

99 points by ezekg 6y ago

devmunchies 6y ago

no visualizations? you just producing raw data? why not just use SQL or excel?

ezekg 6y ago
Because my data is stored in Stripe. I don’t need visualizations. I print all the data in my terminal [0]. [0]: https://twitter.com/ezekgabrielse/status/1345157874356465664 https://twitter.com/ezekgabrielse/status/1345157874356465664

switchstance 6y ago

We always just used Profitwell for free.

toomuchtodo 6y ago
+1 for Profitwell
burlesona 6y ago
ProfitWell is fantastic, can’t +1 this enough

burlesona 6y ago

In case anyone is wondering: Rake is “Ruby Make,” which is just a utility for defining and running Ruby scripts.

amerine 6y ago

Thanks, Jim!

tschellenbach 6y ago

I never found the right solution (ChartMogul has issues with when they recognize revenue. It's based on payment instead of the period the invoice/subscription is for). Anyone have any tips to share about what they use?

ezekg 6y ago
That’s one of the many reasons I wrote this Rake task!
cj 6y ago
Baremetrics revenue is based on the subscription period rather than payment date. (A few years ago they made the switch to that model rather than by payment date)
molsongolden 6y ago
I'm assuming that when you say "instead of the period the invoice/subscription is for" you mean that the invoice isn't being split and recognized across a multi-month service period. Most products will call this Revenue Recognition. Some basic SaaS analytics/metrics products don't offer this functionality but it can be a hassle even with the ones that do. The accuracy will always depend on clean invoicing hygiene when the invoices are created. e.g. if you create a subscription in Stripe and have modeled it properly using products, prices, etc... then the metadata for revenue recognition will be available but if you generate an invoice and write "Annual subscription 1/1/21 - 12/31/21" as the line item description then every tool will recognize that amount in full on the invoice date. Revenue recognition might be a separate add-on for ChartMogul but I think they can do it. Chargebee and Recurly also do subscription management and maintain revenue recognition reports. Stripe semi-recently added revenue recognition to the billing dashboard. Some companies build a waterfall schedule in Excel then manually record revenue every month.
- molsongolden 6y ago
  This sounds a little abrupt on a reread! Feel free to reach out (email in profile) if you'd like to quick chat about the reports you have now and the reports you'd like to see.

messutied 6y ago

Is there an open source alternative to ChartMogul? I feel like it makes sense, at least I'd like to avoid having to share financial information with a 3rd party, would much rather use some self-hosted open source solution.

molsongolden 6y ago
Depending on how technical you are, similar visualizations can be produced in Google Sheets/Excel or Metabase. The issues are: 1) exporting/importing the data (manual CSV, via API, sourced from an existing DB, etc...) then 2) knowing which questions to ask and how to visualize answers to those questions You could also code a custom dashboard using something like D3.js
- michalbugno 6y ago
  A bit of self promotion, though not opensource. If you store your data in sheets, Probe Sheets plugin [1] calculates basic metrics based on some basic data: customer id, mrr, start and end time of subscription. We do not store the data you send (there is no way I can prove that to you though, but you can always obfuscate it somehow) [1] https://workspace.google.com/marketplace/app/probe/1085080411800 https://workspace.google.com/marketplace/app/probe/108508041...

chris_wot 6y ago

I keep hearing that people aren't getting everything they want out of Baremetrics. What is it that they have a problem with?

ezekg 6y ago
I mean, I wrote a blog post about it... :)
- chris_wot 6y ago
  You wrote a blog post that implemented functionality that you liked and also functionality that baremetrics already implemented. You weren't hugely detailed in what in particular you found lacking that actually caused you to roll your own product.
  - ezekg 6y ago
    Yeah, you’re right. I of course wanted to save the $50/mo (or was it $99?) if I could. When I moved to having a free tier, my conversion rate metric got a little wonky. I don’t remember them handling that well. I also wanted to know my time-to-convert and they don’t show that metric, among other metrics. Also, I remember their churn metric was off sometimes. There were also some other things, like their messaging and recover features not working as you’d expect when it came to trials, which was my main need at the time. I ended up doing so much myself that they weren’t worth it to keep. Then I tried ChartMogul, had the same lack of visibility into my metrics, and ended up going all in and writing this [0]. [0]: https://twitter.com/ezekgabrielse/status/1345157874356465664 https://twitter.com/ezekgabrielse/status/1345157874356465664
cj 6y ago
> What is it that they have a problem with? We struggle a lot with “fake churn” in baremetrics. By that, I mean any time a customer on a credit card subscription upgrades to a “pay by invoice” tier, it shows as a churn of the credit card subscription and as new revenue for the invoice subscription. This makes churn wildly inaccurate and we spend a couple hours per month adjusting the numbers to get accurate data. FWIW profitwell suffers the same issue. We’ve looked but haven’t found a great solution for metrics when stripe isn’t the sole source of customer payments. Edit: But all things considered, we love baremetrics, just not their churn calculations (and by proxy, LTV is also broken, and any other metric that uses churn)

BummerCloud 6y ago

I'd be cautious multiplying your MRR x 12 to establish ARR. Things like seasonality or promotions could overestimate your ARR by using inflated subscription numbers. Additionally, if customers are not contractually obligated to stick around for a year, that calculation may not account for month-to-month churn.

ezekg 6y ago
I’m pretty comfortable with MRRx12 for annual run rate, which is pretty normal from what I’ve seen. But I do have other unmentioned revenue forecasting metrics based on current/past growth/churn rate, which I pay more attention to than my ARR. I don’t base a lot on this number either way.
encoderer 6y ago
ARR is always MRRx12. You're talking about a revenue model.
- jasongill 6y ago
  I think the confusion here stems from the terms "annual run rate" and "annual recurring revenue" sharing the same acronym
  - encoderer 6y ago
    I admit it’s confusing, but those are exactly the same thing. Have the same formulas. https://www-priceintelligently-com.cdn.ampproject.org/i/s/www.priceintelligently.com/hs-fs/hubfs/Calculating%20True%20MRR.jpg?width=1511&name=Calculating%20True%20MRR.jpg https://www-priceintelligently-com.cdn.ampproject.org/i/s/ww... https://www-profitwell-com.cdn.ampproject.org/i/s/www.profitwell.com/hs-fs/hubfs/blob-256.png?width=1200&name=blob-256.png https://www-profitwell-com.cdn.ampproject.org/i/s/www.profit...
- thih9 6y ago
  > ARR is always MRRx12. Are you saying that ARR is meaningless for heavily seasonal businesses?

encoderer 6y ago

This is how side projects start :)

ezekg 6y ago
Indeed. Maybe I can run with this and eventually quit my full time job. :P

skaber 6y ago

Your Ruby ActiveRecord query is so under optimized, you are loading a ton of records that instantiate objects and then apply a filter on this huge array. I'm pretty sure you could make it 100x faster.

hiharryhere 6y ago
I don't think those are ActiveRecord queries, they look like calls to the Stripe api via the Stripe gem. It's possible those filters like 'deleted?' are not able to be applied when querying the API.
- ezekg 6y ago
  Correct. Those are Stripe API calls made through their client library.
jon-wood 6y ago
I think someone else has pointed out that they're not ActiveRecord queries, but even if they were, this is a script designed to be run manually a few times a week, which takes maybe a minute. You'd need significantly more than 100x performance increase for any time at all put into optimising queries to pay for themselves.

michalbugno 6y ago

Perhaps I am a bit biased because I create analytics software fos SaaS businesses. This is a nice writeup, could be an intro of tech people to business metrics you mention. I also code Ruby so +1 :-) I would be cautious about rolling your own implementation like this though. 1. This is not your core business. Eventually you want to start segmenting things by verticals, geography etc. You will want to visualise them. It takes time, which you want to spend building your product. 2. There are many important details in how you calculate certain metrics. Either you dive into the topic and again waste time, or you start having metrics which are custom, not really comparable with benchmarks or just plain wrong. Maybe you just spent 1hr on this and that’s ok. Just make sure you spend your time on your product domain, because that’s what’s probably gonna make you happier (unless you pivot into saas metrics world!)