5 ms·

This is a great article for people who are in the situation where they have to make decisions about billing and don't have much experience (and as a handy refer

by jchw 3y ago

This is a great article for people who are in the situation where they have to make decisions about billing and don't have much experience (and as a handy reference for those who do, but want some reminders, too.) However, I will admit, I had a hearty chuckle at this line: > "why can’t we just dump a file of what we need to bill on S3, and have a CRON job pick it up and collect payment?" Under no circumstances does my engineer brain think this is a good idea. At all. But, I will dump one aspect of my engineer-brain thoughts: My favorite "billing architecture" decision is to try to decouple billing as much as possible from credit in a system. For example, if you have a subscription system where the user pays ahead for a given billing period, I prefer to have the entitlement itself just store the expiration date and the details about what entitlements the subscription grants during the time period it is active. The billing system can store the subscription and sync back to the entitlement as-needed. This makes both manual billing by human operators (not to mention debugging and patching around momentary issues) and something like a Stripe integration very easy. You should, of course, be very careful to leave it open for extension in the future, but this seems to be a very nice decision that doesn't, in itself, limit you too much. Obviously, this is not my original idea, but it's still something I've grown to like a lot, especially after having tried other things less successfully.

brettermeier 3y ago
Is that legal in germany? I don't know but I guess it isn't.
- timvdalen 3y ago
  Which part?
  - marcosdumay 3y ago
    Probably sending your customers information in plain text to Amazon. But there is probably a way to do it, and even if there isn't, there certainly is some equivalently insane design that is legal.
    - shepherdjerred 3y ago
      S3 is encrypted in transit (like most data nowadays) S3 can also be encrypted at rest. The encryption can be performed client-side or server-side [0]. Nowadays, AWS performs server-side encryption by default, unless you disable it. [0]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingE...
      - aftbit 3y ago
        Thereby proving that checkbox security is pointless. What threat models does AWS server-side at-rest encryption protect against? Someone breaking into an S3 data center and walking off with a box of hard drives? AWS reselling obsolete hardware on eBay and forgetting to wipe a disk? And yet what does it not protect against at all? A misconfigured load balancer that can access 169.254.169.254 and dump your AWS access key straight to an attacker, who can then use it to ask AWS to politely decrypt all of your data. What does it really not protect again? The US.gov issuing a National Security Letter to Bezos threatening jail time if they don't decrypt and hand over the data. To reach the spirit of the "data must be encrypted at rest" laws, I believe you should encrypt the data using your own keys which are maintained in an HSM that AWS cannot access. This means that you cannot run your entire stack on one cloud, which would be a huge driver of cost and complexity.
        shepherdjerred 3y ago
        > What threat models does AWS server-side at-rest encryption protect against? A lot of threats. A rogue employee sucking up data. Malware on AWS servers. Not likely, but certainly possible. Server-side encryption is useful, but if you want to be _certain_ that your data is safe, then client-side encryption does exactly what you're asking for. Client-side encryption can use a local key, or a key in KMS. https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/concepts.html https://docs.aws.amazon.com/amazon-s3-encryption-client/late... Is there a reason you have such a negative take?
        3y ago
q3k 3y ago
> Under no circumstances does my engineer brain think this is a good idea. At all. Yeah, if that's the model engineer in the "don't build your own billing system" mantra then the article had the opposite effect on me. Now I want to build such a system because this made me feel like I'm significantly more qualified than I previously thought.
- arnon 3y ago
  Don't do it
  - q3k 3y ago
    Now I'll also do it out of spite! :)
    - arnon 3y ago
      Don't say I didn't warn you
- jerf 3y ago
  Naw, you still don't want to, at least not in a context in which you're billing enough to be a real workout, because the non-engineering aspects will still get you. Like this: https://news.ycombinator.com/item?id=37557989 https://news.ycombinator.com/item?id=37557989 which, in short, refers to the fact that nobody in the company considers billing a system to be designed in the first place, so it gets approached with a truncated Waterfall that doesn't even have a requirements analysis phase, so you'll have no support whatsoever for your attempt to bring higher quality to billing. Also, with all due respect to those working on billing I have seen that it is generally handed to engineers who are, shall we say, adequate for the task but no more, and unless you're 100% greenfielding you're going to be dealing with their stuff no matter what. From another cynical point of view, it is a situation where you can fail miserably and bring great pain to the company that is traced directly to you, but there is no success; a functioning billing system is just what is expected. It's all downside and no upside for you personally. I have a billing system (a component of the larger whole) and I'm pleased to be able to hand it off next month. I've not enjoyed my stay there. Though it was miles better than when I brushed the data lake/data analysis world. If I had to maintain this system for another few years I could and I wouldn't scream if I had to do it again someday. Dealing with the data analysis world beyond just throwing them some data is a "never again".
rapind 3y ago
> and something like a Stripe integration very easy What I did (and would recommend others do), was a deep dive into Stripes API when designing my schema. It doesn't even cost anything to create an account and run some test charges, refunds, disputes, balance changes, etc. Of course I was modelling a more complicated setup (charges on behalf of customers), but still, useful stuff IMO. If you have access to an accountant / bookkeeper (relative?) that can help too.
spiffytech 3y ago
Another great article from Arnon on doing exactly that: https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...
ylere 3y ago
My experience has been that there are typically two types of engineers: those who have worked on billing systems before and those who haven't. I think a lot of the issues arise from the difference between payments and billing [0]. When just starting out and signing up your first customers, you primarily care about collecting a few (recurring) payments - and it's really easy to set that up with Stripe (or even just manually invoicing your first customers). However, over time, more billing requirements gradually sneak in, such as more complex entitlements, multiple plans, grandfathering, and eventually enterprise/high touch sales customers (where the money is!) who need custom billing cycles, terms, and entitlement provisioning. Since billing is never a technical focus, numerous additions and small hacks accumulate over time, taking engineering resources away from the actual product. Eventually, this turns into an unmanageable mess that can significantly slow down the sales process or limit what you can sell. The complexity of billing is riddled with hidden pitfalls and edge cases, and it has become even more complex now that most plans include many different limits and usage-based components and that most SaaS companies sell globally. Many later-stage companies have teams of 15+ engineers working solely on billing. I fully agree with the author that, unless it's at the core of your product, no organization should build a billing system themselves (Disclaimer: I'm the CTO of Wingback, a SaaS Billing Platform). [0] https://www.wingback.com/blog/saas-payment-vs-saas-billing https://www.wingback.com/blog/saas-payment-vs-saas-billing
kkajla 3y ago
> I prefer to have the entitlement itself just store the expiration date and the details about what entitlements the subscription grants during the time period it is active. The billing system can store the subscription and sync back to the entitlement as-needed. This makes both manual billing by human operators (not to mention debugging and patching around momentary issues) and something like a Stripe integration very easy. Having worked with SaaS companies using our (Warrant) solution for customer entitlements [0] for the past couple years, this is the approach we arrived at as well (e.g customer stores entitlements in our system and checks against them when needed, adding rules/entitlements as subscriptions are updated/deleted with their payment provider). It makes it easier for companies to work with any (or multiple) payment providers, and there's a clear separation of concerns. Someone shared another blog post by OP about separating your billing and entitlement systems [1] below, but I'll share it here since it's more relevant within the context of this comment thread. I think the ideal entitlement system is (1) dynamic (i.e. rules stored in a database), (2) can handle one off scenarios (for enterprise customers, etc.), and also has a policy layer built on top (so it supports almost any scenario a developer can throw at it -- e.g. pro plan supports <= 5 seats, growth plan supports X feature up to N times per day, etc). I think it's also a huge benefit to have a UI where non-technical folks can make changes for customers without needing to involve engineering (which was always a drag on engineering in my prior roles as an engineer). [0] https://warrant.dev/use-cases/pricing-tiers-and-entitlements/ https://warrant.dev/use-cases/pricing-tiers-and-entitlements... [1] https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...
doctor_eval 3y ago
The way that we ended up with is similar: - your billing is just a mechanism to compute the invoices. - you accumulate billing using a ledger of invoices sent and payments made (and adjustments, etc) - ie, receivables - you have a set of entitlements, yes - but in the middle you have a policy mechanism that determines when the entitlements should/should not be applied. The trick is that entitlements are not just about billing. The policy mechanism is the “glue” between entitlements and everything else. It means you can give an entitlement to the boss of your biggest customer without it necessarily being linked to billing in any way.
code_biologist 3y ago
Speaking of decoupling entitlements, this article is a classic on that theme: https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...

by jchw 3y ago

brettermeier 3y ago

Is that legal in germany? I don't know but I guess it isn't.

timvdalen 3y ago
Which part?
- marcosdumay 3y ago
  Probably sending your customers information in plain text to Amazon. But there is probably a way to do it, and even if there isn't, there certainly is some equivalently insane design that is legal.
  - shepherdjerred 3y ago
    S3 is encrypted in transit (like most data nowadays) S3 can also be encrypted at rest. The encryption can be performed client-side or server-side [0]. Nowadays, AWS performs server-side encryption by default, unless you disable it. [0]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingE...
    - aftbit 3y ago
      Thereby proving that checkbox security is pointless. What threat models does AWS server-side at-rest encryption protect against? Someone breaking into an S3 data center and walking off with a box of hard drives? AWS reselling obsolete hardware on eBay and forgetting to wipe a disk? And yet what does it not protect against at all? A misconfigured load balancer that can access 169.254.169.254 and dump your AWS access key straight to an attacker, who can then use it to ask AWS to politely decrypt all of your data. What does it really not protect again? The US.gov issuing a National Security Letter to Bezos threatening jail time if they don't decrypt and hand over the data. To reach the spirit of the "data must be encrypted at rest" laws, I believe you should encrypt the data using your own keys which are maintained in an HSM that AWS cannot access. This means that you cannot run your entire stack on one cloud, which would be a huge driver of cost and complexity.
      - shepherdjerred 3y ago
        > What threat models does AWS server-side at-rest encryption protect against? A lot of threats. A rogue employee sucking up data. Malware on AWS servers. Not likely, but certainly possible. Server-side encryption is useful, but if you want to be _certain_ that your data is safe, then client-side encryption does exactly what you're asking for. Client-side encryption can use a local key, or a key in KMS. https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/concepts.html https://docs.aws.amazon.com/amazon-s3-encryption-client/late... Is there a reason you have such a negative take?
        3y ago

q3k 3y ago

> Under no circumstances does my engineer brain think this is a good idea. At all. Yeah, if that's the model engineer in the "don't build your own billing system" mantra then the article had the opposite effect on me. Now I want to build such a system because this made me feel like I'm significantly more qualified than I previously thought.

arnon 3y ago
Don't do it
- q3k 3y ago
  Now I'll also do it out of spite! :)
  - arnon 3y ago
    Don't say I didn't warn you
jerf 3y ago
Naw, you still don't want to, at least not in a context in which you're billing enough to be a real workout, because the non-engineering aspects will still get you. Like this: https://news.ycombinator.com/item?id=37557989 https://news.ycombinator.com/item?id=37557989 which, in short, refers to the fact that nobody in the company considers billing a system to be designed in the first place, so it gets approached with a truncated Waterfall that doesn't even have a requirements analysis phase, so you'll have no support whatsoever for your attempt to bring higher quality to billing. Also, with all due respect to those working on billing I have seen that it is generally handed to engineers who are, shall we say, adequate for the task but no more, and unless you're 100% greenfielding you're going to be dealing with their stuff no matter what. From another cynical point of view, it is a situation where you can fail miserably and bring great pain to the company that is traced directly to you, but there is no success; a functioning billing system is just what is expected. It's all downside and no upside for you personally. I have a billing system (a component of the larger whole) and I'm pleased to be able to hand it off next month. I've not enjoyed my stay there. Though it was miles better than when I brushed the data lake/data analysis world. If I had to maintain this system for another few years I could and I wouldn't scream if I had to do it again someday. Dealing with the data analysis world beyond just throwing them some data is a "never again".

rapind 3y ago

> and something like a Stripe integration very easy What I did (and would recommend others do), was a deep dive into Stripes API when designing my schema. It doesn't even cost anything to create an account and run some test charges, refunds, disputes, balance changes, etc. Of course I was modelling a more complicated setup (charges on behalf of customers), but still, useful stuff IMO. If you have access to an accountant / bookkeeper (relative?) that can help too.

spiffytech 3y ago

Another great article from Arnon on doing exactly that: https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...

ylere 3y ago

My experience has been that there are typically two types of engineers: those who have worked on billing systems before and those who haven't. I think a lot of the issues arise from the difference between payments and billing [0]. When just starting out and signing up your first customers, you primarily care about collecting a few (recurring) payments - and it's really easy to set that up with Stripe (or even just manually invoicing your first customers). However, over time, more billing requirements gradually sneak in, such as more complex entitlements, multiple plans, grandfathering, and eventually enterprise/high touch sales customers (where the money is!) who need custom billing cycles, terms, and entitlement provisioning. Since billing is never a technical focus, numerous additions and small hacks accumulate over time, taking engineering resources away from the actual product. Eventually, this turns into an unmanageable mess that can significantly slow down the sales process or limit what you can sell. The complexity of billing is riddled with hidden pitfalls and edge cases, and it has become even more complex now that most plans include many different limits and usage-based components and that most SaaS companies sell globally. Many later-stage companies have teams of 15+ engineers working solely on billing. I fully agree with the author that, unless it's at the core of your product, no organization should build a billing system themselves (Disclaimer: I'm the CTO of Wingback, a SaaS Billing Platform). [0] https://www.wingback.com/blog/saas-payment-vs-saas-billing https://www.wingback.com/blog/saas-payment-vs-saas-billing

kkajla 3y ago

> I prefer to have the entitlement itself just store the expiration date and the details about what entitlements the subscription grants during the time period it is active. The billing system can store the subscription and sync back to the entitlement as-needed. This makes both manual billing by human operators (not to mention debugging and patching around momentary issues) and something like a Stripe integration very easy. Having worked with SaaS companies using our (Warrant) solution for customer entitlements [0] for the past couple years, this is the approach we arrived at as well (e.g customer stores entitlements in our system and checks against them when needed, adding rules/entitlements as subscriptions are updated/deleted with their payment provider). It makes it easier for companies to work with any (or multiple) payment providers, and there's a clear separation of concerns. Someone shared another blog post by OP about separating your billing and entitlement systems [1] below, but I'll share it here since it's more relevant within the context of this comment thread. I think the ideal entitlement system is (1) dynamic (i.e. rules stored in a database), (2) can handle one off scenarios (for enterprise customers, etc.), and also has a policy layer built on top (so it supports almost any scenario a developer can throw at it -- e.g. pro plan supports <= 5 seats, growth plan supports X feature up to N times per day, etc). I think it's also a huge benefit to have a UI where non-technical folks can make changes for customers without needing to involve engineering (which was always a drag on engineering in my prior roles as an engineer). [0] https://warrant.dev/use-cases/pricing-tiers-and-entitlements/ https://warrant.dev/use-cases/pricing-tiers-and-entitlements... [1] https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...

doctor_eval 3y ago

The way that we ended up with is similar: - your billing is just a mechanism to compute the invoices. - you accumulate billing using a ledger of invoices sent and payments made (and adjustments, etc) - ie, receivables - you have a set of entitlements, yes - but in the middle you have a policy mechanism that determines when the entitlements should/should not be applied. The trick is that entitlements are not just about billing. The policy mechanism is the “glue” between entitlements and everything else. It means you can give an entitlement to the boss of your biggest customer without it necessarily being linked to billing in any way.

code_biologist 3y ago

Speaking of decoupling entitlements, this article is a classic on that theme: https://arnon.dk/why-you-should-separate-your-billing-from-entitlement/ https://arnon.dk/why-you-should-separate-your-billing-from-e...