29 ms·

Pains of building your own billing system

• Vinaydatta2020 3y ago
  (Co-Founder of Justpaid.io here)Thank you, Arnon, for sharing this article. I have firsthand experience working with a homegrown billing system for a fairly large EHR company that processes millions of dollars in claims daily. It was a nightmare for billing team and account managers to determine the invoicing amounts for hospitals each month. The system was complex, with a flat platform fee, discounts for the first year, usage based on the number and value of claims processed, and different items billed at different times. The billing system did not scale, and every month I ran reports without confidence in the accuracy of the billed amounts. Currently, I am addressing this problem by developing a fully customizable billing system tailored for SaaS companies. One unique aspect of our billing solution is the use of signed contract documents as a foundation. This eliminates the need to decide on a plan for new customers or be limited to specific pricing structures due to system constraints. By leveraging detailed contract information in natural language, we employ LLM to make custom exceptions when generating invoices. For instance, if you wish to offer a discount to an existing customer for November and December due to low activity, traditional subscription-based billing methods lack flexibility. Our dynamic contract-based billing allows for such exceptions without the need to cancel and recreate subscriptions. In addition, our collections workflow will complete the billing workflow. We leverage agent type LLM for collection workflow, which drafts and sends emails impersonating a founder and to notify the founder/account representative about customer queries while collecting the invoiced amount. I know this is a new and untested approach, but we aim to simplify billing for founders and we value any constructive feedback or inquiries.
• jchw 3y ago
  This is a great article for people who are in the situation where they have to make decisions about billing and don't have much experience (and as a handy reference for those who do, but want some reminders, too.) However, I will admit, I had a hearty chuckle at this line: > "why can’t we just dump a file of what we need to bill on S3, and have a CRON job pick it up and collect payment?" Under no circumstances does my engineer brain think this is a good idea. At all. But, I will dump one aspect of my engineer-brain thoughts: My favorite "billing architecture" decision is to try to decouple billing as much as possible from credit in a system. For example, if you have a subscription system where the user pays ahead for a given billing period, I prefer to have the entitlement itself just store the expiration date and the details about what entitlements the subscription grants during the time period it is active. The billing system can store the subscription and sync back to the entitlement as-needed. This makes both manual billing by human operators (not to mention debugging and patching around momentary issues) and something like a Stripe integration very easy. You should, of course, be very careful to leave it open for extension in the future, but this seems to be a very nice decision that doesn't, in itself, limit you too much. Obviously, this is not my original idea, but it's still something I've grown to like a lot, especially after having tried other things less successfully.
  • brettermeier 3y ago
    Is that legal in germany? I don't know but I guess it isn't.
    • timvdalen 3y ago
      Which part?
      • marcosdumay 3y ago
        Probably sending your customers information in plain text to Amazon. But there is probably a way to do it, and even if there isn't, there certainly is some equivalently insane design that is legal.
        • shepherdjerred 3y ago
          S3 is encrypted in transit (like most data nowadays) S3 can also be encrypted at rest. The encryption can be performed client-side or server-side [0]. Nowadays, AWS performs server-side encryption by default, unless you disable it. [0]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingE...
          • aftbit 3y ago
            Thereby proving that checkbox security is pointless. What threat models does AWS server-side at-rest encryption protect against? Someone breaking into an S3 data center and walking off with a box of hard drives? AWS reselling obsolete hardware on eBay and forgetting to wipe a disk? And yet what does it not protect against at all? A misconfigured load balancer that can access 169.254.169.254 and dump your AWS access key straight to an attacker, who can then use it to ask AWS to politely decrypt all of your data. What does it really not protect again? The US.gov issuing a National Security Letter to Bezos threatening jail time if they don't decrypt and hand over the data. To reach the spirit of the "data must be encrypted at rest" laws, I believe you should encrypt the data using your own keys which are maintained in an HSM that AWS cannot access. This means that you cannot run your entire stack on one cloud, which would be a huge driver of cost and complexity.
• chasd00 3y ago
  I worked on a billing and AR system for a small pharmacy chain years ago. Our company would get pretty large random checks from ins carriers totally out of the blue. As in, we had billed them nothing and they would send us $150k checks in snail mail. They wouldn’t realize their mistake and come asking for it back for months. We had a special account named “magic money” for it. The billing world is crazy town.
  • dns_snek 3y ago
    Fascinating, is there ever a point where this "magic money" legally becomes your property, after a certain amount of time?
    • kasey_junk 3y ago
      Depends on the jurisdiction but typically no.
    • scott_w 3y ago
      Short answer "it depends." Long answer: It depends on your jurisdiction, the entities involved, the amount of money involved, and probably more. However, it's not possible to know ahead of time, you'd need to go to court and argue why this entity is being unreasonable in demanding the money be returned.
    • indymike 3y ago
      In the US, you are supposed to turn it over to the state, and the government holds it for an extremely long time until it's owner claims it.
      • alsetmusic 3y ago
        Also, you're probably not going to win a dispute with a / the state. The USA government stole my money once (bank transfer). It was a rent payment to a landlord with a Muslim-sounding name. I contested it by sending in their form with the case number that they'd provided me. They responded saying they had no case matching my info. I gave up before investing more time because I thought it was unlikely to end well for the amount of effort I'd have to invest. I started directly depositing money orders into his account. No problems after that. The USA government profited one month of my rental rate. I nearly got evicted when my rent wasn't delivered. No explanation was ever given to me.
• mattbee 3y ago
  I don't know about this advice, you ain't gonna need half the features it talks about, and you absolutely can build it out gradually as your business grows. You do need to understand the concepts of invoices, credit, tax periods, pro-rata billing changes and so on... but all of that knowledge can be used to make an informed decision on build vs buy, rather than an automatic reason to outsource. The only external API you need for software-as-a-service is a credit card processor, two if you're fancy. Sure after the first year you will probably have a bunch of manual work to do and your accountants will tell you the dumb stuff you've done, and you'll learn a lot about accounting :) (I would still shop around to start a new business today, but with the confidence that building isn't very scary)
  • rtpg 3y ago
    The core thing with B2B SAAS is you make money off of higher touch sales, that lead to requirements. The objective as someone working on this kind of billing is simple: offer your sales teams the right options so they can close deals. This involves collaborating with your sales team to figure out what is in the “easy to implement x good enough to close deals” bucket I can’t stess this enough. Your sales team is closing sales! Stop whining about your perfect billing system. Yes it is shocking but sometimes you have to apply elbow grease to problems involving business relationships You should try to do things in straightforward ways when possible but really you should have a system that is legible to other teams in your company, and one that is easily to manually tweak even with automation running around it. This is like 80% of why Stripe feels so good to use. Manual tweaks are a fact of life, even if for some businesses they’re rare
  • scott_w 3y ago
    As someone who's been involved in both building and integrating billing systems, I can say you speak from a position of blissful ignorance. Building the system always looks easy, and on day 1, it really is. It also guarantees you'll spend many hours with your Finance Director explaining why the reports you send them are complete garbage; many hours with your Support team explaining why your invoicing failed, why you charged incorrect subscription prices, any many more fun edge cases you never knew existed. Next up, regulations change that you need to adapt to, or maybe your chosen gateway doesn't support a growing region. And before you say "just build it better," remember: that's also time. Time not spent on your product, improving your pricing model (oh, you need to build that yourself, too, of course), or any number of things you want to do that actually grow your business instead of standing still.
• stevev 3y ago
  For such systems that are commonly used by many, it’s best to open source it. Don’t reinvent the wheel.
  • tuyiown 3y ago
    We have hard time having proper implementations of interesting problems, how can we expect an workable open source billing system implementation ? With the proper engineering tradeoffs and flexibility needed to comply with all businesses and legal specificity ?
  • TomNguy 3y ago
    Have you tried Lago (open source billing)? This is precisely what they do, their HN launch is here: https://news.ycombinator.com/item?id=34773442 https://news.ycombinator.com/item?id=34773442 They listed the billing engineering nightmares too: https://news.ycombinator.com/item?id=31424450 https://news.ycombinator.com/item?id=31424450
    • Axsuul 3y ago
      We looked into them. Unfortunately they’re very expensive.
      • AnhTho_FR 3y ago
        Lago co-founder here, although we have an Enterprise plan (which, by definition, is for Enterprises), we invest considerable resources in the open-source and free product, used by a lot of early stage companies/bootstrapped projects, let me know if we can be helpful.
        • Axsuul 3y ago
          I’m referring to the Premium Plan. We are not an enterprise but need a lot of the features offered at that tier, stuff that I feel most businesses need like credit notes and invoices sent by email.
    • 3y ago
• 6510 3y ago
  What really impressed me was the need to be able to run the entire Rube Goldberg machine backwards. You really want to have "the order" as static as possible, for "the invoice" there is even legal obligation. Then it goes something like... I bought 5 and got a discount but I want only 3 and ohh there is a typo in my name and one in my address - sorry about that. No problem, you credit the invoice, make a new one for the same amount with the name and address correction, then you wait for the items to be returned and checked and credit it again, make a new invoice again. The return shipment stays in limbo for a while, they order 5 more, one arrives broken, they want a refund rather than a replacement. With 2 delivery dates the system doesn't apply the 5+ discount.... and all of a sudden you have tons of transactions and "paperwork" for what you initially imagined to be 2 simple orders. They also forgot their password so they made a new account using a different email address. Looking over the logs a year later it is hard to figure out why they got a discount for just 3 units.
• bearjaws 3y ago
  Worked for a large travel website that rolled its own billing... Absolute nightmare when we blew up, had to hire 6 people to beef up the billing side of the website and it definitely cost us way more than those 6 people due to mistakes. I ended up working on parts of it for our customer service portal, lots of band-aides applied in a rush as we scaled.
  • tuyiown 3y ago
    How would you balance failure attribution of that bad scenario between "own billing" and "problematic implementation" ?
    • bearjaws 3y ago
      I am not going to go into every nuanced issue, but to give context rounding errors alone costed us hundreds of thousands of dollars in customer support and eng time. I am talking Office Space style fractions of pennies type problems. Mostly due to how long it was an issue before it was caught. Sure we could have probably botched implementation of some vendor, but we made every dumb mistake you could make and instead of doing it slowly we did it at scale.
      • 6510 3y ago
        I wrote my own arithmetic functions just to deal with the paranoia.
      • madsr 3y ago
        Speaking of fractions of pennies: I've configured my AWS Cloud invoice to be in Euros, and every month the Net amount plus the VAT amount is always 1 cent off the Total fee :P
      • zie 3y ago
        Don't worry, every jurisdiction and bank will do the rounding of pennies differently, so you know, that's always a fun time.
    • scott_w 3y ago
• nickjj 3y ago
  One thing this article doesn't mention are affiliate sales which depending on how far you want to go can be a decent amount of work. This is the system that lets other people get commission based payouts for sales within your platform. You need to track affiliate codes back to sales and a user, handle sending payouts to an affiliate at their configured payment provider and if you want to go all out then handle metrics around visitors and present a UI to the affiliate so they can see their conversion rates and payout history. Fortunately most of that can be built incrementally. As long as you associate a unique code to a user and wire up associating that to a sale with a specific commission % amount everything else can be done manually or skipped. For example I pay affiliates out once per month where I goto Zelle or PayPal and send out the payments. It takes less than 10 minutes. There is no front end for tracking conversions and it's also never been a cause of someone saying they don't want to be an affiliate because of that.
• etewiah 3y ago
  There are actually quite a lot of solutions out there that fly under the radar but do a great job. Billsby for example has slowly crept up to becoming a significant player in the field by starting with a focus on one segment: recurrent billing. I can see a world where there could be multiple unicorns just in the billing sector of fintec.
  • splitstud 3y ago
    [dead]
• whydoineedthis 3y ago
  I know the founders of bunny.com (which tries to solve this very problem). They sold a different b2b startup after 10 years and billing was such a pain they created a whole nother startup just to try and solve it. Lol.
  • baq 3y ago
    Don’t know about them but they surely aren’t the only ones. It’s a proper mess.
• talkingtab 3y ago
  The 14 pains of crossing the street and why you want to pay me to read my book. Many people think it is a good idea to close your eyes and just start walking, resulting in untimely injuries or even death. 1. Countries and even cities have different rules for crossing the street. 2. You may not know this yet but you need to look both ways. 3. Many people do not realize they are color blind, resulting in death and injuries because they cannot tell red from green. 4. You may look both ways, but do you look up as well? In cities people can drop things from windows, in the country birds may fly over head. More and more space junk is falling to earth. 5. Thing may come at you from multiple directions. 6. etc. Not saying this article is that bad, but I would appreciate an article framed as a how to. I am much more likely to read articles that are not framed negatively.
  • brainzap 3y ago
    streets are made up, a virtual seperation, just walk
  • baq 3y ago
    I work in a billing team now. Haven't been for majority of my career, though. The article is spot on. It's the ideal 'how hard can it be?' joke project, except it's all true. The list of examples is too long to count, but if you approach the problem incorrectly you're soon left with the question 'how much should the customer pay' and then 'why didn't we charge this amount but something else' and when this happens too often finance comes in and asks what did you do because the reports which run the company (including whether to lay off people or not) are not trustworthy.
    • talkingtab 3y ago
      I do not disagree. Like many problems, the simple cases are okay, but it is what you don't know that can lead you down the wrong path. But this is true of almost everything I can think of in software products. Databases? OMG don't get me started. After some short amount of experience everyone will flinch if someone says "how hard can it be". The issue I have is that while an article that gives people a heads up of what lies ahead would be very useful and helpful this was not that. Instead it was framed as "don't do it". Just not my cup of tea.
• spintin 3y ago
  You should never build your own billing system because you do not want to do the global VAT work. itch.io is a good alternative if you sell digital assets and even services as you can verify payments with mail.
  • surfingdino 3y ago
    I really want one for physical goods. The global sales/vat network is a nightmare to navigate for a small business.
    • komali2 3y ago
      I encountered this recently when I set up dolibarr for my restaurant in Taiwan. We have VAT in Taiwan but I feel like nobody actually knows how VAT is supposed to work. Sometimes a shop will charge us VAT, sometimes not, but what never happens is seeing VAT on your bill as a consumer, which means that every B2C in the country, other than Uber Eats, is eating the VAT. I assume. Or everyone's committing tax fraud all the way up the chain. Honestly both are just as likely. If dolibarr didn't have auto handling for it I would have encountered it for the first time in my own program and instantly thrown my hands in the air and just done paper ledgers. I can't even imagine coding around it.
      • baq 3y ago
        The point of VAT (…in EU) in B2C is that the B eats the VAT. You’ll see the VAT-less subtotal on receipts - but prices on shelves are strictly including VAT.
  • bluGill 3y ago
    You build things yourself because A: you really like doing that (that is a hobby thing, not something to run a business on); B: you are really good at it and so have made it a core business item you use to make money (you are selling your system to others) C: you can't trust anyone else to do it. In finance C is a big one! A bad billing system can kill you, and so despite the complexity it might be worth doing just because you now are in control. If you choose to buy a billing system anyway, you still need to understand how it works in enough detail to audit it. A corrupt billing system can hide how it is siphoning your money away and the numbers seem to balance and so you don't realize it is at fault. A bad billing system will not apply some tax it should and when you fail an audit the government will demand you pay - the unexpected bill will kill you. There are many variations on both of the above. You need to audit your systems to ensure they don't happen to you. Despite the above I generally would suggest you buy a billing system not build your own. However that doesn't absolve you from understanding billing in enough detail to audit yours on a high level. You also need to select one that your independent auditors (which might be too expensive to have but you really want) can audit in whatever detail needed.
• huqedato 3y ago
  AFAIK, in EU countries you cannot build/run "your own" billing system. Unless you have a special type of company/enterprise that is accredited and registered specifically for this purpose. This is officially named "Payment Processor".
  • brettermeier 3y ago
    Really? Do you have a link to laws for that?
    • huqedato 3y ago
      It depends on each EU country. For Italy: https://www.bancaditalia.it/compiti/sispaga-mercati/normativa-sorveglianza/Elenco.pdf https://www.bancaditalia.it/compiti/sispaga-mercati/normativ...
  • gizmo 3y ago
    Why do you believe that? You can just write software to bill your customers. There are no special laws prohibiting automation.
    • huqedato 3y ago
      At least in EU, nope! Because it's not just "automation". It's more about dealing with other people's money, accounting, taxation, sensitive data collection etc.,etc. You should be first registered as a "payment processor" type.
      • gizmo 3y ago
        You can charge customers by direct debit (bank to bank) with an API. You can create your own pdf invoices and email them. You can charge customers with paypal or use credit cards or any of the EU specific online payment methods. There is no EU directive or law prohibiting this. You don't need any kind of "special data collection" authorization. Most SMBs just print their own invoices with MS Word or some generic software package.
      • 3y ago
• atonse 3y ago
  I don't know what this person is talking about, our billing was brain dead simple, it's only got like 20 lines of Stripe Billing code. :-) But in all seriousness, we looked at Stripe, thought about "aw man is it worth spending the .5% extra" and then spent about 30 seconds thinking about how much time it would take to roll our own, and it was an absolute no brainer. (replace Stripe with any of the other similar competitors, not trying to be too biased, but stripe is kind of the mind-share default for SaaS startups, aren't they?)
  • Rafsark 3y ago
    (Lago co-founder here.) I guess it depends on how complex your pricing and monetization process are (is it the same pricing for everyone or custom ; is it a simple subscription: or a is there revenue share, transactional pricing, usage-based, tiers ; is it self-serve or sales-led with a quoting system, do you have grandfathered plans, do you need to use other payment processors than Stripe - "Stripe Billing" is only usable with "Stripe Payments"). In some cases, it's "brain dead simple", in a lot of cases it ends up "being much more complex than it seemed". Related threads: - "Why Stripe doesn't use Stripe Billing": https://news.ycombinator.com/item?id=33191307 https://news.ycombinator.com/item?id=33191307 - "Stripe's real pricing: a primer" : https://news.ycombinator.com/item?id=33920019 https://news.ycombinator.com/item?id=33920019
    • atonse 3y ago
      I wouldn’t actually expect stripe to use stripe billing. Just like I don’t expect AWS to be built on top of AWS. The point is that I’d happily pay someone else to sweat the details than my own team. Our volume is so small that it is a no brainer, especially if you want to sell in multiple countries.
• godzillabrennus 3y ago
  I helped build a billing system for a smallish family business. It had cost overruns and was way behind schedule. A year or so after planned release and after bringing in some more senior leadership it was a huge success for them. Integrated their customers database with their new SaaS products and facilitated very unique workflows that a legacy business needed to get through a digital transformation. Very unique circumstances but huge undertaking.
• deleted 3y ago
  [deleted]
• epberry 3y ago
  On the other side of this, understanding usage based consumption billing as a customer is very tough. First got exposure to this working on https://ec2instances.info https://ec2instances.info.
  • baq 3y ago
    I postulate this has been the true cause of the cloud revolution: accurate usage billing. Everyone can rent out a server. Not everyone can rent out 16.83% of a server and bill it by the second and by the byte.
• idlewords 3y ago
  Summary: just use picodollars!
• jdwyah 3y ago
  What are y'all using for entitlements? Do you use your feature flag system? A different saas like Stigg? Or a separate internal system? I was just coming to hn this morning because I wrote about using FF for entitlements: https://prefab.cloud/blog/modeling-product-entitlements-with-feature-flags/ https://prefab.cloud/blog/modeling-product-entitlements-with... I took some inspiration from another of Arnon's posts about SKU format for the post. FF don't seem like the perfect place for entitlements, but in my experience they're often the best tool at hand to deal with the challenges. I'd love to hear alternative opinions.
  • heipei 3y ago
    We just use a combination of numerical limits (e.g. API calls per day) and product flags as an array of tags (features:["module1", "module2"]). These limits and flags are attached set on "plans" which can be attached to accounts. Plans can be combined, in which case we'll take the larger value for each numerical value and combine the tags using union. Additionally one can override / add to any of these values on a per-account basis, so if your customer needs PlanX but a custom API quota then you just override that single value directly on their account. Call me old-school, but I don't get why something like this should be outsourced to a third party.
    • jdwyah 3y ago
      yeah, the overhead of another system for people to understand is non-trivial. So your accounts can have N plans. Do each of those plans map directly to a sku that they are paying for?
    • baq 3y ago
      If you already have a process for repackaging entitlements in plans and addons whenever some bright mind in marketing has an idea, the it’s completely fine.
  • LeFever 3y ago
    I'm actually working on a solution to plan entitlements among other similar functionality right now. While you can get a basic solution set up with feature flags, we've found that the organization and evolution of billing/pricing-related entitlements (E.g. plans, editions, etc.), especially over time, is increasingly complex with lots of requirements in the peripheral. Think things like varying combinations of feature, seats, and usage-based/metered strategies, team subscriptions, plan migrations, one-off enterprise plans, multi-subscription customers (E.g. promotion periods and layered subscriptions), usage aggregation, automated upselling, etc. As you show in your blog (Nice post btw!), while you can have a flag with a numerical value representing a limit, the infrastructure for tracking usage is left to the business to implement. Imagine instead that you emit usage of that lever to an entitlement service and entitlements based on that usage are updated in real-time, even across teams. Also imagine that you have other entitlements that may be dependent on that entitlement that update as well. In addition, as limits are approached or crossed, you can choose to have soft enforcement (I.e. let them continue, but notify sales to reach out) or hard enforcement and display a prompt to upgrade. In the spirit of OP's link, we work alongside existing billing solutions rather than try to reinvent the wheel there. We're bootstrapped via a previous successful exit and working with early customers, so if anyone's interested in chatting, even to just geek out on this topic, please reach out: trent at planship.io.
• d_burfoot 3y ago
  I don't buy the overall structure of this argument: system X is very very complex, so you should never try to do it yourself, instead use an off-the-shelf solution for X. Perhaps the general case of X is enormously tricky and complex, but in my use case I only need to handle a specific subset of the complexity. Therefore I can build my own solution that only handles the complexity I need, and it will be much simpler than off-the-shelf tools. I absolutely adopt this stance for X=datetime. My approach to datetime requires two function calls to be provided by the library: convert an epoch time to an ISO formatted time string in a particular TZ, and the inverse. I never touch any other library code; I do all other time manipulation in my own code, in terms of those two functions.
  • makeitdouble 3y ago
    > in my use case I only need to handle a specific subset of the complexity Thing is billing money leaves very little room for error and is regulated. From the start you'll need to understand all the dos and don't of personal info handling, the cycles and lifetime of the different billing options, cashback thresholds, support idempotency, sane DB transaction management with models that are adapted to the task, invoicing and exposing that to your client etc. It's not everything at once, but at least half of it just for your first workable implementation.
• msluyter 3y ago
  I worked on a fairly complex invoicing system and couple of things not (directly) mentioned: * Forward billing vs billing in arrears. We had to treat different customers differently. * Tiered billing -- e.g., if a customer used > X amount of services, they got a discount. * Special rules around when billing starts for new customers (e.g., no bill for the first month). There are probably more I've forgotten. We billed by data usage, mostly, and I used to say that their bill was the integral of the customer's usage graph over a month -- which was true -- but that explanation didn't gain much traction with the accounting folks.
• wdb 3y ago
  Can't be worse then needing to change a SAP system to work with the business processes than instead what is the cheaper and quicker way of changing the processes to better match SAP. Sometimes you had to reimplement whole systems that SAP comes out of the box with. Thanks not to be named fashion brand.
• paulhart 3y ago
  If you're building a product, is the billing component a part of your USP? Really? Odds are that it's not, and therefore you should farm out that work to someone who _does_ make it their job. There are reasons why companies implement software from Microsoft, Oracle, and SAP, including that it's better to reduce costs on things that don't differentiate you in your market (and it's nice to have someone to "pointedly talk at" when things go wrong).
• 6510 3y ago
  I haven't really explored it but one idea I had (for small volume) was to make everything manual data entry then put suggestions and warnings with the form fields. Even if you perfect automation there will be exceptions and mistakes made. By doing it manually everything also gets reviewed.
• forwardemail 3y ago
  [dead]
• zie 3y ago
  My rule of thumb: Always ask about rounding. If they don't have a sane answer, it's basically guaranteed that their system is broken in countless ways. Pretty much the only sane answer: Until you realize that every jurisdiction, bank and organization you interact with financially can and might round pennies differently, you don't understand financial systems enough to implement one. i.e. rounding is configurable or you are doing it wrong.
  • deleted 3y ago
    [deleted]
  • gizmo 3y ago
    Why does rounding matter? It's not important from an accounting perspective. And your billing system doesn't need to angrily send invoices when a customer owes a penny.
    • zie 3y ago
      For accounting, if your auditors and accountants don't care, I'm jealous. I've never met one that doesn't care.
      • gizmo 3y ago
        Tax laws were written for cash businesses where the cash in the register never 100% matches the sum of receipts. What matters in my experience is that your books are balanced, but there is no such thing as "perfect accounting." In the real world inventory goes missing for reasons unknown (broken, stolen, never delivered properly) and as long as you've got an entry in the books that plugs the hole it's fine in practice. Accountants and auditors don't do forensic investigations, not even for publicly listed businesses. I have never met an accountant who cared even slightly even about very significant tax compliance issues. Businesses don't want to change their sloppy ways either. They want to make money first and let the accountants clean up the mess later (impossible).
        • zie 3y ago
• Aspos 3y ago
  Each individual problem listed isn't that hard on its own. Some are even trivial. However there are so so many constraints one has to keep track of so system complexity quickly snowballs into something unmanageable. Each cog is simple, but the whole clock is insanely complex. And by the way, the clock can't stop and has to be running 24/7/365 while you keep changing it. Your code will quickly turn into a tangled up mess of stinky petrified spaghetti, so you will end up re-building your entire system from scratch every few years. And with each rebuild you will keep having the same trivial problems you already solved again and again and again. Billing problems in general are a subset of problems one has in banking because banking system basically = billing + interest calculation + payments + accounting + multiple equally complex things and it has to work in real-time in a super-regulated environment while working in perfect sync with other systems you don't control. If you do everything just right and manage to keep the can rolling down the street for sometime, there will be a moment when regulation changes or your business changes and your architects suddenly resign and you will end up with a mess which requires more engineering dollars than off-the-shelf system would. Don't build your business on such foundation. It is XXI century already, no need to re-invent stuff which is readily available for peanuts. Yet, every new neobank comes up with their own shiny corebanking ledger because they can't be bothered to look into somebody else's petrified spaghetti.
• WirelessGigabit 3y ago
  > why can’t we just dump a file of what we need to bill on S3, and have a CRON job pick it up and collect payment? Sounds like an engineer with not enough experience. We all did thought like this once. This is a solved problem. Don't roll your own queue / job system where you inevitably run into transactional problems. Use something like Airflow. And mentioning S3 in this sentence has no value. S3 is merely a way to store the data temporarily. But using S3 immediately makes us not consider (more suitable) alternatives.
• wolfspider 3y ago
  I built one for the government keeping track of licensed professionals and receiving their payments for things reported in the field (mobile first web app). It collected over a quarter million dollars in 30-40 dollar payments with different tiers, refunds, overrides, and even penalties for non-payment all while being PCI compliant. One thing that helped a lot- create an auth system that lets the admin impersonate the customer to walk them through it. It’s a tough paradigm to start with but pays off immensely. Another was generating excel files and reports on demand from any view of the data in the app. One of the developers on the project implemented a simple state machine for payment histories and stored it in a number of tables with FK constraints. Do not do this! That means in the future your app will need to deserialize every customer’s history and after a few years the app will grind to a halt. This was the one issue with the app looking back. A state machine looks like a good fit for billing but if my future self could go back in time and warn everyone it would be with this one common problem or I wouldn’t even make this comment in the first place. If you do consider using a state machine just create thousands of customers with long detailed histories up front and if you can load them all up quickly then that is a good sign. Your billing system will only perform as well as you can transform and index data from customer history in bulk.
• zengid 3y ago
  I'm on the billing team and we just shipped our new platform send me your regards!
• jongjong 3y ago
  I think a lot of the problems which companies experience are all related to them not being able to find good software engineers. The amount of money I've seen startups forking out annually for various billing and platform subscriptions would have been enough for me to build them a better system from scratch in under 6 months. The amount of money that companies waste on rents is ridiculous and it's sad because that money could have gone to skilled engineers instead of hotshot entrepreneurs. For my own SaaS startup, I built the billing system from scratch, it measures every millisecond of CPU time and every operation used by each account and adds them to the current active bill. At the end of the month (or whenever), I manually trigger a process to close off active bills and the system will automatically display that one as due in the UI and will open up a new one and new usage will be recorded against that new active one. The trick is to use a function which automatically figures out what bill to use based on a specific flag so you don't need to worry about the implementation details of which bill to record usage against. This function should handle all possible situations; including the few milliseconds of delay which can exist between the old bill being closed and the new bill being opened. That's where idempotence can be useful as mentioned in the article; that way if the system fails to record usage against a new bill because (for example) that bill was already just created concurrently by a different process then it will retry again later and record the new usage stats against that existing bill. The trick I use is to have deterministic IDs for bills that are based on the nearest whole unit of time in UTC (rounded up or down); the amount of rounding I use allows me to control the allowable delay between concurrent processes and determines the amount of pending usage data which is kept in-memory within each process before it is flushed to its bill in the database. If the chosen unit of time is a 'whole minute' then it means that it's not possible to generate two bills within 30 seconds of each other by concurrent processes. This is greater than my database update timeout so it effectively guarantees that two processes cannot accidentally create two bills for the same period. If a specific process took longer than 30 seconds to update their usage info, that process would reconnect and either realize that an active bill has now already been created by a different process or, if not, it would try to create it again with a new ID corresponding to the new 'whole minute'. I have full flexibility to change my billing to any time interval I want. It doesn't even have to be the same for all users. Anyway, as you can see, it's very simple.
• coolThingsFirst 3y ago
  Seems simple enough for a startup idea but prolly has been done thousands of times
• zackmorris 3y ago
  This is why I'm so disappointed with Stripe. I found the documentation both overly verbose and lacking, so it took me longer than it should have to understand basics like subscription item ids and the 50-100 id types which people have collected into lists. Then there are simple bugs like how the go live button only shows in test mode, with no simple way to copy live mode data back into test environment(s), so developers have to manually export/import stuff that business people set up in their attempt to save time. The most basic features are missing, like a way to copy any object as json from the browser dashboard, forcing us to look down in the logs/events to copy the latest version, often with wildly inconsistent/unpredictable formats. And Stripe does little to support actual real-world use-cases, for example: subscription events come through webhooks ok, but subscriptions have a state like "active" or "incomplete" (meaning that payment hasn't gone through yet), causing subscriptions to become stateful. Meaning that instead of a user being subscribed (yes/no), the app has to consider additional criteria and edge cases around missing or lapsed payments. And there doesn't appear to be a synchronization mechanism for eventualities like a backend server being down for a day, causing events to be missed. Stripe does a best-effort resend some low number of times, but then the events are lost. It's up to the developer to diff the backend's database with the Stripe subscription list and then sync individual subscriptions manually. These are all just exactly the kinds of bugs/features I predicted would be there before I used it, which is why they felt like such a slap in the face. I suspect that there are conceptual shortcomings throughout nearly every service provided by Stripe, and I'd be over 50% confident that any ones I predict here would be found upon first use. These are artifacts of favoring agile over waterfall for formal engineering challenges. Basically what I'm saying is that I consider the Stripe API to be what an MVP payment processing system might have looked like back in the 1990s under older paradigms like SOAP/XML. But nobody has created a wrapper for Stripe yet that works how people expect, more like Venmo/PayPal that maps customer use-cases to CRUD operations. I thought that maybe Laravel Cashier would do some of the heavy lifting, but it appears to be just a thin wrapper over Stripe, with its own set of oversights. I know that everyone uses Stripe and apparently loves it, and I applaud its efforts and accomplishments around reducing the friction of payment processing. But I can't help but feel that there is an element of having to drink the kool-aid here. I would still recommend Stripe though, so this rant is directed more at its developers who may not be aware of these issues. Stripe would do well to perform some new user testing like Apple used to do when designing its human interface guidelines, to enumerate the pitfall(s) in each step of Stripe onboarding.
• ivanmontillam 3y ago
  I'd argue that the pains of building a billing system are not the right approach to the topic. If building your own billing system is a path so much sought after, well, let that be. Billing systems are of high complexity; I recognise that. However, if Chargebee, Solvimon, Stripe, Recurly, Orb, Metronome, Lago, Togai or anyone else has that body of knowledge, we could instead collect that knowledge in one place. Indeed, there's no better approach than the one that serves you. If you're a subscription-based SaaS, you have specific solutions for your business. If you're a usage-based API, you have specific solutions to the billing. But we could have all that knowledge, approaches, paradigms, programming patterns, better and best practices in one place, instead of discouraging the practice. There are also edge cases where a company is not U.S.-based or European, and a billing solution like Stripe wouldn't work, e.g. your company is based in Venezuela, and you can't have a Stripe account. What do you do in that case? You must forcefully build your own billing solution and connect it to the local payment gateways with their arcane SOAP-XML APIs. -- On a separate note, "building your own billing system" reminds me of the topic of "rolling your own SIEM" with the typical Elastic + Grafana setup. I don't recommend it, but I understand why it's such a hot path for an IT Security department to do it.
  • gopher_space 3y ago
    The article considers billing systems in an interesting way. There are topics presented as problems that I'd hand off to accountants or other specialists we're probably already employing. > Billing systems are of high complexity; I recognise that. [...] They're less complex than whatever your developers are working on. The article tries to paint hard legal requirements as a difficulty, but in practice this means the specs are easy to find and well documented. Parts of the process do change frequently, but those parts are well labeled and well explained. > Indeed, there's no better approach than the one that serves you. If you're a subscription-based SaaS, you have specific solutions for your business. If you're a usage-based API, you have specific solutions to the billing. I mean, will your customers allow you to shift the burden of responsibility? Is your income important enough to verify? Can you afford the haircut?
    • doctor_eval 3y ago
• tombert 3y ago
  My first job after dropping out of college the first time was working for a Tae Kwon Do management firm, writing software to help people run martial arts studios. The job had me doing ColdFusion and Adobe Flash [1]. Most of my job involved working on their marketing pages, but at some point they had me work on the credit card processing platform, and I've sort drawn a soft line in the sand that I won't do that ever again. Part of it was just that the code was really messy (it was ColdFusion after all), but a lot of it boiled down to having to deal with the million edge case conditions required to achieve PCI Compliance. Somehow, that company had managed to get a PCI Compliance label, and I have no idea how, because the code was held together with duct tape and prayers; there were hundreds of nested if statements, and if's nested inside else nested inside other ifs. I'm not saying that it's unnecessarily complicated, but I know I'm not smart enough to deal with it. [1] I'd like to point out, this was 2012; it was considered kind of outdated even at the time!
• kyrra 3y ago
  2 problems not talked about here that are things I've had to deal with: 1) Month/Quarter close. While this post talks about the account ledger, when you start dealing with a public company that has to report numbers, you have hard-cutoffs and have to make sure everything goes smoothly for month or quarter close. 2) Cash-in-transit accounting: this post assumes your payment processor is 100% correct and nothing goes wrong. When you get big enough, you need to be able to match any money that lands in a bank account with a given invoice/billing entry. You need to be able to detect any invoices that do not have a paired bank statement line. And as others have called out, the reverse can happen where you get paid for something that may not have an invoice item associated with it. Being able to deal with credits on a bank account not associated with a billing invoice can be equally important. Maybe you can say that these are all accounting departments problem, but they are tightly coupled and the 2 teams need to work together that there are no discrepancies between their books.
  • dmoy 3y ago
    > Month/Quarter close. While this post talks about the account ledger, when you start dealing with a public company Anyone even accounting adjacent to a big public company probably just had a mental brain shudder by mentioning this Whether it be the actual accountant having to stay up to midnight repeatedly because of close, or someone on tech/business side who needed something from accounting but they just dropped off the face of the planet for N time period because of close, or a family member, or whatever.
  • gizmo 3y ago
    If you don’t do (2) from the very beginning you have no hope of getting your taxes correct. Your internal accounting and your bank account(s) will slowly diverge and the accountants won’t know where to start looking for the problem.
• _pdp_ 3y ago
  The general rule is not to build anything that is not your core expertise.
• welder 3y ago
  I built my own hybrid billing system and yes it was painful, but it opened the door to more payment methods and allowed prorating across the different payment methods seamlessly. I think it increased my revenue, but I can't know for sure because maybe everyone would have just used credit cards if that was the only supported payment method.
  • aChattuio 3y ago
    What exactly did you build? PCI dss conform cc handling? What about liability in case of fraud?
    • welder 3y ago
      Just integrated with payment APIs Stripe, Braintree & Coinbase. I let them handle recurring transactions while my system handles setting up those recurring transactions including pricing, prorating, discounts, and trials.
• rahimnathwani 3y ago
  "Let your ERP handle your RevRec/Accounting (or use what’s built in with something else)." With that approach, how do you: 1. Make your ERP recognize revenue correctly? An ERP system isn't magically going to know how your revenue should be amortized based on certain events, or what should happen when a partially-amortized order is later upgraded/downgraded to a different package/plan. 2. Ensure that your refund/upgrade/downgrade calculations are consistent between product systems and the ERP system. Even if you emit all the necessary events (and each contains all the necessary data), there has to be some system that's programmed (or, at least, configured) to apply your revenue recognition rules to the event stream. Given how much difficulty people have even describing how things should work, I don't think you can just abdicate the revenue recognition to another system and just assume whoever configures/programs that system will do it right. Even if you can assume that, it's equivalent to saying "it's complicated so you shouldn't do it yourself; have someone else do it".
• Havoc 3y ago
  Host own email and host/run own billing is definitely on my list of things I'd rather not do
• sbrossie 3y ago
  (killbill.io co-founder here). Building a billing system is indeed quite complex, and for a variety of reasons: For once thing, most departments within the company will be impacted (finance, accounting, product, engineering, legal, ...), so the cognitive load on the team building the solution is quite high, and requirements are usually poorly defined. Then, as mentioned in the comments above, there are tons of complicated use cases and edge cases (time zones, pro-rations, entitlement v.s. billing, in-arrear/in-advance models, usage-based, credits, coupons, tax integration, refunds, chargebacks, and on and on ...). Another aspect is that once you have built the billing system, this is just one part of a bigger quote-to-cash picture, and the solution needs to fit within the rest of such platform, so it needs to be made flexible enough to integrate with such systems. I would typically not recommend building your own unless you have acquired lots of knowledge over the years, and have a motivated team in place to execute on it.
• gmfawcett 3y ago
  Whenever someone mentions building a billing system, I feel compelled to share Mark Dominus' delightful article, "Moonpig: a billing system that doesn't suck": > Sometimes I see other people fuck up a project over and over, and I say “I could do that better”, and then I get a chance to try, and I discover it was a lot harder than I thought, I realize that those people who tried before are not as stupid as as I believed. That did not happen this time. Moonpig is a really good billing system. It is not that hard to get right. Those other guys really were as stupid as I thought they were. https://blog.plover.com/prog/Moonpig.html https://blog.plover.com/prog/Moonpig.html
• bombi 3y ago
  I'm about to implement https://www.getlago.com/ https://www.getlago.com/ on my SAAS. Spoke to the team, great bunch
• llamaLord 3y ago
  I'm sure the person that invented mantras like "seperation of concerns" and "single responsibility principle" did so based on their trauma of building a billing system. The biggest mistake I see companies making is not properly appreciating the fact that there is in fact 4-5 entirely independent business processes going on inside that monolithic concept they refer to as a "billing system", and they REALLY need to be kept seperate. IMO the fundamental seperation that is absolutely critical to make is distinguishing between your "entitlements system" that tracks what SKUs you offer and which customers have what SKUs (and in what quantity), the "accounting system" that takes input from the entitlements system and tracks the over-time net balanced owed by the customer, and finally the actual "billing system" which periodically takes the balance from the the Accounting system, zeroes the account back to nil, and generates an actual point in time "bill" for the customer to pay. So many systems don't properly respect these boundaries and suffer the unbelievably painful consequences.
  • llamaLord 3y ago
    Oh, and for the love of all things holy, NEVER do ANYTHING in real-time... Schedules are your friend... Every system driven action in a billing system should be run off a configurable schedule, not done in response to actions (except updating customer entitlements)... You'll thank me later when you encounter dunning, invoicing terms, etc. P.s But you'll hate me when you encounter pro-rating...
• estebarb 3y ago
  What about using merchant of record solutions that pays taxes instead of you? It is a good solution for a self bootstrap startup, or it would be better to do it yourself?