5 ms·

Lessons Learned from Payments Startups

72 points by ben_s 3y ago

tommiegannert 3y ago
This is a nicely varied list of lessons. > Often, I will prefer the database to logging Just don't start writing exception logs to the database, and not tell anyone that you've created a flood gate that will make you run out of production data storage at the worst possible time.
zeroxfe 3y ago
I wrote a [1] post-mortem when I shut down my payment startup a few years ago. The vast majority of our challenges were non-technical: navigating regulation, high legal costs, complex insurance requirements, etc. on an uneven playing field with competitors with much deeper pockets than even a well-funded startup. [1] https://www.linkedin.com/pulse/quid-sudo-shutdown-h-now-mohit-muthanna-cheppudira/ https://www.linkedin.com/pulse/quid-sudo-shutdown-h-now-mohi...
- algo_trader 3y ago
  Nice write up, and i hope you dont regret the journey I am surprised you didnt mention fraud - is it because micro-payments are not "fraud worthy" or because it is a manageable problem ?
  - zeroxfe 3y ago
    We designed our systems to be resistant to certain types of fraud, but never got successful enough to be a target. :-/ But also, yes, micropayments are far less "fraud worthy" from an attackers perspective.
- potamic 3y ago
  The payments platform you talk about is a piece of tech every financial company ends up building. What did you do with all that code after shutting down? Did you consider either selling or open sourcing it?
umeshunni 3y ago
These are great lessons from an engineering perspective. From a product perspective, I've learned 3 hard lessons: 1. Fraudsters will find your product and abuse it before real users do. If you see a sudden spike in usage, it's probably fraud. 2. Real users are very emotional when it comes to their money, even if it's a "low volume transaction" like $20. Anything you build that introduces friction between them and their money will get your support team yelled at. 3. Regulators and banks are suspicious of you, for lessons they have learned from 100+ years of balancing lessons 1 & 2 above. Balancing 1, 2 & 3 is an art and the difference between building a successful product and failure. All the cute UI and funnel analysis you do breaks when it hits fraud, user emotions and regulation.
- user_7832 3y ago
  I'm curious, would you say using a third party to handle payments (stripe/paypal/shopify etc) would help avoid some of these issues? I'm not sure I understand point 2, especially if there's a "regular" checkout page by one of these companies.
  - PeterisP 3y ago
    It would help avoid these issues, but if you're a payments startup then you probably want to actually keep some control over your core business instead of just being a reseller intermediary for an actual payments company which holds your customer's money, has all the leverage over what features and policies "your" product will have and gets most of the profit.
  - monknomo 3y ago
    In some ways it makes it worse, because your users will not understand that you aren't screwing up their transaction, it's Paypal. They'll just yell at you
1905 3y ago
I believe a big part of the Musks twitter acquisition was to transition into payments, like WeChat in China. How do you guys think that will play out?
jaredliu233 3y ago
[dead]
threeseed 3y ago
None of this is unique to payments or the financial sector. And it is bizarre to me that you could run a payments system without high-availability or immutability at its core. Also putting logs in a database alongside your core data is such a bad idea. There are plenty of solid, proven systems for log management that allow you to correlate against other databases. And it avoids the issue of you wasting limited write I/O on a non-priority task.
- rrr_oh_man 3y ago
  > And it avoids the issue of you wasting limited write I/O on a non-priority task. Is it really an issue, though, in most cases?
  - threeseed 3y ago
    For a database. Absolutely.
    - anonzzzies 3y ago
      In how many instances though? This sounds like premature optimising. We are not all Stripe; many companies making millions, but not billions, would have an easier life without doing more work or paying a fortune to niche saas companies for these type of things.
    - rrr_oh_man 3y ago
      Food for thought: https://news.ycombinator.com/item?id=39276069 https://news.ycombinator.com/item?id=39276069
- pgr0ss 3y ago
  Hi, I’m the author of the blog post. > And it is bizarre to me that you could run a payments system without high-availability or immutability at its core. I’m not sure what you’re referring to here as my post didn’t really go into these topics. > Also putting logs in a database alongside your core data is such a bad idea. I’m not suggesting putting all logs into the database. I’m suggesting that for a certain type of info (that is often logged), I find it more useful to put it in the database instead.

72 points by ben_s 3y ago

This is a nicely varied list of lessons. > Often, I will prefer the database to logging Just don't start writing exception logs to the database, and not tell anyone that you've created a flood gate that will make you run out of production data storage at the worst possible time.

zeroxfe 3y ago

I wrote a [1] post-mortem when I shut down my payment startup a few years ago. The vast majority of our challenges were non-technical: navigating regulation, high legal costs, complex insurance requirements, etc. on an uneven playing field with competitors with much deeper pockets than even a well-funded startup. [1] https://www.linkedin.com/pulse/quid-sudo-shutdown-h-now-mohit-muthanna-cheppudira/ https://www.linkedin.com/pulse/quid-sudo-shutdown-h-now-mohi...

algo_trader 3y ago
Nice write up, and i hope you dont regret the journey I am surprised you didnt mention fraud - is it because micro-payments are not "fraud worthy" or because it is a manageable problem ?
- zeroxfe 3y ago
  We designed our systems to be resistant to certain types of fraud, but never got successful enough to be a target. :-/ But also, yes, micropayments are far less "fraud worthy" from an attackers perspective.
potamic 3y ago
The payments platform you talk about is a piece of tech every financial company ends up building. What did you do with all that code after shutting down? Did you consider either selling or open sourcing it?

umeshunni 3y ago

These are great lessons from an engineering perspective. From a product perspective, I've learned 3 hard lessons: 1. Fraudsters will find your product and abuse it before real users do. If you see a sudden spike in usage, it's probably fraud. 2. Real users are very emotional when it comes to their money, even if it's a "low volume transaction" like $20. Anything you build that introduces friction between them and their money will get your support team yelled at. 3. Regulators and banks are suspicious of you, for lessons they have learned from 100+ years of balancing lessons 1 & 2 above. Balancing 1, 2 & 3 is an art and the difference between building a successful product and failure. All the cute UI and funnel analysis you do breaks when it hits fraud, user emotions and regulation.

user_7832 3y ago
I'm curious, would you say using a third party to handle payments (stripe/paypal/shopify etc) would help avoid some of these issues? I'm not sure I understand point 2, especially if there's a "regular" checkout page by one of these companies.
- PeterisP 3y ago
  It would help avoid these issues, but if you're a payments startup then you probably want to actually keep some control over your core business instead of just being a reseller intermediary for an actual payments company which holds your customer's money, has all the leverage over what features and policies "your" product will have and gets most of the profit.
- monknomo 3y ago
  In some ways it makes it worse, because your users will not understand that you aren't screwing up their transaction, it's Paypal. They'll just yell at you

1905 3y ago

I believe a big part of the Musks twitter acquisition was to transition into payments, like WeChat in China. How do you guys think that will play out?

jaredliu233 3y ago

[dead]

threeseed 3y ago

None of this is unique to payments or the financial sector. And it is bizarre to me that you could run a payments system without high-availability or immutability at its core. Also putting logs in a database alongside your core data is such a bad idea. There are plenty of solid, proven systems for log management that allow you to correlate against other databases. And it avoids the issue of you wasting limited write I/O on a non-priority task.

rrr_oh_man 3y ago
> And it avoids the issue of you wasting limited write I/O on a non-priority task. Is it really an issue, though, in most cases?
- threeseed 3y ago
  For a database. Absolutely.
  - anonzzzies 3y ago
    In how many instances though? This sounds like premature optimising. We are not all Stripe; many companies making millions, but not billions, would have an easier life without doing more work or paying a fortune to niche saas companies for these type of things.
  - rrr_oh_man 3y ago
    Food for thought: https://news.ycombinator.com/item?id=39276069 https://news.ycombinator.com/item?id=39276069
pgr0ss 3y ago
Hi, I’m the author of the blog post. > And it is bizarre to me that you could run a payments system without high-availability or immutability at its core. I’m not sure what you’re referring to here as my post didn’t really go into these topics. > Also putting logs in a database alongside your core data is such a bad idea. I’m not suggesting putting all logs into the database. I’m suggesting that for a certain type of info (that is often logged), I find it more useful to put it in the database instead.