4 ms·

This post goes into a lot of concepts but doesn't really touch on anything concrete. Also the justification for the Actor model is flimsy compared to the other

This post goes into a lot of concepts but doesn't really touch on anything concrete. Also the justification for the Actor model is flimsy compared to the other concepts. I'd love to see an actual diagram of the architecture that was built, and maybe also why sharded + replicated postgres (+/- kafka maybe) wasn't enough to solve the data needs of this application. Assuming they farm out the actual processing of payments (talking to credit card companies) to stripe/paypal/braintree or whatever, all this "payment system" is doing is taking API requests to make payments, calling out to whatever processor they're using, and saving the results, and probably maintaining the state machines for every transaction (making sure that invalid state transitions don't happen).... It sounds like they broke up a monolith but it's completely unclear how they made it better.

MrBuddyCasino 9y ago
And this is why the whole thing reads like an advert towards future employers to me. Thats ok, but as an engineer I want to know about options and tradeoffs.
- hliyan 9y ago
  Curiously, I had the same thought reading this. I found the bolded/italicized sentences particularly telling. I would expect this from an undergrad, but not someone who has worked on a large production system: > Why did data durability matter when building a payments system? For many parts of the system, no data could be lost, given this being something critical, like payments. I wish the author had shared more first-hand experience, especially those little anecdotes you can never find in textbooks...
cornholio 9y ago
Contrast this article with the buzzword collider we had last week about how movio saved 50K$ by moving to a Go-microservice-InfiniDB-neural-kubernetes-something-something, that turned out to be a lack of basic knowledge on how to use databases. I found this piece interesting, it introduces some basic concepts in a structured and easy to understand manner. Not very many writers take the time to develop such articles, they are rare thus useful as reference to point coworkers and for a quick recap. The details of the precise technologies used will be irrelevant in 3 years, the fundamental concepts will not. It's not a recipe, it's a conceptual primer to build an architecture that scales.
- hardwaresofton 9y ago
  I do remember that article -- this article is certainly better, but the actor section explanation reeks of the one you buzzword collider soup you mentioned: > Why did the actor model matter when building a large payments system? We were building a system with many engineers, a lot of them haivng distributed experience. We decided to follow a standard distributed model over coming up with distributed concepts ourselves, potentially reinventing the wheel. Literally, the word "distributed" is just used repeatedly to get the point across. While I might have a bit of an axe to grind with people that just throw the actor model (or any pardigm for that matter, functional programming doesn't solve every problem, sometimes you should maybe just mutate some state and call it a day) around as a way to fix things, I am well aware that it is indeed a useful paradigm, this was a bad way to explain your choice of using it. If he built the kind of system I think he did, I'm not sure that the actor model even conveys much of a benefit, and was looking to that explanation to drop some wisdom on me. It did not. My problem is that this article is indeed too basic. The problem is that he goes to great lengths to desribe why the architecture he built has the methodologies/approaches he's laid out, but then forgets to lay out the actual architecture -- this post is all ivory tower and none of the rubber-hits-the-road that you'd expect from a good technical piece. Things fall apart when the rubber hits the road, and sharing how they fell apart enriches people who read the article. Most intermediate engineers working with distributed systems (in good engineering orgs) have already realized/run into these concepts, for those who may not have heard of these concepts, the article is very helpful, but for those who already know them, it's basically a rehashing of level 1. For example, take any single chapter of the Google SRE handbook (e.g. https://landing.google.com/sre/book/chapters/service-level-objectives.html https://landing.google.com/sre/book/chapters/service-level-o...), his post reads like the first half of a chapter, not the complete thing. Of course, the SRE handbook is a high bar to set for quality, but at least a whiff of this production-grade payments system he built would have been nice.
- trhway 9y ago
  >the buzzword collider we had last week about how movio saved 50K$ by moving to a Go-microservice-InfiniDB-neural-kubernetes-something-something, that turned out to be a lack of basic knowledge on how to use databases. even the rudimentary experience/skills with the former gets you hot $400K+ jobs while even the advanced experience/skills with the latter gets you only $200K jobs.

MrBuddyCasino 9y ago

And this is why the whole thing reads like an advert towards future employers to me. Thats ok, but as an engineer I want to know about options and tradeoffs.

hliyan 9y ago
Curiously, I had the same thought reading this. I found the bolded/italicized sentences particularly telling. I would expect this from an undergrad, but not someone who has worked on a large production system: > Why did data durability matter when building a payments system? For many parts of the system, no data could be lost, given this being something critical, like payments. I wish the author had shared more first-hand experience, especially those little anecdotes you can never find in textbooks...

cornholio 9y ago

Contrast this article with the buzzword collider we had last week about how movio saved 50K$ by moving to a Go-microservice-InfiniDB-neural-kubernetes-something-something, that turned out to be a lack of basic knowledge on how to use databases. I found this piece interesting, it introduces some basic concepts in a structured and easy to understand manner. Not very many writers take the time to develop such articles, they are rare thus useful as reference to point coworkers and for a quick recap. The details of the precise technologies used will be irrelevant in 3 years, the fundamental concepts will not. It's not a recipe, it's a conceptual primer to build an architecture that scales.

hardwaresofton 9y ago
I do remember that article -- this article is certainly better, but the actor section explanation reeks of the one you buzzword collider soup you mentioned: > Why did the actor model matter when building a large payments system? We were building a system with many engineers, a lot of them haivng distributed experience. We decided to follow a standard distributed model over coming up with distributed concepts ourselves, potentially reinventing the wheel. Literally, the word "distributed" is just used repeatedly to get the point across. While I might have a bit of an axe to grind with people that just throw the actor model (or any pardigm for that matter, functional programming doesn't solve every problem, sometimes you should maybe just mutate some state and call it a day) around as a way to fix things, I am well aware that it is indeed a useful paradigm, this was a bad way to explain your choice of using it. If he built the kind of system I think he did, I'm not sure that the actor model even conveys much of a benefit, and was looking to that explanation to drop some wisdom on me. It did not. My problem is that this article is indeed too basic. The problem is that he goes to great lengths to desribe why the architecture he built has the methodologies/approaches he's laid out, but then forgets to lay out the actual architecture -- this post is all ivory tower and none of the rubber-hits-the-road that you'd expect from a good technical piece. Things fall apart when the rubber hits the road, and sharing how they fell apart enriches people who read the article. Most intermediate engineers working with distributed systems (in good engineering orgs) have already realized/run into these concepts, for those who may not have heard of these concepts, the article is very helpful, but for those who already know them, it's basically a rehashing of level 1. For example, take any single chapter of the Google SRE handbook (e.g. https://landing.google.com/sre/book/chapters/service-level-objectives.html https://landing.google.com/sre/book/chapters/service-level-o...), his post reads like the first half of a chapter, not the complete thing. Of course, the SRE handbook is a high bar to set for quality, but at least a whiff of this production-grade payments system he built would have been nice.
trhway 9y ago
>the buzzword collider we had last week about how movio saved 50K$ by moving to a Go-microservice-InfiniDB-neural-kubernetes-something-something, that turned out to be a lack of basic knowledge on how to use databases. even the rudimentary experience/skills with the former gets you hot $400K+ jobs while even the advanced experience/skills with the latter gets you only $200K jobs.