3 ms·

I'm not surprised. I've tried to get Netflix's OSS tools like Spinnaker running and it's a total nightmare with how many interdependent services need to run. I

I'm not surprised. I've tried to get Netflix's OSS tools like Spinnaker running and it's a total nightmare with how many interdependent services need to run. It took me days to get running and was never reliable. I think they drank a little too much koolaid. Microservices + async DB updates = hell. After working on such projects, I respect the wisdom of Google making all of their data stores immediately consistent. A monolith isn't bad at all if it's organized and built to scale horizontally. Move your consistency concerns back to the database where they belong, not between services. It's a joke that monoliths are bad when you look at every popular operating system kernel. These things are multi million line binary blobs in languages that aren't friendly to mistakes and they run EVERYTHING

jedberg 8y ago
> I've tried to get Netflix's OSS tools like Spinnaker running and it's a total nightmare You may want to check out Armory Spinnaker[0]. It's a commercial version of Spinnaker that takes care of all the hard bits of setting it up. [0] https://www.armory.io https://www.armory.io Disclaimer: I'm an investor in Armory
- danielodio 8y ago
  Hey @thermodynthrway -- CEO of Armory here. You can give our enterprise distribution a try at http://go.Armory.io/install http://go.Armory.io/install We'd love to hear how it goes for you!
Redsquare 8y ago
Bit drastic considering this was their first major global outage...
TotempaaltJ 8y ago
Seeing as this is their first major outage, and Netflix-the-streaming-service has existed since 2007, I'd say don't know what you're talking about.
- jedberg 8y ago
  For the record, there was a global Netflix outage Christmas 2012. It's what motivated us to move to multiple Amazon regions.
- thermodynthrway 8y ago
  Assuming their internal projects are built similar to OSS, I ask you to try playing with them yourself and see if you come to the same opinion. There's better ways to critique an arch than just measuring outages. Netflix has built an amazing system for messing with their servers to try find failure points. And they have a bunch of cleanup jobs that run to fix consistency errors. This is just what I picked up from a past obsession with their architecture. Then I worked on a few projects built that way, and realized it was a horrible nightmare. By having a global immediately consistent datastore you can push all these concerns back to the database, and your codebase ends up far smaller. The failure modes tend to explode in complexity when you're doing rpc across services with different datastoress because you have to deal with distributed transactions yourself. Every single call needs to be able to unwind itself across all services. If you have a call that fans out to 10 other service calls, you need to be able to unwind any of them in any order. It quickly becomes untenable

jedberg 8y ago

> I've tried to get Netflix's OSS tools like Spinnaker running and it's a total nightmare You may want to check out Armory Spinnaker[0]. It's a commercial version of Spinnaker that takes care of all the hard bits of setting it up. [0] https://www.armory.io https://www.armory.io Disclaimer: I'm an investor in Armory

danielodio 8y ago
Hey @thermodynthrway -- CEO of Armory here. You can give our enterprise distribution a try at http://go.Armory.io/install http://go.Armory.io/install We'd love to hear how it goes for you!

Redsquare 8y ago

Bit drastic considering this was their first major global outage...

TotempaaltJ 8y ago

Seeing as this is their first major outage, and Netflix-the-streaming-service has existed since 2007, I'd say don't know what you're talking about.

jedberg 8y ago
For the record, there was a global Netflix outage Christmas 2012. It's what motivated us to move to multiple Amazon regions.
thermodynthrway 8y ago
Assuming their internal projects are built similar to OSS, I ask you to try playing with them yourself and see if you come to the same opinion. There's better ways to critique an arch than just measuring outages. Netflix has built an amazing system for messing with their servers to try find failure points. And they have a bunch of cleanup jobs that run to fix consistency errors. This is just what I picked up from a past obsession with their architecture. Then I worked on a few projects built that way, and realized it was a horrible nightmare. By having a global immediately consistent datastore you can push all these concerns back to the database, and your codebase ends up far smaller. The failure modes tend to explode in complexity when you're doing rpc across services with different datastoress because you have to deal with distributed transactions yourself. Every single call needs to be able to unwind itself across all services. If you have a call that fans out to 10 other service calls, you need to be able to unwind any of them in any order. It quickly becomes untenable