5 ms·
I don't disagree with you, a single server can go a really, really long way in scale before you run into problems. I know because I've done it a few times. Th
by freedomben 5y ago
I don't disagree with you, a single server can go a really, really long way in scale before you run into problems. I know because I've done it a few times.
The problem to me isn't ability to scale on one server, it's the single point of failure. My biggest site is a wordpress box with one instance on a pretty big VPS. In the last year I've had several outages big enough to require a post-mortem (not complete outages, but periods with high levels of error rates/failures), and every time it has been because of internal data center networking issues at my preferred cloud provider (and thankfully their customer service is amazing and they will tell me honestly what the problem was instead of leaving me to wonder and guess). So the main incentive for me to achieve horizontal scalability in that app is not scaling, it's high availability so I can survive temporary outages because of hardware or networking, and other stuff outside of my control.
- jimmygrapes 5y agoI don't mean to be too glib, but to me this reads like: "I am concerned about SPOF so I outsourced that." Except now you are not in control of plugging the cable back in, and pay (less?) for that convenience.
- freedomben 5y agoI'm not sure I understand your point? Even if I ran it on my own hardware, where I was responsible for plugging the cable in, it would still have the same single point of failure problem in the event that my ISP started suffering Network issues. I've never had an ISP that didn't have some sort of issues at some point. Alternatively, if you think I am blaming my cloud provider for being the single point of failure, please read again. I specifically mentioned that I want to fix up the application so that it is highly available. I entirely and completely blame myself for the current situation. I didn't build it (otherwise it would be 12 factor[1] from the start and horizontally scalable), but I maintain it now. [1]: https://www.youtube.com/watch?v=REbM4BDeua0 https://www.youtube.com/watch?v=REbM4BDeua0
- dmux 5y agoWouldn't a snapshot mechanism plus load-balancer handle this case and still keep the architecture simple?
- freedomben 5y agoThe biggest problem is that the snapshot mechanism would need to traverse multiple data centers. The current arch uses the disk for storage in several ways which aren't easy to fix, and because of the way several of the plugins were built (with lots of the nastiest custom code I've ever seen) each time I start experimenting I run into weird breakage and debugging hell. Enough so that I decided it's not worth it for HA at this point. Will be rebuilding much of it in Elixir/Phoenix though, and that will be 12 factor/HA[1] from the start. [1]: Me explaining 12 factor for anyone not familiar already (and I've actually had people who were well experienced with 12 factor tell me they got a lot out of this video): https://www.youtube.com/watch?v=REbM4BDeua0 https://www.youtube.com/watch?v=REbM4BDeua0
- rytill 5y ago> and thankfully their customer service is amazing and they will tell me honestly what the problem was instead of leaving me to wonder and guess Which cloud provider is this, if you don’t mind sharing?
- freedomben 5y agoSure! I wasn't going to say because I didn't want to people to judge them for the network issues, but it is Linode. I've become a huge fan of them and use them for almost everything now. The Newark data center has been struggling a bit with networking issues but they are working hard on the problem. I really can't say enough good things about the Linode customer service. They respond timely and with honest info, which is invaluable to me as a head of engineering trying to make important platform and infrastructure decisions. After a terrible experience with Digital Ocean, before moving over to Linode (so I still had a pretty small account, less than $10/month) I actually opened some mostly stupid support tickets just to see how they would respond. They were professional, courteous, and helpful. For the record I have no affiliation with Linode beyond being a happy customer.