4 ms·
I’m in the very unusual situation of being tasked to set up a self-sufficient, local development team for a significant national enterprise in a developing coun
by doctor_eval 3y ago
I’m in the very unusual situation of being tasked to set up a self-sufficient, local development team for a significant national enterprise in a developing country. We don’t have AWS, Google or any other cloud service here, so getting something running locally, that they can deploy code to, is part of my job. I also want to ensure that my local team is learning about modern engineering environments. And there is a large mix of unrelated applications to build, so a monolith of some sort is out of the question; there will be a mix of applications and languages and different reliability requirements.
In a nutshell, I’m looking for a general way to provide compute and storage to future, modern, applications and engineers, while at the same time training them to manage this themselves. It’s a medium-long term thing. The scale is already there - one of our goals is to replace an application with millions of existing users.
Importantly, the company wants us to be self sufficient. So a RedHat contract to manage an OpenShift cluster won’t fly (although maybe openshift itself will?)
For the specific goals that we have, the broad features of Kubernetes fit the bill - in terms of our ability to launch a set of containers or features into a cluster, run CICD, run tests, provide storage, host long- and short lived applications, etc. But I’m worried about the complexity and durability of such a complex system in our environment - in the medium term, they need to be able to do this without me, that’s the whole point. This article hasn’t helped me feel better about k8s!
I personally avoided using k8s until the managed flavours came about, and I’m really concerned about the complexity of deploying this, but I think some kind of cluster management system is critical; I don’t want us to go back to manually installing software on individual machines (using either packaging or just plain docker). I want there to be a bunch of resources that we can consume or grow as we become more proficient.
I’ve previously used Nomad in production, which was much simpler than K8s, and I was wondering if this or something else might be a better choice? How hard is k8s to set up today? What is the risk of the kind of failures these guys hit, today?
Are there any other environments where I can manage a set of applications on a cluster of say 10 compute VMs? Any other suggestions?
Without knowing a lot about their systems, I suspect something like Oxide might be the best bet for us - but I doubt we have the budget for a machine like that. But any other thoughts or ideas would be welcome.
- geodel 3y agoWell Amazon CEO himself said, there is no shortcut to experience. I am sure gaining experience in developing infrastructure solution will give you respectable return in long term. Of course Cloud vendors will be happy to sell turnkey solutions to you though.
- doctor_eval 3y agoYeah - I do agree, over the long term, that deploying this internally would be the best outcome for them, and would give them some great skills. But this article did put the Fear in me a little. One of the main drivers of the project is to reduce our reliance on software vendors and move to open source solutions, so a hybrid/on-prem cloud vendor is probably not on the cards for us either.
- steveklabnik 3y ago(I work at Oxide) > I doubt we have the budget for a machine like that. Before even thinking about budget, > for a significant national enterprise in a developing country. I suspect we just aren't ready to sell in your country, whatever it is, for very normal "gotta get the product certified through safety regulations" kinds of reasons. We will get there eventually. buuuuut also, > Are there any other environments where I can manage a set of applications on a cluster of say 10 compute VMs? Any other suggestions? Oxide would give you those VMs, but if you want orchestration with them, you'd be running kubes or whatever else, yourself, on top of it. So I don't think our current product would give you exactly what you want anyway, or at least, you'd be in the same spot you are now with regards to the orchestration layer.
- doctor_eval 3y agoHey Steve, thank you for this comment, I did wonder if Oxide systems did container orchestration; now I know :) Totally get it re certification etc. There is probably some kind of bilateral standards arrangement with one of the neighbouring countries but I agree with your general thrust - we’re a long way from the point where that’s actually a consideration.