5 ms·

Where do I go for my 3-5 node cluster which I want to schedule containers to, run dynamic workloads etc? How do I build a container based DAG in a small cluste

Where do I go for my 3-5 node cluster which I want to schedule containers to, run dynamic workloads etc? How do I build a container based DAG in a small cluster today without k8s? Solutions that are not k8s tend to be single programming language/sdk based or not easier to set up. I get the feeling the alternatives are dying off unnaturally fast.

drbojingle 4y ago
What exactly are you making that requires the use of multiple languages?
- discreteevent 4y ago
  It would seem that the requirement for multiple languages often has nothing to do with making something or the customer requirements. It's often a developer requirement. Developers want to work in their favorite language. In the 90s you just wrote in whatever the company or ecosystem had mandated. Now that developers are in high demand they can specify what they want to use. It started with books like "Beyond java" (and java was beyond C++) and disparaging articles by the likes of Paul Graham. They made some good points but if you have ever worked on a single system with multiple languages and are honest about it you would have to say that it wastes an enormous amount of time and generates unnecessary complexity. When a developer has to make a microservice just to write part of the system in another language you have wonder if they have the ability to evaluate technical trade-offs. Just pick one language and get on with it.
  - drbojingle 4y ago
    Kinda what I expect as well. I could understand it if a company was bringing in whole teams to scale up development of a large project quickly. Even then support would could be a nightmare. Doing it just because sounds painful.
- guywhocodes 4y ago
  My problem needing multiple machines is less weird than needing more than python for my dag?
  - drbojingle 4y ago
    No, I honestly just don't know what you're talking about lol. What exactly are you building?
    - guywhocodes 4y ago
      I'm building several things, some of them need puppeteer with plugins so have to use a lot of node.js there. Which I have no interest in otherwise. Some of it is built in higher performance languages. Some of it is off the shelf solutions And most of all by having every node in the dag a container I don't have versioning issues. Yes everything can be wrapped in python, doesn't make it a good idea however.
josephcsible 4y ago
Docker Swarm?
- guywhocodes 4y ago
  Ideally it would be a hub and spoke model. I don't know how much swarm hammers etcd but k8s in my cluster produces >1kIOps on my SSDs doing nothing at all. Just health checks and cascading effects

drbojingle 4y ago

What exactly are you making that requires the use of multiple languages?

discreteevent 4y ago
It would seem that the requirement for multiple languages often has nothing to do with making something or the customer requirements. It's often a developer requirement. Developers want to work in their favorite language. In the 90s you just wrote in whatever the company or ecosystem had mandated. Now that developers are in high demand they can specify what they want to use. It started with books like "Beyond java" (and java was beyond C++) and disparaging articles by the likes of Paul Graham. They made some good points but if you have ever worked on a single system with multiple languages and are honest about it you would have to say that it wastes an enormous amount of time and generates unnecessary complexity. When a developer has to make a microservice just to write part of the system in another language you have wonder if they have the ability to evaluate technical trade-offs. Just pick one language and get on with it.
- drbojingle 4y ago
  Kinda what I expect as well. I could understand it if a company was bringing in whole teams to scale up development of a large project quickly. Even then support would could be a nightmare. Doing it just because sounds painful.
guywhocodes 4y ago
My problem needing multiple machines is less weird than needing more than python for my dag?
- drbojingle 4y ago
  No, I honestly just don't know what you're talking about lol. What exactly are you building?
  - guywhocodes 4y ago
    I'm building several things, some of them need puppeteer with plugins so have to use a lot of node.js there. Which I have no interest in otherwise. Some of it is built in higher performance languages. Some of it is off the shelf solutions And most of all by having every node in the dag a container I don't have versioning issues. Yes everything can be wrapped in python, doesn't make it a good idea however.

josephcsible 4y ago

Docker Swarm?

guywhocodes 4y ago
Ideally it would be a hub and spoke model. I don't know how much swarm hammers etcd but k8s in my cluster produces >1kIOps on my SSDs doing nothing at all. Just health checks and cascading effects