7 ms·

writing one application was hard so instead let's assume writing lots of tiny applications will be easier (???) but isolating them is hard so let's put them in

by zemo 6y ago

writing one application was hard so instead let's assume writing lots of tiny applications will be easier (???) but isolating them is hard so let's put them in containers but running containers is complicated so we need orchestration managers like kubernetes or something but running those things is too hard for someone whose job isn't to run kubernetes for a living so microk8s or minikube or k3s or whatever? that's the gist of the industry? that's really what we're doing these days?

manigandham 6y ago
No, that's not what we're doing at all. I agree that "microservices" is mostly useless hype but running applications reliably and efficiently is an ongoing challenge which Kubernetes solves pretty well. Of course it does shift the complexity into the underlying K8S so installing and operating it can be difficult but there are manys to avoid that as a user. Overall you gain much more in productivity and usability, which is why it's taken off so much.
- krimeo 6y ago
  > Running applications reliably and efficiently is an ongoing challenge Is it? I can take a program written for Windows 95 and run it on Windows 7 (maybe even newer) just fine and it will run reliable and efficiently and integrate better than containers. It is problem only on Linux because user space ABI keeps breaking. Notice that the containers run on the same linux kernel and not in VMs, why? Because "we do not break userspace!".
  - lifty 6y ago
    What are you talking about? The Linux ABI is very stable.
    - krimeo 6y ago
      Linux kernel ABI is stable as rock. To answer your question, I'm talking about Linux userspace ABI - you can't rely on ABI of essential libraries, openssl for example. That's why docker was born back then.
  - manigandham 6y ago
    How is that related? Environments and languages change. That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security roles, and much more. What is your replacement for all that?
    - krimeo 6y ago
      > How is that related? I thought we were talking about running applications reliably. Why is complete linux userspace bundled separately in each container? > Environments and languages change. Yes. > That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security roles, and much more. That's a lot of new requirements in addition to "running applications reliably". Most applications simply do not need that. And I believe this is the point of the original comment you replied to. - zero-downtime deployments -> not needed for most applications (for example, twitter outages are not a big deal either). Btw how do you do zero-downtime of (websocket) streams with kubernetes? ;) - load-balancing and traffic management -> in standard k8s you are pushing all traffic through one active LB (nginx) anyway => strip most of the extra layers and you dont even need that LB - health monitoring, logging etc. -> you can use an existing solution that provides only the functionality you need, most of the work will be in your app anyway (every application needs different metrics ..) The argument is that most applications do not need to scale at this level (until you need anycast DNS returning per-node IPs or at least geodns, you are not scaling that much anyway) and can be implemented in simpler manner hence easier and cheaper to maintain, audit and secure. I do not want security roles, storage volumes and config management or observability I want my application to reliably and quickly serve my customers and be easy to maintain and debug. If you want to discuss how to design architecture for scalable applications which keep all state in distributed databases but based on a platform with stable ABI that would surely be an interesting debate as well.
      - manigandham 6y ago
        It's a container and can be as thin or fat as you want with it's contents. You don't need to include linux inside if you don't want to. I've built containers with nothing more than a few native executables. It's just a packaging format, but easier to build and deploy than other formats like tarballs. If you don't need K8S then don't use it. What's the problem? Run your app on your server and ignore everything else. But most of these features have nothing to do with scale and are more about usability, reliability and consistency. Sure you can do it yourself but that's less efficient than just letting K8S do it all in one standardized way and interface. > "I want my application to reliably and quickly serve my customers and be easy to maintain and debug." That's what K8S helps with. I've spent 10 years running large distributed applications handling billions of requests per day in multiple regions. I don't care about the ABI and don't see why that's relevant, but I do know that K8S has made many things easier in actually running these apps.
merb 6y ago
everybody who doesn't writes thousands of shell scripts/ansible/puppet whatever stuff to emulate a small version of kubernetes
- raffraffraff 6y ago
  Which is exactly what I did in my previous job before Kubernetes existed. When you're cobbling things together yourself you can leave features out of the design of your deployment / orchestration system. That is a blessing and a curse. A lot of the time you just end up ducktaping that feature on later because "oh shit, it keeps breaking". Eventually your "simpler" system is just as complex a design, but the system has lots of major flaws: * It doesn't work as well * Nobody outside your company has heard of it * Most of the code is once-off hacky scripts that nobody really maintains * Breakages will occur when underlying dependencies get upgraded when you patch your OS. I'll even wager that nobody inside your company completely understand how it works. And when it breaks, you are 100% on the hook for it. There's no chance of official documentation or training, and no point asking for help on stackoverflow.
  - merb 6y ago
    well I did this as well that's why I'm pro k8s. heck setting up k8s and then deploying with that is certainly 1000 times easier than my ansible script was. worse my ansible script had a small downtime. k3s does not have that. the problem is a lot of people try to solve a problem with k8s that k8s does not solve. k8s is mostly a deployment scheduler with application configuration and an api for secrets. thats it. it comes with an api for loadbalancer and ingress but it does not have one built in, but you can schedule these things with it. so basically even your low level primitives can be scheduled with the same api.
hinkley 6y ago
I have only gotten as far as setting up K3s and am still figuring out how I want to deploy code to it. Gotta say, the docker ps output on those machines looks like line noise to me. Not all movement is progress. Young developers are incentivized to embrace new technology because it levels a playing field where time with a tool is your most important asset. That playing field is not real, but a lot of managers seem to think it is, so the strategy works. I don't know how we sell a different version of reality there, but we need to figure it out. At my first big job, the oldest developer told me shortly before he left that essentially we keep facing the same set of problems in a loop, and that if I watch for it I'll see it happening. That was in many ways a very big shoulder to stand on. That challenge, added to historical information I had learned in a class on distributed computing, changed my perception of my first loop, like I'd found a shortcut. In my second loop, I found myself having productive conversations with people on their third loop, while my coworkers were still chirping on about how it's going to be different this time. We just keep playing a game where the rules (like cost inequalities between resource classes) get tweaked every game, but quite often they revert back to the previous rules in the following game (because the people who make hardware for that resource finally figure out how to fix their bottleneck). But instead of recycling or democratizing the tech that worked last time the rules looked that way, we reinvent it badly with new names. Elixir is a rare exception in this case, which is part of what attracts me to it. It's essentially recycling 25% of Erlang and 45% of Rails and being transparent about it, creating a new recipe out of old ingredients that have worked well in the previous 4 tech cycles.
pjmlp 6y ago
You are missing the business opportunities to do conference talks, write blog posts, best practices books, and sell consultancy services on how to sort out those problems. That is why we are stuck in fashion industry nowadays.

by zemo 6y ago

manigandham 6y ago

No, that's not what we're doing at all. I agree that "microservices" is mostly useless hype but running applications reliably and efficiently is an ongoing challenge which Kubernetes solves pretty well. Of course it does shift the complexity into the underlying K8S so installing and operating it can be difficult but there are manys to avoid that as a user. Overall you gain much more in productivity and usability, which is why it's taken off so much.

krimeo 6y ago
> Running applications reliably and efficiently is an ongoing challenge Is it? I can take a program written for Windows 95 and run it on Windows 7 (maybe even newer) just fine and it will run reliable and efficiently and integrate better than containers. It is problem only on Linux because user space ABI keeps breaking. Notice that the containers run on the same linux kernel and not in VMs, why? Because "we do not break userspace!".
- lifty 6y ago
  What are you talking about? The Linux ABI is very stable.
  - krimeo 6y ago
    Linux kernel ABI is stable as rock. To answer your question, I'm talking about Linux userspace ABI - you can't rely on ABI of essential libraries, openssl for example. That's why docker was born back then.
- manigandham 6y ago
  How is that related? Environments and languages change. That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security roles, and much more. What is your replacement for all that?
  - krimeo 6y ago
    > How is that related? I thought we were talking about running applications reliably. Why is complete linux userspace bundled separately in each container? > Environments and languages change. Yes. > That's entirely different to running programs with zero-downtime deployments, load-balancing and traffic management, health monitoring, logging and observability, secret and config management, storage volumes, security roles, and much more. That's a lot of new requirements in addition to "running applications reliably". Most applications simply do not need that. And I believe this is the point of the original comment you replied to. - zero-downtime deployments -> not needed for most applications (for example, twitter outages are not a big deal either). Btw how do you do zero-downtime of (websocket) streams with kubernetes? ;) - load-balancing and traffic management -> in standard k8s you are pushing all traffic through one active LB (nginx) anyway => strip most of the extra layers and you dont even need that LB - health monitoring, logging etc. -> you can use an existing solution that provides only the functionality you need, most of the work will be in your app anyway (every application needs different metrics ..) The argument is that most applications do not need to scale at this level (until you need anycast DNS returning per-node IPs or at least geodns, you are not scaling that much anyway) and can be implemented in simpler manner hence easier and cheaper to maintain, audit and secure. I do not want security roles, storage volumes and config management or observability I want my application to reliably and quickly serve my customers and be easy to maintain and debug. If you want to discuss how to design architecture for scalable applications which keep all state in distributed databases but based on a platform with stable ABI that would surely be an interesting debate as well.
    - manigandham 6y ago
      It's a container and can be as thin or fat as you want with it's contents. You don't need to include linux inside if you don't want to. I've built containers with nothing more than a few native executables. It's just a packaging format, but easier to build and deploy than other formats like tarballs. If you don't need K8S then don't use it. What's the problem? Run your app on your server and ignore everything else. But most of these features have nothing to do with scale and are more about usability, reliability and consistency. Sure you can do it yourself but that's less efficient than just letting K8S do it all in one standardized way and interface. > "I want my application to reliably and quickly serve my customers and be easy to maintain and debug." That's what K8S helps with. I've spent 10 years running large distributed applications handling billions of requests per day in multiple regions. I don't care about the ABI and don't see why that's relevant, but I do know that K8S has made many things easier in actually running these apps.

merb 6y ago

everybody who doesn't writes thousands of shell scripts/ansible/puppet whatever stuff to emulate a small version of kubernetes

raffraffraff 6y ago
Which is exactly what I did in my previous job before Kubernetes existed. When you're cobbling things together yourself you can leave features out of the design of your deployment / orchestration system. That is a blessing and a curse. A lot of the time you just end up ducktaping that feature on later because "oh shit, it keeps breaking". Eventually your "simpler" system is just as complex a design, but the system has lots of major flaws: * It doesn't work as well * Nobody outside your company has heard of it * Most of the code is once-off hacky scripts that nobody really maintains * Breakages will occur when underlying dependencies get upgraded when you patch your OS. I'll even wager that nobody inside your company completely understand how it works. And when it breaks, you are 100% on the hook for it. There's no chance of official documentation or training, and no point asking for help on stackoverflow.
- merb 6y ago
  well I did this as well that's why I'm pro k8s. heck setting up k8s and then deploying with that is certainly 1000 times easier than my ansible script was. worse my ansible script had a small downtime. k3s does not have that. the problem is a lot of people try to solve a problem with k8s that k8s does not solve. k8s is mostly a deployment scheduler with application configuration and an api for secrets. thats it. it comes with an api for loadbalancer and ingress but it does not have one built in, but you can schedule these things with it. so basically even your low level primitives can be scheduled with the same api.

hinkley 6y ago

I have only gotten as far as setting up K3s and am still figuring out how I want to deploy code to it. Gotta say, the docker ps output on those machines looks like line noise to me. Not all movement is progress. Young developers are incentivized to embrace new technology because it levels a playing field where time with a tool is your most important asset. That playing field is not real, but a lot of managers seem to think it is, so the strategy works. I don't know how we sell a different version of reality there, but we need to figure it out. At my first big job, the oldest developer told me shortly before he left that essentially we keep facing the same set of problems in a loop, and that if I watch for it I'll see it happening. That was in many ways a very big shoulder to stand on. That challenge, added to historical information I had learned in a class on distributed computing, changed my perception of my first loop, like I'd found a shortcut. In my second loop, I found myself having productive conversations with people on their third loop, while my coworkers were still chirping on about how it's going to be different this time. We just keep playing a game where the rules (like cost inequalities between resource classes) get tweaked every game, but quite often they revert back to the previous rules in the following game (because the people who make hardware for that resource finally figure out how to fix their bottleneck). But instead of recycling or democratizing the tech that worked last time the rules looked that way, we reinvent it badly with new names. Elixir is a rare exception in this case, which is part of what attracts me to it. It's essentially recycling 25% of Erlang and 45% of Rails and being transparent about it, creating a new recipe out of old ingredients that have worked well in the previous 4 tech cycles.

pjmlp 6y ago

You are missing the business opportunities to do conference talks, write blog posts, best practices books, and sell consultancy services on how to sort out those problems. That is why we are stuck in fashion industry nowadays.